TechnologyOne incident

Spatial Service Disruption for a subset of customers ANZ Region All Releases

Critical Resolved View vendor source →

TechnologyOne experienced a critical incident on May 14, 2025 affecting Spatial Cloud, lasting 7h 4m. The incident has been resolved; the full update timeline is below.

Started
May 14, 2025, 09:05 PM UTC
Resolved
May 15, 2025, 04:10 AM UTC
Duration
7h 4m
Detected by Pingoru
May 14, 2025, 09:05 PM UTC

Affected components

Spatial Cloud

Update timeline

  1. investigating May 14, 2025, 09:05 PM UTC

    We are investigating an issue impacting the Spatial service for ANZ Region / All Releases. Impact/Error/How to verify: A subset of customers in ANZ Region for All Spatial Releases are experiencing the spatial services being offline for all modules. Due to the investigation, the next update will be provided in 60 minutes, or sooner if new information becomes available.

  2. identified May 14, 2025, 10:06 PM UTC

    Our team has identified a fix which is being implemented now for the subset of impacted Spatial customers. Our investigation to date has noted that the nightly update has not been run so when the service comes back online (our focus) the data may be out of date. We anticipate the implementation of the fix to take 2 hours to complete and verify.

  3. monitoring May 14, 2025, 11:55 PM UTC

    Our team has verified the implementation of a fix is complete for the affected spatial customers and services. The overnight processing for these customers has not completed. This means that data may be out of date. We will monitor the logs for the next 2 hours to ensure no further customers are impacted.

  4. investigating May 15, 2025, 12:50 AM UTC

    Our monitoring has shown that login errors are present. Our team is troubleshooting an issue impacting Spatial Services for a subset of customers. Due to the investigation, the next update will be provided in 60 minutes, or sooner if new information becomes available.

  5. identified May 15, 2025, 01:16 AM UTC

    Our team has identified a fix which is being implemented now for the affected customers and service. We anticipate the implementation of the fix to take 60 minutes to complete and verify.

  6. monitoring May 15, 2025, 01:57 AM UTC

    Our team has verified the implementation of a fix is complete for the affected spatial customers and services. Reminder: The overnight processing for these customers has not completed. This means that data may be out of date. We will monitor the logs for the next 2 hours to ensure no further customers are impacted.

  7. resolved May 15, 2025, 04:10 AM UTC

    After 2 hours monitoring this incident is now resolved. We will undertake a Post Incident Review and findings will be posted here on completion. We apologise for how you and your business may have been affected by this incident. Please contact the TechnologyOne Support team if you continue to experience issues.

  8. postmortem May 29, 2025, 06:09 AM UTC

    **Issue Summary:** On 14 May at 11:46 PM, a subset of customers in ANZ experienced errors while attempting to access Spatial Cloud due to an infrastructure failure. The issue was promptly detected by the alerting system, and affected customers moved to alternative infrastructure. Service to Spatial Cloud was restored by 15 May 02:00 PM. **Root Cause Analysis:** Failure in upstream service. **Corrective Measures:** Affected customers were migrated to alternative infrastructure to restore the service. **Preventive Measures:** A new infrastructure layer has been deployed to support customers and prevent future similar failures.