TechnologyOne incident

SaaS Service Degradation for a subset of customers / ANZ Region / All Release

Minor Resolved View vendor source →

TechnologyOne experienced a minor incident on June 4, 2024, lasting 2d. The incident has been resolved; the full update timeline is below.

Started
Jun 04, 2024, 02:19 AM UTC
Resolved
Jun 06, 2024, 02:22 AM UTC
Duration
2d
Detected by Pingoru
Jun 04, 2024, 02:19 AM UTC

Update timeline

  1. investigating Jun 04, 2024, 02:19 AM UTC

    We are investigating an issue impacting service for the ANZ Region across all releases A subset of customers in ANZ Region for 22B/23A/23B/24A Release are experiencing sub-optimal performance. Due to the investigation, the next update will be provided in 60 minutes, or sooner if new information becomes available.

  2. investigating Jun 04, 2024, 03:25 AM UTC

    We are continuing to investigate this issue. We shall aim to provide you with an update in the next 60 minutes or sooner if new information becomes available. Thank you for your patience while we address this incident. Thank you

  3. investigating Jun 04, 2024, 04:30 AM UTC

    Our team of engineers are actively investigating this issue. We shall aim to provide you with an update in the next 60 minutes or sooner if new information becomes available. Thank you

  4. investigating Jun 04, 2024, 05:35 AM UTC

    Investigation is still in progress. Next update shall be provided once we have identified the cause and potentially a fix for this incident. Thank you for your patience.

  5. monitoring Jun 04, 2024, 10:47 PM UTC

    The issue has been identified and a fix has been implemented overnight. Our team will continue to monitor this before resolving the incident.

  6. resolved Jun 06, 2024, 02:22 AM UTC

    After 48 hours monitoring this incident is now resolved. We will perform a post incident review to identify underlying cause, and preventive action to avoid a repeat in the future and post here on completion. We apologise for how you and your business may have been affected by this incident.

  7. postmortem Jul 05, 2024, 05:31 AM UTC

    **Issue Summary:** On the 4th of June 2024, the SaaS operations team identified an increase in user traffic, which caused degraded performance primarily in CES, ECM and P&R. **Root Cause Analysis:** Queue and processing limits were reached due to an unexpected increase in traffic and resource demand. The issue was isolated to 2 customers. **Preventive Measures:** Previously implemented preventive measures worked as the impact was contained.