TechnologyOne incident

Resource Booker & Publish Performance Issues

Minor Resolved View vendor source →

TechnologyOne experienced a minor incident on January 21, 2025 affecting Timetabling & Scheduling and Timetabling & Scheduling, lasting 9h 29m. The incident has been resolved; the full update timeline is below.

Started
Jan 21, 2025, 09:39 AM UTC
Resolved
Jan 21, 2025, 07:08 PM UTC
Duration
9h 29m
Detected by Pingoru
Jan 21, 2025, 09:39 AM UTC

Affected components

Timetabling & SchedulingTimetabling & Scheduling

Update timeline

  1. investigating Jan 21, 2025, 09:39 AM UTC

    Please be advised that we are aware of Resource Booker performance issues and the team are actively working towards a resolution.

  2. identified Jan 21, 2025, 09:49 AM UTC

    The team have identified the issue and are working on a solution.

  3. identified Jan 21, 2025, 10:53 AM UTC

    Progress is being made in restoring services. We continue to deploy our fix and will update once service has been fully restored

  4. identified Jan 21, 2025, 01:14 PM UTC

    We continue to make good progress in restoring services. Work is still ongoing to fully restore services.

  5. monitoring Jan 21, 2025, 04:55 PM UTC

    Performance and stability continues to improve. We're now monitoring this issue

  6. resolved Jan 21, 2025, 07:08 PM UTC

    Our engineers are now happy with the performance and stability of RB and PB.

  7. postmortem Feb 12, 2025, 06:57 AM UTC

    Issue Summary: On Tue, Jan 21, a subset of customers experienced sub-optimal performance with Resource Booker, including slow response times and errors loading admin functions. Root Cause: The system degradation was caused by Resource Booker overloading the system with large volume of data when it searched for pending approvals and found none. This was caused by a bug that was introduced into the software that surfaced in the production environment. Corrective Actions: Temporarily optimised the search services to handle more requests. Preventive Measures: Updated Resource booker to only search for events when necessary and only a subset of the data.