Intermittent Latency and Errors Across Application - US Swimlanes

Update timeline

1. investigating Mar 03, 2026, 03:59 PM UTC

   Some customers on a US swimlane may experience latency and errors intermittently across the application. We are actively investigating the issue as our top priority. We will share regular updates as they become available.

2. monitoring Mar 03, 2026, 04:56 PM UTC

   A fix has been implemented and we are monitoring the results.

3. monitoring Mar 03, 2026, 07:44 PM UTC

   We are maintaining the status in Monitoring while we continue to ensure stability is sustained. We will continue to provide updates until the issue is considered fully resolved.

4. identified Mar 03, 2026, 08:27 PM UTC

   We are currently observing intermittent latency spikes. The root cause has been identified, and our teams are actively working on a remediation plan. We are targeting deployment of the fix during non-business hours to minimize further customer impact. We will continue to provide updates as more information becomes available.

5. identified Mar 03, 2026, 09:51 PM UTC

   Some customers may continue to experience intermittent latency spikes. Our teams remain actively engaged and are making progress on remediation efforts. Deployment of the proposed fix remains scheduled for non-business hours to minimize any further customer impact. We will continue to share regular updates.

6. identified Mar 03, 2026, 11:46 PM UTC

   Some customers may continue to experience intermittent latency spikes. Our teams remain actively engaged and are making progress on remediation efforts. Deployment of the proposed fix remains scheduled for non-business hours to minimize any further customer impact. We will continue to share regular updates.

7. identified Mar 04, 2026, 01:35 AM UTC

   Some customers may continue to experience intermittent latency spikes. Our teams remain actively engaged and are making progress on remediation efforts. Deployment of the proposed fix remains scheduled for non-business hours to minimize any further customer impact. We will continue to share regular updates.

8. identified Mar 04, 2026, 05:41 AM UTC

   Some customers may continue to experience intermittent latency spikes. Our teams remain actively engaged and are making progress on remediation efforts. Deployment of the proposed fix remains scheduled for non-business hours to minimize any further customer impact. We will continue to share regular updates.

9. identified Mar 04, 2026, 08:16 AM UTC

   Some customers may continue to experience intermittent latency spikes. Our teams remain actively engaged and are making progress on remediation efforts. Deployment of the proposed fix remains scheduled for non-business hours to minimize any further customer impact. We will continue to share regular updates.

10. identified Mar 04, 2026, 10:34 AM UTC

    Some customers may continue to experience intermittent latency spikes. Our teams remain actively engaged and are making progress on remediation efforts. Deployment of the proposed fix remains scheduled for non-business hours to minimize any further customer impact. We will continue to share regular updates.

11. monitoring Mar 04, 2026, 01:01 PM UTC

    A fix for this issue was deployed successfully. We are monitoring the swimalanes before concluding this incident as resolved.

12. identified Mar 04, 2026, 06:43 PM UTC

    We observed brief latency spikes this morning. Our teams are actively working on a permanent fix to fully address the issue. We will continue to provide regular updates until it is resolved. Thank you for your patience.

13. monitoring Mar 04, 2026, 08:36 PM UTC

    We observed a brief spike in latency between 11:00 AM and 11:15 AM PT. Mitigation measures have been implemented to help minimize intermittent performance degradation while a permanent fix is being developed. We will continue to share regular updates as progress is made.

14. resolved Mar 07, 2026, 05:01 AM UTC

    The CSOD Technology Team observed a performance degradation affecting this swimlane. During this time, clients with portals on this swimlane may have experienced slow performance or intermittent errors while accessing the application. Our Engineering teams have implemented a fix and after a period of monitoring we are considering this issue resolved.

15. postmortem Mar 15, 2026, 07:09 PM UTC

    **Issue Summary:** Between February 26 and March 12, 2026, users experienced intermittent latency while accessing the platform hosted in the US SL1, SL2, SL3, and SL5 environments. During this period, some requests took longer than expected to process, resulting in degraded performance across the application. The issue impacted multiple swimlanes but did not result in complete service unavailability. **Root Cause:** The latency was caused by stability issues within an underlying authentication service responsible for user authentication and session management. Increased authentication activity, partly driven by the enablement of global SSO capabilities for multiple customers, placed additional load on the service. This resulted in intermittent service instability and elevated request processing times across the affected environments. **Corrective Action:** Cornerstone Engineering teams implemented the following actions to restore service stability: * Increased authentication service capacity to reduce immediate load and stabilize the authentication infrastructure. * Conducted a detailed review of the service implementation to identify inefficiencies. * Optimized the service by removing unnecessary code references and improving memory management. Following these actions, the authentication service stabilized and platform latency returned to normal levels. **Preventive Measures:** To reduce the risk of recurrence, Cornerstone has taken the following actions: * Increased authentication service capacity to better support growing authentication demand. * Implemented code and memory management optimizations to improve service stability and efficiency. * Introduced enhanced monitoring to provide greater visibility into authentication service performance and detect anomalies earlier.