Cornerstone incident

Service Disruption - Error logging in to US SL1, SL2, SL3 and SL5 portals

Critical Resolved View vendor source →

Cornerstone experienced a critical incident on November 27, 2025 affecting Uptime and Uptime and 1 more component, lasting 2h 4m. The incident has been resolved; the full update timeline is below.

Started
Nov 27, 2025, 01:38 PM UTC
Resolved
Nov 27, 2025, 03:43 PM UTC
Duration
2h 4m
Detected by Pingoru
Nov 27, 2025, 01:38 PM UTC

Affected components

UptimeUptimeUptimeUptime

Update timeline

  1. investigating Nov 27, 2025, 01:38 PM UTC

    US swim lanes (SL1, SL2, SL3 and SL5) are experiencing a service disruption. This is our top priority and we are working to resolve the problem as soon as possible. Please check back periodically for additional updates, which will be posted as they become available.

  2. monitoring Nov 27, 2025, 02:40 PM UTC

    The CSOD Technology Team observed a service disruption on US swim lanes (SL1, SL2, SL3 and SL5). The problem began on 27 Nov 5:20 AM Pacific Time and service have been restored now. Clients should now be able to access the portals successfully now. We are actively monitoring the system performance before we confirm the incident to be resolved.

  3. resolved Nov 27, 2025, 03:43 PM UTC

    The CSOD Technology Team observed a service disruption on US SL1, SL2, SL3 and SL5. The problem began on 27 Nov 5:20 AM Pacific Time and service was restored at 6:26 AM Pacific Time. During this time, clients with portals on these swimlanes were not able to access the application. Clients should now be able to access the portals successfully. RCA will be provided in 7-10 business days.

  4. postmortem Dec 03, 2025, 11:51 PM UTC

    **Incident Summary:** On November 27th, users accessing the US PROD environment experienced 504 Gateway Errors, resulting in intermittent inability to access platform features. The issue affected all clients in the region during the incident window. **Root Cause:** The errors were caused by traffic from the CDN being temporarily blocked, which prevented requests from reaching the platform. This disruption was related to network configuration that impacted traffic flow to the US PROD environment. **Corrective Action:** A network configuration adjustment was applied to restore CDN traffic to the platform, which immediately resolved the errors. Platform performance and availability were verified, confirming full resolution of the incident. **Preventive Measures:** To reduce the likelihood of similar incidents in the future: * Cornerstone is reviewing network configurations and implementing safeguards to prevent traffic blockage. * Internal processes for network changes are being strengthened to minimize impact on production traffic.