"Invalid Login Credentials” error on LAX SL4

Update timeline

1. monitoring Aug 25, 2025, 01:13 PM UTC

   The CSOD team has identified that users were unable to log in to their production and stage environments and were receiving “Invalid Login Credentials” errors for LAX SL4. The CSOD team confirmed that the issue should now be resolved. The incident will be monitored for another 1h to ensure system stability.

2. resolved Aug 25, 2025, 02:16 PM UTC

   The incident has been resolved at 05:40 AM PDT.

3. postmortem Sep 24, 2025, 02:20 PM UTC

   **Incident Summary:** On August 25, 2025, customers hosted in the LAX PRD SL4 swimlane experienced intermittent access issues. Users reported errors when attempting to log in due to a disruption in authentication-related microservices. **Root Cause:** The disruption was traced to a problem on one of the servers hosting the authentication microservices. The affected node was unable to properly handle requests, leading to intermittent authentication failures. **Corrective Action:** Cornerstone engineers promptly investigated the issue, recycled the impacted node, and reintroduced it into the cluster. These actions stabilized the service and restored normal operations for the swimlane. **Preventive Measures:** To reduce the risk of recurrence, Cornerstone is implementing the following improvements: * Advanced Health Checks: Implementing additional health checks within the microservices architecture to detect and isolate problematic nodes earlier * Ongoing Optimization: Cross-functional teams are conducting reviews and implementing optimizations to strengthen resilience and reduce the likelihood of recurrence