One Identity Starling experienced a major incident on November 22, 2024 affecting Connect and Connect and 1 more component, lasting 1h 57m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Nov 22, 2024, 07:05 PM UTC
Connector customers with Safeguard Privileged Password may experience issues performing certain tasks against the Registered Connector Asset [Test system, Check Password, Change Password, etc...]. The issue has been identified and we are working on a solution. Then next update will occur at 1PM PST
- monitoring Nov 22, 2024, 08:44 PM UTC
A fix has been applied and we are currently monitoring.
- resolved Nov 22, 2024, 09:03 PM UTC
The issue has been verified to be resolved, all related services have returned to normal functional status.
- postmortem Jan 08, 2025, 12:16 PM UTC
### **What happened?** Between 00:38 and 20:50 UTC on 2024-11-22, customers experienced connection errors related to the Starling Connect service. ### **What went wrong and why?** At 16:06 UTC on 2024-11-21, a change was introduced to Connect Supervisor restricting certain content to JSON format. The Connector application used by Safeguard Privileged Passwords \(SPP\) was not configured to reflect this limitation. Consequently, the Connector application accepted content in other formats such as TXT and plain text. When we introduced the JSON format restriction, connections to SPP in non-JSON format began to fail. This impacted some, but not all, of our customers. ### **How did we respond?** This incident was detected at 00:38 UTC on 2024-11-22. After receiving the alert, we started to investigate the incident by analyzing support bundles of various customer instances. At 17:14 UTC, a decision was made to roll back Connect Supervisor to its prior version. After successfully testing the rollback in a non-production environment, the rollback change was applied to the production environment at 20:43 UTC. The Statuspage incident was updated to reflect that a fix was applied at 20:44 UTC, and we confirmed that the fix had resolved the issue at 21:05 UTC on 2024-11-22. ### **How are we making incidents like this less likely or less impactful?** We have updated our test plan to ensure SPP teams are included.