Thycotic Outage History

**Incident Overview** On April 24, 2026, a subset of Secret Server Cloud customers in the US region experienced intermittent errors or connectivity issues when accessing their tenants. The disruption was caused by an outage in our cloud infrastructure provider at the East US data center. The issue originated in a single part of the data center but spread to additional areas as traffic was automatically redistributed, extending the scope and duration of the incident. Full-service recovery was confirmed at 00:15 UTC on April 25, 2026. **Root Cause** The outage impacted internal networking components that our platform depends on, disrupting connectivity across the environment. These failures contributed to the intermittent HTTP 500 errors experienced by some Delinea customers during the incident window. The issue originated in one of the Availability zones in East US data center and expanded to other zones as traffic was automatically redistributed, broadening the impact. Our team responded promptly, identifying the root cause at our cloud infrastructure provider and redistributing traffic to other zones. As a precautionary measure, our team initiated a failover to a secondary region, but later determined it was no longer needed and safely rolled back with no additional impact. Service was fully restored after our infrastructure team redirected traffic away from the affected nodes within the primary environment, with full recovery confirmed at approximately 00:15 UTC on April 25, 2026. **Preventive Actions** * Evaluate multi-availability-zone node pool configurations to improve platform resilience and reduce the impact of localized data center failures. * Establish clear thresholds and decision criteria for triggering regional failover versus in-region mitigation to improve response speed during incidents.

### Incident Overview On Friday, April 3rd, 2026, a subset of SAML/Federation users were unable to authenticate to their Delinea Platform tenants. The issue was first reported at approximately 5:01 PM ET and was fully resolved by 8:38 PM ET following a rollback of a recent platform update. Tenants using the default SAML issuer configuration were not impacted. Customers with local \(non-SAML\) accounts retained access throughout the incident via breakglass or local admin credentials. ### Root Cause A platform update deployed at approximately 4:20 PM ET introduced a code change that inadvertently caused custom SAML authentication configurations to not be correctly applied for affected tenants. This resulted in authentication failures for tenants that relied on a custom SAML issuer, while tenants using default settings were unaffected. A contributing factor was insufficient test coverage for this specific authentication scenario prior to the global rollout of the update. ### Preventive Actions * A code fix is being implemented to ensure custom SAML authentication configurations are correctly applied in all cases going forward. * Automated test coverage is being expanded to include SAML authentication scenarios with custom configurations to prevent similar issues in future releases.

## Incident Overview On Monday, March 16, 2026, customers hosted on the Delinea Secret Server Cloud \(SSC\) UK cluster experienced a service disruption that prevented access to the SecretServer endpoint. The Delinea WAF blocked inbound traffic to the affected endpoint as an automated protective response after regional traffic volumes exceeded predefined DDoS threshold limits. * Start Time: February 16, 2026 – 10:03 UTC * End Time: February 16, 2026 – 10:57 UTC ## Root Cause The incident was caused by abnormally high traffic generated by a single customer Secret Server engine, which exceeded predefined DDoS protection threshold limits in the UK region. Key contributing factor: * Engine request spike:` `Traffic from secret-server-engines increased dramatically from a typical baseline of fewer than 100 calls to approximately 27,000 calls, triggering automated DDoS protection mechanisms. ## Preventive Actions * The UK site DDoS threshold was temporarily increased to 4500, allowing legitimate but abnormal traffic patterns to be handled while ensuring continued regional protection. * Enhance monitoring to identify sharp, customer-specific engine call spikes.

## Incident Overview On March 12, 2026, a subset of customers in Australia \(AU\) and the United States \(US\) regions began experiencing Federation configuration error. The issue rooted from a planned software update that prevented some users from logging in to Platform. This was caused by the user session becoming corrupted or inconsistent. Start Time: March 12, 2026 – 01:33 UTC End Time: March 12, 2026 – 14:43 UTC At the time of the incident only AU and US were marked as impacted, but later investigations identified that it impacted subset of clients globally. ## Root Cause The incident was caused by a defect introduced in the Identity API release that corrupted user authentication state for federated users. This error persisted even after the deployment was rolled back, requiring a targeted fix to repair affected tenants. Key contributing factors included: A defect in the new Identity API release caused user accounts to enter a broken state upon authentication. Automated background session refreshed affected users who were not actively logged in, extending the scope of the impact beyond those actively using the platform. The sequence of multiple coordinated releases made it more difficult to isolate the faulty component quickly ## Preventive Actions * A targeted new build was developed and deployed to handle users who entered the bad state, fully resolving the incident. * We are adding additional pre-release testing for federated authentication flows: All releases that touch authentication, Identity, or federation service will have dedicated test coverage. * Cross-service release dependency review: We are confirming that the combination of releases has been tested together in lower silos, not just each service in isolation.

**Incident Overview** Between 2:20 PM and 3:34 PM EST on February 9, 2026, Platform users experienced red banner errors, “Cannot access ‘t’ before initialization”, when clicking on the Discovery link in the left-hand navigation. These errors occurred due to client-side bundle incompatibility introduced during a recent Platform update. * Start of Impact: February 9, 2026, at 19:20 UTC * End of Impact: February 9, 2026, at 20:34 UTC **Resolution** We resolved the incident by pinning our build tooling back to the previous compatible version of webpack, which restored correct bundle generation and eliminated the errors. Normal service was confirmed to be restored by 3:34 PM EST. **Root cause** The incident was caused by a minor Angular framework update that indirectly upgraded the webpack to a newer version - a change that was not explicitly requested or anticipated. This newer version of webpack generated application bundles in a format that was incompatible with our existing module federation setup, which is the mechanism we use to load different sections of the platform as separate, independently deployable units. When the incompatible bundle was loaded, it caused client-side errors that surfaced to users as red banner messages. **Preventative Actions** We are taking the following steps to reduce the risk of similar incidents occurring in the future: * As a longer-term improvement, we are planning a migration away from our current module federation setup to native federation, which will eliminate the dependency on webpack and reduce the risk of similar issues in the future.