Thycotic incident
Platform: US - Disruption in data fetching for Privilege Control for Cloud Entitlements (PCCE)
Thycotic experienced a minor incident on June 5, 2025 affecting Platform, lasting 20h 3m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Jun 05, 2025, 09:25 PM UTC
We are currently investigating an issue where the Privilege Control for Cloud Entitlements (PCCE) integration service is not fetching new data as expected. Despite this, the user interface (UI) remains fully operational, allowing users to access and interpret previously fetched data without interruption. Our engineering team is actively working to restore the functionality.
- resolved Jun 06, 2025, 05:28 PM UTC
This incident has been resolved. If you have any questions please contact our support team at https://support.delinea.com
- postmortem Jun 16, 2025, 02:46 PM UTC
### Incident Overview On June 5, 2025, at 5:25 PM EDT, we identified an issue where the Privilege Control for Cloud Entitlements \(PCCE\) integration service stopped fetching new data. Despite this disruption in data ingestion, the PCCE user interface \(UI\) remained fully functional, allowing users uninterrupted access to previously fetched data. ### Root Cause The issue was traced to failures within our ArangoDB infrastructure, which supports PCCE data ingestion. The primary ArangoDB cluster encountered stability issues, and recovery efforts on the existing cluster were unsuccessful. We executed our contingency plan, treating this as a disaster recovery scenario, and migrated customer databases to a healthy ArangoDB cluster. Once the migration was completed, the data pipeline resumed normal operation. ### Preventive Actions * We are actively working to significantly reduce the write load on the database, which we believe is a contributing factor to the issue. * This effort will also enable us to leverage a secondary hot backup instance for faster and more seamless recovery in future incidents.