Secret Server Cloud: Issues with RDP and Remote Password Changes

Update timeline

1. investigating Jul 27, 2026, 06:05 PM UTC

   We have identified an issue preventing some customers from connecting via RDP or changing passwords remotely. We are actively working to resolve the issue and will provide another update soon.

2. identified Jul 27, 2026, 07:34 PM UTC

   We have identified the issue preventing some customers from connecting via proxy or changing passwords remotely. During this incident, some customers also experienced errors when creating folders. Our engineering team has isolated the root cause and is actively implementing a fix to restore normal service. We will provide another update as progress continues.

3. monitoring Jul 27, 2026, 08:10 PM UTC

   We have applied mitigation measures to restore service availability and are actively monitoring the environment to ensure continued stability. We apologize for any inconvenience this may have caused and appreciate your patience.

4. resolved Jul 27, 2026, 09:30 PM UTC

   The issues affecting our Secret Server Cloud customers has been resolved. We apologize for any inconvenience this may have caused.

5. postmortem Aug 04, 2026, 08:24 PM UTC

   ## Incident Overview On July 27, 2026, Secret Server Cloud \(US region\) customers experienced widespread failures with RDP sessions, remote password rotation, and folder creation. The root cause was a critical internal caching layer that filled up to capacity because session entries were being stored in it without expiration. As the cache filled, it began rejecting write operations, causing a backend processing service to fail. Messages continually accumulated in an internal message queue which degraded performance for all affected tenants. * Start: July 27, 2026, approximately 2:00 PM EDT * End: July 27, 2026, approximately 4:10 PM EDT * Region impacted: US ## Root Cause and Remediation The internal caching layer had been accumulating session data with no expiration for over 20 days. At the time of investigation, approximately 86% of cache entries had no expiration set. The root cause is a code-level bug that was not setting expirations properly on session entries. Once the cache reached capacity, new writes were rejected, breaking the log pipeline and cascading into broader service failures across tenants. To remediate, the caching layer was manually flushed at approximately 4:04 PM EDT, restoring write capacity and allowing the message queue to drain. A code fix to add TTL \(time-to-live\) to the affected keys has been identified and is in progress. ## Preventative Actions * Apply a code fix to add TTL to session entries and other unbounded growth keys in the caching layer, preventing recurrence regardless of deployment cadence. * Add memory utilization alerting for the caching layer at defined thresholds, so memory pressure is detected proactively before capacity is reached. * We are looking into adding a post-release action for flushing the cache as a standard step in the deployment process We sincerely apologize for the disruption this caused and the inconvenience to your operations. We are committed to preventing recurrence through the above actions.