StrongDM incident

Routing Issues

Critical Resolved View vendor source →

StrongDM experienced a critical incident on September 8, 2023 affecting API, lasting 11m. The incident has been resolved; the full update timeline is below.

Started
Sep 08, 2023, 04:45 PM UTC
Resolved
Sep 08, 2023, 04:57 PM UTC
Duration
11m
Detected by Pingoru
Sep 08, 2023, 04:45 PM UTC

Affected components

API

Update timeline

  1. investigating Sep 08, 2023, 04:45 PM UTC

    Some organizations may experience connection issues. We are currently investigating the issue and will update here with additional information.

  2. resolved Sep 08, 2023, 04:57 PM UTC

    We made an enhancement to our routing protocol for compatibility with a new operating mode, which caused the current routing system to fail. The change has since been rolled back, and the issue identified to be resolved. The service should be functioning normally at this time.

  3. postmortem Sep 08, 2023, 10:01 PM UTC

    At 16:04 UTC, September 8th, we made an enhancement to our routing protocol for compatibility with a new operating mode, which caused the current routing system to fail. The change has since been rolled back, and the issue identified to be resolved. As of 16:48 UTC, September 8th, full functionality has been restored. In response to this we will be augmenting our monitoring processes for new version rollouts, ensuring we can react to these kinds of changes more quickly.