Flightcontrol incident

Setting secrets and creating apps is degraded

Major Resolved View vendor source →

Flightcontrol experienced a major incident on March 11, 2026 affecting Machines API and Deployments, lasting 33m. The incident has been resolved; the full update timeline is below.

Started
Mar 11, 2026, 11:03 AM UTC
Resolved
Mar 11, 2026, 11:37 AM UTC
Duration
33m
Detected by Pingoru
Mar 11, 2026, 11:03 AM UTC

Affected components

Machines APIDeployments

Update timeline

  1. identified Mar 11, 2026, 09:19 AM UTC

    An ongoing data migration in our secret storage service is causing degraded Machines API functionality.

  2. monitoring Mar 11, 2026, 10:14 AM UTC

    A fix has been implemented and we are monitoring the results.

  3. monitoring Mar 11, 2026, 11:03 AM UTC

    While the secret storage service was in a read-only state, app creation requests queued up, due to the retry logic and insufficient request concurrency limits in our GraphQL API. This prevented our GraphQL API from serving any other requests. We have scaled up the GraphQL API and are continuing to monitor the situation.

  4. resolved Mar 11, 2026, 11:37 AM UTC

    This incident has been resolved.