Firebolt incident

Engines failing to start in us-east-1

Major Resolved View vendor source →

Firebolt experienced a major incident on April 19, 2022 affecting Engines, lasting 2h 45m. The incident has been resolved; the full update timeline is below.

Started
Apr 19, 2022, 03:18 PM UTC
Resolved
Apr 19, 2022, 06:04 PM UTC
Duration
2h 45m
Detected by Pingoru
Apr 19, 2022, 03:18 PM UTC

Affected components

Engines

Update timeline

  1. investigating Apr 19, 2022, 03:18 PM UTC

    Hello - we are experiencing issues provisioning engines in us-east-1. Please do not stop/restart existing engines during this time. We are actively investigating and will update with an ETA on the fix shortly.

  2. monitoring Apr 19, 2022, 03:46 PM UTC

    AWS has issued a message as of 8:39am PDT stating they are having issues with all engines in us-east-1 and have identified and are working to fix the root of the issue. We will continue to monitor the AWS event and ensure we are operational asap.

  3. resolved Apr 19, 2022, 06:04 PM UTC

    The AWS issue has been resolved and any engine that was stuck will start up shortly. You may execute engine start ups and restarts in us-east-1 again. We appreciate your patience on this issue.