Sporadic transaction timeouts
Timeline · 1 update
- resolved Apr 09, 2026, 10:01 PM UTC
One of the storage pods in metadata service got overloaded, and both reads and writes started queuing causing transactions to fail with timeouts
Firebolt had 10 outages in the last 2 years totaling 45h 48m of downtime — averaging 0.4 incidents per month.
There were 10 Firebolt outages since June 21, 2024 totaling 45h 48m of downtime. Each is summarised below — incident details, duration, and resolution information.
One of the storage pods in metadata service got overloaded, and both reads and writes started queuing causing transactions to fail with timeouts
We are currently experiencing service disruptions due to an ongoing Cloudflare outage. This issue is affecting several Firebolt services, including: - Logins management - New user registrations - Documentation search These disruptions may prevent some users from accessing their accounts, creating new accounts, or searching our documentation. We are monitoring Cloudflare’s status closely and will provide updates as the situation evolves.
Cloudflare services are operating as normal, and we have not identified any ongoing disruptions.
We are performing a scheduled maintenance as part of our ongoing improvements to authentication and service account infrastructure. During this time, users may experience temporary login or registration interruptions (timeouts or brief sign-in errors). No action is required, and service account authentication will remain unaffected. Once the update is complete, all systems will resume normal operation.
The scheduled maintenance related to the SSO domain update has been successfully completed. All authentication and sign-in services are now fully operational, and no further interruptions are expected. Thank you for your patience during this maintenance window.
Some Firebolt services may experience latency or disruptions due to an ongoing AWS outage in the US-EAST-1 region. The issue is impacting multiple AWS services, which may affect engine operations and data access in this region. Our team is actively monitoring the situation and working to minimize customer impact. We'll share updates as more information becomes available from AWS.
AWS is currently throttling requests for new EC2 instance launches to aid recovery and actively working on mitigations. This may impact the start up times of your engines.
Engine and database operations in us-east-1 are generally available but may experience increased error rates as AWS continues to stabilize services. We are monitoring and will update this page once all services have been confirmed to resume normal performance levels.
Post-Incident Update: On October 20, Firebolt services in the US-EAST-1 region experienced disruptions related to a broader AWS outage impacting multiple services, including DynamoDB. During this time, some customers in the us-east-1 region may have encountered increased latency, query suspensions, or ingestion job failures. The underlying AWS issue has since been fully resolved, and all Firebolt services have returned to normal operation. No other regions were affected. We continue to monitor system performance closely to ensure stability.
We are currently experiencing an unplanned service outage affecting all regions. As of 19:00 UTC, you may encounter difficulties starting engines in our FB 2.0 environment. Our engineering team is actively working to resolve the issue as quickly as possible. We will provide updates on our Statuspage at least every 15 minutes. Thank you for your patience and understanding.
We've performed a rollback and engines are starting successfully now. The issue was resolved at 19:15 UTC. We apologize for this temporary outage. Services are now fully operational. Please reach out if you experience any further issues.
Due to an Auth0 incident, we are experiencing an interruption to logins (possibly including programmatic access such as service accounts) if the user's token has expired. We will update as we have more information.
We are continuing to investigate this issue.
For users currently on V1 of Firebolt, your authentication should not be affected. Auth0 has advised they have identified the root cause and are working on a remediation.
Auth0 has partially recovered services. You may be able to resume logins/ authentications, however there may still be intermittent errors. We will post here when all services have been 100% recovered.
Auth0 has indicated all systems are operational. You should now be able to resume logins and authentication without interruption. Thank you for your patience.
This incident has been resolved.
Access to Firebolt V2 via the UI is temporarily down, and we are actively investigating. All other operations and access to Firebolt data are unaffected (programmatic access through SDKs and connectors, for example).
We are continuing to investigate the UI access issue. It is possible that programmatic access may be impacted should a user token expire.
Firebolt V2 UI and possible programmatic access issues have been resolved. Full operation has been restored.
This incident has been resolved.
We’re currently experiencing an unplanned service outage that impacts the US-EAST-1 region only. As a result, you may encounter issues accessing the Develop space and receiving API call responses listing engines and accounts. Our team is actively investigating the issue and working toward a resolution. We understand the inconvenience this may cause and appreciate your patience. We will be updating the Statuspage with new information at the latest every 15 minutes.
We are continuing to investigate this issue.
We have identified the root cause of the issue and are making significant progress toward resolving it.
We are still experiencing the issue and are actively working on a resolution. To clarify the impact: - You will not be able to start engines or - you will not be able to use the USE ENGINE command. However, if you already have a running engine, queries should execute as expected. We apologize for the inconvenience and appreciate your patience as we work to resolve this as quickly as possible.
We are continuing to work toward a resolution and will provide an update as soon as operations are fully restored.
We’ve deployed a fix, and the situation is showing signs of improvement. We are monitoring closely as we wait for operations to stabilize further.
The issue is now fully resolved and services are fully operational.
We are currently experiencing an unplanned service outage affecting all regions. As of 18:26 UTC, you may encounter difficulties starting engines in our FB 2.0 environment. Our engineering team is actively working to resolve the issue as quickly as possible. We will provide updates on our Statuspage at least every 15 minutes. Thank you for your patience and understanding.
We are still experiencing the issue, and continuing to work on a fix Updated Impact: - Engines are unable to start. - Engines are unable to stop with Autostop and remain in the "Running" state, even when no queries are executed. - Insert commands are failing on currently running engines. We continue to look into this with the highest priority.
The issue was resolved at 20:03 UTC. We experienced an outage in engine operations, which was addressed by reverting a recent network change. Services are now fully operational. Please reach out if you experience any further issues.
We experienced an incident that caused the engine not to start and also impacted the ingest process. Issue started at Fri Jun 21 2024 06:42:13
We are continuing to investigate this issue.
Our engineering team has identified and resolved the issue, and we are currently monitoring the situation to ensure stability. Issue solved at Fri Jun 21 2024 08:41:36 GMT+0200
We are continuing to monitor for any further issues.
This incident has been resolved.