Sigma Computing incident

Query failures for select organizations connected to Snowflake

Critical Resolved View vendor source →

Sigma Computing experienced a critical incident on October 3, 2023 affecting Sigma - GCP (US) and Sigma - AWS (US), lasting 5h 7m. The incident has been resolved; the full update timeline is below.

Started
Oct 03, 2023, 01:41 PM UTC
Resolved
Oct 03, 2023, 06:48 PM UTC
Duration
5h 7m
Detected by Pingoru
Oct 03, 2023, 01:41 PM UTC

Affected components

Sigma - GCP (US)Sigma - AWS (US)

Update timeline

  1. investigating Oct 03, 2023, 01:41 PM UTC

    We are investigating an issue with queries resulting in the error message "Bad request; operation not supported." We are treating this as our highest priority. Our apologies for any inconvenience caused.

  2. investigating Oct 03, 2023, 02:43 PM UTC

    We are investigating an issue with queries resulting in the error message "Bad request; operation not supported." We have engaged with Snowflake support and are treating this as our highest priority. Our apologies for any inconvenience caused.

  3. investigating Oct 03, 2023, 04:45 PM UTC

    We were able to to reproduce locally and issue is fixed in our environment after Snowflake rolled back a server change on their side. We're now working with them to apply the rollback to all accounts.

  4. resolved Oct 03, 2023, 06:48 PM UTC

    We have received confirmation from Snowflake that they've rolled back globally for all impacted users. At this time we believe full functionality has been restored and expect all Snowflake users can now access their workbooks and data warehouse connections. Impacted Cloud Regions & Services: Snowflake connections for a subset of Sigma users hosted on GCP and AWS User Experience: Affected users can now access their workbooks and data warehouse connections. Incident Start Time: Approximately 13:00 UTC October 3, 2023 Incident End Time: Approximately 18:03 UTC October 3, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days.

  5. postmortem Oct 17, 2023, 09:15 PM UTC

    Sigma Engineering has completed the postmortem of this incident. A Root Cause Analysis \(RCA\) is available on the Sigma Community site: [Postmortem for Oct 03, 2023 Incident: Query failures for select organizations connected to Snowflake ](https://community.sigmacomputing.com/t/postmortem-for-oct-03-2023-incident-query-failures-for-select-organizations-connected-to-snowflake/2634)