Sigma Computing incident
Errors in exports and existing workbooks on Sigma Azure
Sigma Computing experienced a minor incident on August 24, 2023 affecting Sigma - GCP (US) and Sigma - AWS (Europe) and 1 more component, lasting 11h 36m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Aug 24, 2023, 02:16 PM UTC
We are currently investigating an issue with opening existing workbooks and scheduled exports for customers on Sigma Azure. Users will still be able to create new workbooks. This issue is being treated with the highest priority and we will update the status accordingly. Impacted Cloud Regions & Services: Sigma organizations hosted on Azure User Experience: Errors when opening existing workbooks and on scheduled exports Incident Start Time: Approximately 12:53 UTC Aug 24, 2023
- identified Aug 24, 2023, 03:27 PM UTC
Current Status: Our engineers have identified a problem with our workbook retention mechanism. This is being fixed now and we're looking at recovery of inaccessible workbooks Impacted Cloud Regions & Services: Sigma users hosted on Azure only User Experience: Errors when opening existing workbooks and on scheduled exports Incident Start Time: Approximately 12:53 UTC Aug 24, 2023
- monitoring Aug 24, 2023, 06:04 PM UTC
Current Status: Our engineers have implemented mitigation measures to circumvent the issue and we are monitoring their effect closely. We are in the process of fixing impacted workbooks. Impacted Cloud Regions & Services: Sigma users hosted on Azure only User Experience: We expect the rate of errors for new workbooks to decrease. Workbooks already impacted are still being recovered. Incident Start Time: Approximately 12:53 UTC Aug 24, 2023
- monitoring Aug 24, 2023, 07:24 PM UTC
Current Status: Our engineers have implemented mitigation measures to circumvent the issue and we are monitoring closely. We are still actively working on fixing impacted workbooks. Impacted Cloud Regions & Services: Sigma users hosted on Azure only User Experience: We expect the rate of errors for new workbooks to decrease. Workbooks already impacted are still being recovered. Incident Start Time: Approximately 12:53 UTC Aug 24, 2023
- resolved Aug 25, 2023, 01:52 AM UTC
Current Status: The mitigation measures have proven effective and the issue is resolved. Impacted workbooks have been restored. Impacted Cloud Regions & Services: Sigma users hosted on Azure only User Experience: This errors for new workbooks has already been resolved and impacted workbooks are now restored. Incident Start Time: Approximately 12:53 UTC Aug 24, 2023 Incident End Time: Approximately 01:30 UTC Aug 25, 2023 We will perform a root cause analysis (RCA) and publish our findings in a postmortem note to the Sigma Community site in coming days. You will receive a notification when that occurs.
- postmortem Sep 09, 2023, 12:39 AM UTC
[08/24/2023 Incident](https://community.sigmacomputing.com/t/postmortem-for-august-24-2023-errors-in-exports-and-existing-workbooks-on-sigma-azure/2564)