Omni Analytics experienced a minor incident on May 15, 2025 affecting EastUsa, lasting 6h 7m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating May 15, 2025, 05:36 PM UTC
We detected a spike in internal errors affecting some hosts. Behavior is intermittent and refreshing the page should result in successful load.
- identified May 15, 2025, 06:06 PM UTC
The issue has been identified and temporary mitigation put in place. We have not observed internal errors being surfaced in the last 20 minutes. We are continuing to monitor while we work on a longer term fix.
- identified May 15, 2025, 08:23 PM UTC
We are continuing to work on a long term fix and monitoring to ensure our mitigation is holding.
- resolved May 15, 2025, 11:43 PM UTC
This incident has been resolved. A resource intensive process was initiated during a busy period that overwhelmed a subset of our internal resources and resulted in intermittent failure to complete requests. Manual scaling of component-specific resources was needed in addition to our automated scaling to recover. We have implemented some immediate efficiency improvements and are working on implementing additional larger improvements in the coming days.