Orb experienced a notice incident on October 29, 2024 affecting Reads and Writes and 1 more component, lasting 16h 34m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 29, 2024, 01:09 PM UTC
A query load issue has been identified starting at 2024/10/29 8:45 UTC which was causing intermittent spikes of errors accessing internal data storage, resulting in API errors and delays in async processing. This issue has been mitigated for the majority of customers as we continue to investigate.
- monitoring Oct 29, 2024, 02:12 PM UTC
API services stabilized at 12:30 UTC and async processing has been restored, and we are continuing to monitor our systems.
- monitoring Oct 29, 2024, 07:39 PM UTC
API services and asynchronous processes are stable. The query load issue has been fully isolated and we're working on improvements to prevent this issue in the future.
- resolved Oct 30, 2024, 05:44 AM UTC
This incident has been resolved.