Orb experienced a minor incident on October 20, 2025 affecting Reads and Writes and 1 more component, lasting 16h 17m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 20, 2025, 07:28 AM UTC
We are currently investigating API degradation. This seems to be related to an upstream cloud provider outage, and we are closely monitoring.
- investigating Oct 20, 2025, 07:38 AM UTC
We are continuing to see and investigate errors.
- investigating Oct 20, 2025, 08:02 AM UTC
We are continuing to investigate errors.
- investigating Oct 20, 2025, 08:17 AM UTC
We are continuing to investigate this issue. We apologize for the disruption to impacted customers.
- investigating Oct 20, 2025, 08:47 AM UTC
We are continuing to investigate this issue.
- investigating Oct 20, 2025, 09:14 AM UTC
We are continuing to investigate the issue.
- investigating Oct 20, 2025, 09:53 AM UTC
API services are broadly recovered; we're continuing to monitor the situation.
- investigating Oct 20, 2025, 10:19 AM UTC
We are seeing recovery on most API endpoints - some analytics endpoints may still be degraded. We are continuing to work with upstream providers to resolve. Note also that consumption from payment processing may be delayed; we are working through the event backlog here.
- investigating Oct 20, 2025, 10:50 AM UTC
We are failing over affected services gradually for our analytics APIs, which we expect to lead to errors in order to assist with full recovery.
- identified Oct 20, 2025, 11:17 AM UTC
We're continuing to investigate, and working through usage calculation delays.
- identified Oct 20, 2025, 11:24 AM UTC
We are working with our cloud provider to bring on more capacity to help with operational delays.
- identified Oct 20, 2025, 11:51 AM UTC
We are continuing to investigate this issue. We are working with our cloud provider to bring on more capacity to help with operational delays.
- identified Oct 20, 2025, 12:43 PM UTC
We are continuing to work with our cloud provider to bring on more capacity to help with operational delays.
- identified Oct 20, 2025, 02:00 PM UTC
We are continuing to work with our cloud provider to bring on more capacity to help with operational delays. Asynchronous processing may be delayed, including the sending of webhooks.
- identified Oct 20, 2025, 02:51 PM UTC
We are continuing to work with our cloud provider to bring on more capacity to help with operational delays. API calls may not reflect the latest usage data as we work through this. Asynchronous processing may be delayed, including the sending of webhooks and the receipt of information from your payment gateway connection. We are working diligently on our side to bring these up to date.
- identified Oct 20, 2025, 03:51 PM UTC
We are continuing to work on a fix for this issue.
- monitoring Oct 20, 2025, 06:39 PM UTC
We are seeing significant recovery as capacity is coming back online.
- monitoring Oct 20, 2025, 09:03 PM UTC
Most async processing is operational, including webhooks and receipt of information from your payment gateway connection. We are continuing to monitor remaining recovery of usage calculations and alerting pipelines.
- resolved Oct 20, 2025, 11:55 PM UTC
All systems are operational.