Omnivore experienced a critical incident on September 11, 2025 affecting API and Brink and 1 more component, lasting 2h 15m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Sep 11, 2025, 05:05 PM UTC
Around 16:47 UTC, the edge nodes serving the Omnivore API went offline. We are currently investigating and hope to provide a better update once we have more information.
- identified Sep 11, 2025, 05:12 PM UTC
We have identified the trigger to be a bad patch that was released. This patch caused all of our edge servers to become unhealthy. We are currently working to rollback the edge servers to a known good state.
- identified Sep 11, 2025, 05:13 PM UTC
We are continuing to work on a fix for this issue.
- identified Sep 11, 2025, 05:14 PM UTC
We are continuing to work on a fix for this issue.
- identified Sep 11, 2025, 05:45 PM UTC
We have partially restored some traffic to Omnivore API. We are still working on full resolution and stabilization.
- identified Sep 11, 2025, 06:45 PM UTC
We are seeing traffic approaching baseline levels. Webhooks have begun firing again. We are still working to improve stability.
- monitoring Sep 11, 2025, 06:53 PM UTC
A fix has been implemented, with all edge nodes now back in rotation and reporting healthy. We will continue to monitor for any further impacts.
- resolved Sep 11, 2025, 07:21 PM UTC
Omnivore API and webhooks have been restored to full functionality.