Omnivore incident

Omnivore API Outage

Resolved after 2 hours 15 minutes, affecting API and Brink and 1 more, on September 11, 2025.

monitoring Sep 11, 2025, 06:53 PM UTC

A fix has been implemented, with all edge nodes now back in rotation and reporting healthy. We will continue to monitor for any further impacts.

Started
Sep 11, 2025, 05:05 PM UTC
Resolved
Sep 11, 2025, 07:21 PM UTC
Duration
2h 15m
Detected by Pingoru
Sep 11, 2025, 05:05 PM UTC

Affected components

APIBrinkWebhooksAloha Cloud ConnectControl PanelToastLavuLightspeedDoshiiSimphony Cloud

Update timeline

  1. investigating Sep 11, 2025, 05:05 PM UTC

    Around 16:47 UTC, the edge nodes serving the Omnivore API went offline. We are currently investigating and hope to provide a better update once we have more information.

  2. identified Sep 11, 2025, 05:12 PM UTC

    We have identified the trigger to be a bad patch that was released. This patch caused all of our edge servers to become unhealthy. We are currently working to rollback the edge servers to a known good state.

  3. identified Sep 11, 2025, 05:13 PM UTC

    We are continuing to work on a fix for this issue.

  4. identified Sep 11, 2025, 05:14 PM UTC

    We are continuing to work on a fix for this issue.

  5. identified Sep 11, 2025, 05:45 PM UTC

    We have partially restored some traffic to Omnivore API. We are still working on full resolution and stabilization.

  6. identified Sep 11, 2025, 06:45 PM UTC

    We are seeing traffic approaching baseline levels. Webhooks have begun firing again. We are still working to improve stability.

  7. monitoring Sep 11, 2025, 06:53 PM UTC

    A fix has been implemented, with all edge nodes now back in rotation and reporting healthy. We will continue to monitor for any further impacts.

  8. resolved Sep 11, 2025, 07:21 PM UTC

    Omnivore API and webhooks have been restored to full functionality.

Your product runs on more than Omnivore

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →