SondeHub incident

OpenSearch outage

Resolved after 23 hours 10 minutes, affecting API and Frontend and 1 more, on September 30, 2021.

resolved Oct 01, 2021, 12:03 PM UTC

The workaround appears to be working our DB vendor is working to resolve the problem.

Started
Sep 30, 2021, 12:52 PM UTC
Resolved
Oct 01, 2021, 12:03 PM UTC
Duration
23h 10m
Detected by Pingoru
Sep 30, 2021, 12:52 PM UTC

Affected components

APIFrontendPredictions

Update timeline

  1. identified Sep 30, 2021, 12:52 PM UTC

    Performance issues occurring from earlier opensearch upgrade

  2. identified Oct 01, 2021, 07:58 AM UTC

    We are working with our DB vendor to resolve the issue however this may take sometime. Live view should continue to operate normally however predictions and history are likely not performing well.

  3. monitoring Oct 01, 2021, 09:32 AM UTC

    A workaround has been implemented and we are monitoring to investigate if it has worked

  4. resolved Oct 01, 2021, 12:03 PM UTC

    The workaround appears to be working our DB vendor is working to resolve the problem.

Your product runs on more than SondeHub

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →