Vero incident

Elevated API Errors

Major Resolved View vendor source →

Vero experienced a major incident on December 7, 2021 affecting Vero 1.0: Ingestion API, lasting 16h 48m. The incident has been resolved; the full update timeline is below.

Started
Dec 07, 2021, 04:49 PM UTC
Resolved
Dec 08, 2021, 09:37 AM UTC
Duration
16h 48m
Detected by Pingoru
Dec 07, 2021, 04:49 PM UTC

Affected components

Vero 1.0: Ingestion API

Update timeline

  1. investigating Dec 07, 2021, 04:49 PM UTC

    We're experiencing an elevated level of API errors and are currently looking into the issue.

  2. identified Dec 07, 2021, 05:05 PM UTC

    We've identified an issue with an upstream service provider and will update again when we have further information on a fix

  3. identified Dec 07, 2021, 05:53 PM UTC

    We've seen some improvements in API performance and error rates however we are still impacted at this time, our upstream service provider is working on a fix

  4. monitoring Dec 07, 2021, 07:39 PM UTC

    We're still waiting on a resolution from our upstream service provider, but thanks to their mitigation efforts we've seen a return to typical API performance and error rates in the past hour. We will continue to monitor for further issues.

  5. monitoring Dec 08, 2021, 12:33 AM UTC

    API performance and error rates have continued to be at normal levels over the past few hours. We are still awaiting confirmation of final resolution from our upstream service provider but in the meantime we will continue to monitor for further issues.

  6. resolved Dec 08, 2021, 09:37 AM UTC

    This incident has been resolved.