Avochato incident

Issues with Telecom Provider

Critical Resolved View vendor source →

Avochato experienced a critical incident on February 26, 2021 affecting avochato.com and API and 1 more component, lasting 2h 58m. The incident has been resolved; the full update timeline is below.

Started
Feb 26, 2021, 01:30 PM UTC
Resolved
Feb 26, 2021, 04:28 PM UTC
Duration
2h 58m
Detected by Pingoru
Feb 26, 2021, 01:30 PM UTC

Affected components

avochato.comAPIMobile

Update timeline

  1. identified Feb 26, 2021, 03:02 PM UTC

    We've identified an issue with our Telecom Provider where outbound messages are disabled.

  2. identified Feb 26, 2021, 03:04 PM UTC

    Calls seem to be unaffected, however sending SMS and MMS are experiencing errors. We're in direct contact with our Telecom Provider to resolve this ASAP.

  3. monitoring Feb 26, 2021, 03:39 PM UTC

    We've stated seeing success sending outbound SMS and MMS through our Telecom Providers. We'll continue monitoring the situation until it fully improves.

  4. resolved Feb 26, 2021, 04:28 PM UTC

    We've seen most Telecom Errors errors resolve for the past hour. We're continuing to monitor the situation and will update as we know more.

  5. postmortem Mar 11, 2021, 11:18 PM UTC

    ## What Happened One of our third-party software vendors suffered a platform-wide outage which prevented us from leveraging their APIs to deliver SMS and Voice functionality. During the impact period, we were not able to leverage their APIs for sending messages and managing Avochato Numbers, as well as routing calls. Attempts to send messages were queued and retried until we could successfully deliver the messages. Customers sending traffic through our other SMS and Voice partners were not impacted. ## Resolution The vendor performed their procedures for triaging and resolving the incident, and successfully returned to full operational capability.