Avochato incident

East Coast platform latency

Minor Resolved View vendor source →

Avochato experienced a minor incident on November 23, 2020 affecting avochato.com and API and 1 more component, lasting 6h 1m. The incident has been resolved; the full update timeline is below.

Started
Nov 23, 2020, 05:20 PM UTC
Resolved
Nov 23, 2020, 11:21 PM UTC
Duration
6h 1m
Detected by Pingoru
Nov 23, 2020, 05:20 PM UTC

Affected components

avochato.comAPIMobile

Update timeline

  1. identified Nov 23, 2020, 05:20 PM UTC

    We are aware of an infrastructure issue that is causing slower than average page times for US East Coast users. We are working on a resolution.

  2. monitoring Nov 23, 2020, 06:01 PM UTC

    We have made an infrastructure update which appears to have resolved latency impacting East Coast customers. We will continue to monitor page load times for impacted routes.

  3. resolved Nov 23, 2020, 11:21 PM UTC

    This incident has been resolved.

  4. postmortem Nov 25, 2020, 08:23 PM UTC

    ## What happened Our East Coast cloud infrastructure was routing requests to West Coast databases, sometimes multiple trips for a single request. This caused delays for customers whose DNS were automatically routed to the East Coast, as well as network requests from API servers with an East Coast region. Messages and application load times were delayed for customers closer to the East Coast region than the West Coast. ## Resolution We altered the threshold for sending traffic to the East Coast data center. We have rolled back networking changes to East Coast infrastructure and systems have returned to normal