Chord incident

Chord Hub Timeouts

Critical Resolved View vendor source →

Chord experienced a critical incident on May 30, 2023 affecting Hub, lasting 2h 26m. The incident has been resolved; the full update timeline is below.

Started
May 30, 2023, 03:30 PM UTC
Resolved
May 30, 2023, 05:56 PM UTC
Duration
2h 26m
Detected by Pingoru
May 30, 2023, 03:30 PM UTC

Affected components

Hub

Update timeline

  1. investigating May 30, 2023, 05:17 PM UTC

    Some analytics requests from the Chord Hub are currently timing out for some customers.

  2. identified May 30, 2023, 05:19 PM UTC

    Our engineers are currently investigating root cause, and requests to a specific Redis cache are timing out, causing a cascade of timeouts downstream. We are currently working on a fix for this.

  3. resolved May 30, 2023, 05:56 PM UTC

    This incident has been resolved.