Discord Extreme List incident

Kubernetes Issues

Critical Resolved View vendor source →

Discord Extreme List experienced a critical incident on December 17, 2021 affecting Website and API and 1 more component, lasting 5d 14h. The incident has been resolved; the full update timeline is below.

Started
Dec 17, 2021, 10:10 PM UTC
Resolved
Dec 23, 2021, 01:02 PM UTC
Duration
5d 14h
Detected by Pingoru
Dec 17, 2021, 10:10 PM UTC

Affected components

WebsiteAPILoad Balancer

Update timeline

  1. identified Dec 17, 2021, 10:10 PM UTC

    Kubernetes is returning 401 for our services.

  2. identified Dec 21, 2021, 08:43 AM UTC

    One of our nodes is not currently being affected by this Kubernetes issue, it will start handling traffic as soon as routes propagate.

  3. monitoring Dec 23, 2021, 11:49 AM UTC

    Nodes were misconfigured and invalid certificates were issued, we are monitoring the results hoping the cluster recovers soon.

  4. resolved Dec 23, 2021, 01:02 PM UTC

    This incident has been resolved.