Discord Extreme List incident

Cannot Connect to MongoDB Replicaset

Critical Resolved View vendor source →

Discord Extreme List experienced a critical incident on August 25, 2020 affecting Website and API and 1 more component, lasting 3h 59m. The incident has been resolved; the full update timeline is below.

Started
Aug 25, 2020, 12:58 PM UTC
Resolved
Aug 25, 2020, 04:58 PM UTC
Duration
3h 59m
Detected by Pingoru
Aug 25, 2020, 12:58 PM UTC

Affected components

WebsiteAPIMongoDB Replicaset

Update timeline

  1. investigating Aug 25, 2020, 12:58 PM UTC

    We are currently investigating this issue.

  2. investigating Aug 25, 2020, 01:08 PM UTC

    We are continuing to investigate this issue.

  3. resolved Aug 25, 2020, 04:58 PM UTC

    This incident has been resolved.

  4. postmortem Aug 25, 2020, 04:58 PM UTC

    When in production, it might be a wise idea to prevent pods \(that can use hostPath volumes\)! I applied some pod security policies to help solve this problem. As many know, eu2 had an unknown issue, which caused it to not reboot, but become un-schedulable, resulting in the policies not applying. The way I fixed this, was to get rid of pod security policies, the resources will still be on the servers, but when time comes to use them again, I will.