DFDS IT incident

Prometheus is down due to a node in NotReady state

Notice Resolved View vendor source →

DFDS IT experienced a notice incident on May 9, 2022 affecting Kubernetes [Hellman] - Capacity/Scheduling, lasting 1h 43m. The incident has been resolved; the full update timeline is below.

Started
May 09, 2022, 01:05 PM UTC
Resolved
May 09, 2022, 02:48 PM UTC
Duration
1h 43m
Detected by Pingoru
May 09, 2022, 01:05 PM UTC

Affected components

Kubernetes [Hellman] - Capacity/Scheduling

Update timeline

  1. identified May 09, 2022, 01:05 PM UTC

    The issue has been identified and a fix is being implemented.

  2. monitoring May 09, 2022, 01:15 PM UTC

    A new node has been spawned, and workloads migrated over. Prometheus might still run a bit unstable, and we will continue to monitor the situation until it has stabilized.

  3. resolved May 09, 2022, 02:48 PM UTC

    This incident has been resolved.