Hosted Mender incident

Problems with device authentication and inventory

Major Resolved View vendor source →

Hosted Mender experienced a major incident on April 23, 2024 affecting Hosted Mender US, lasting 1h 27m. The incident has been resolved; the full update timeline is below.

Started
Apr 23, 2024, 06:36 PM UTC
Resolved
Apr 23, 2024, 08:03 PM UTC
Duration
1h 27m
Detected by Pingoru
Apr 23, 2024, 06:36 PM UTC

Affected components

Hosted Mender US

Update timeline

  1. investigating Apr 23, 2024, 06:36 PM UTC

    We are investigating reports of authentication and inventory services failing for devices.

  2. monitoring Apr 23, 2024, 07:43 PM UTC

    We have drained the message queue, which had gotten stuck, and things seems to have improved. Will keep monitoring to ensure it does not accumulate again.

  3. resolved Apr 23, 2024, 08:03 PM UTC

    We are seeing no further evidence of message buildup. Closing incident.