Hosted Mender incident

Deployments service overloaded

Minor Resolved View vendor source →

Hosted Mender experienced a minor incident on September 7, 2023 affecting Hosted Mender US, lasting 1h 9m. The incident has been resolved; the full update timeline is below.

Started
Sep 07, 2023, 03:09 PM UTC
Resolved
Sep 07, 2023, 04:19 PM UTC
Duration
1h 9m
Detected by Pingoru
Sep 07, 2023, 03:09 PM UTC

Affected components

Hosted Mender US

Update timeline

  1. investigating Sep 07, 2023, 03:09 PM UTC

    The Hosted Mender Deployments service is under high pressure. We are seeing an unusual usage pattern in the Deployment service. The autoscaler is handling it and adding more resources to keep usage at an acceptable level, but performance could be degraded. We are currently investigating the issue.

  2. identified Sep 07, 2023, 03:34 PM UTC

    The issue has been identified and a fix is being implemented

  3. monitoring Sep 07, 2023, 03:45 PM UTC

    A fix has been implemented and we are monitoring the results.

  4. resolved Sep 07, 2023, 04:19 PM UTC

    This incident has been resolved.