Hosted Mender incident

Sporadic 500 errors on the Deployment service

Major Resolved View vendor source →

Hosted Mender experienced a major incident on December 21, 2022 affecting Hosted Mender US, lasting 3h 11m. The incident has been resolved; the full update timeline is below.

Started
Dec 21, 2022, 12:50 PM UTC
Resolved
Dec 21, 2022, 04:02 PM UTC
Duration
3h 11m
Detected by Pingoru
Dec 21, 2022, 12:50 PM UTC

Affected components

Hosted Mender US

Update timeline

  1. investigating Apr 18, 2023, 08:00 PM UTC

    Our monitor confirm that we're experiencing an usual 500 error on the Deployment service. We are currently investigating the issue.

  2. identified Apr 18, 2023, 08:01 PM UTC

    The issue has been identified and a fix is being implemented.

  3. monitoring Apr 18, 2023, 08:02 PM UTC

    A fix has been implemented and we are monitoring the results.

  4. identified Apr 18, 2023, 08:02 PM UTC

    The fix didn't resolved the issue completely, and errors in the Deployment service persists. We identified the issue, and we're resolving it.

  5. identified Apr 18, 2023, 08:02 PM UTC

    The fix didn't resolved the issue completely, and errors in the Deployment service persists. We identified the issue, and we're resolving it. The symptom is the Releases no being visible, and error message: "Please check your connection internal error"

  6. monitoring Apr 18, 2023, 08:03 PM UTC

    A fix has been implemented: the Releases should be visible again now. We are still monitoring the results.

  7. resolved Apr 18, 2023, 08:03 PM UTC

    This incident has been resolved.