Balena incident

Elevated Dashboard Errors

Minor Resolved View vendor source →

Balena experienced a minor incident on December 9, 2024 affecting BalenaOS Download, lasting 1h 46m. The incident has been resolved; the full update timeline is below.

Started
Dec 09, 2024, 06:44 PM UTC
Resolved
Dec 09, 2024, 08:31 PM UTC
Duration
1h 46m
Detected by Pingoru
Dec 09, 2024, 06:44 PM UTC

Affected components

BalenaOS Download

Update timeline

  1. investigating Dec 09, 2024, 06:44 PM UTC

    We're experiencing an elevated level of errors on our Dashboard and are currently looking into the issue.

  2. identified Dec 09, 2024, 07:05 PM UTC

    The issue has been identified and a fix is being implemented.

  3. monitoring Dec 09, 2024, 07:15 PM UTC

    A fix has been implemented and we are monitoring the results.

  4. monitoring Dec 09, 2024, 07:15 PM UTC

    We are continuing to monitor for any further issues.

  5. resolved Dec 09, 2024, 08:31 PM UTC

    This incident has been resolved.

  6. postmortem Dec 10, 2024, 06:46 PM UTC

    A configuration change to our API that enforced stricter policies for new tokens resulted in validation failures when the Dashboard was used to download balenaOS images. We quickly noticed the issue when our internal tests started failing, and we rolled back the API to the previous release. Once the changes have been retested against the current releases of the Dashboard they will be redeployed.