Camunda incident

Critical bug in 8.6+Gen1

Major Resolved View vendor source →

Camunda experienced a major incident on October 11, 2024 affecting Zeebe, lasting 6h 3m. The incident has been resolved; the full update timeline is below.

Started
Oct 11, 2024, 11:33 AM UTC
Resolved
Oct 11, 2024, 05:36 PM UTC
Duration
6h 3m
Detected by Pingoru
Oct 11, 2024, 11:33 AM UTC

Affected components

Zeebe

Update timeline

  1. identified Oct 11, 2024, 11:33 AM UTC

    An issue has been identified as a critical bug in the 8.6+Gen1 generation in Zeebe. This can potentially lead to data corruption. We have not detected any data corruption yet and we consider the likelihood of a data corruption low, but we want to proactively prevent this through a hotfix. We have disabled the creation of new clusters using this generation until the hotfix is available. We are working to mitigate the issue and will provide an update in the next 1hr. Affected Versions: 8.6+Gen1 Affected Regions: Belgium, Europe (europe-west1), London, Europe (europe-west2), South Carolina, North America (us-east1), Iowa, North America (us-central1), Toronto, North America (northamerica-northeast2), Sydney, Australia (australia-southeast1)

  2. identified Oct 11, 2024, 11:38 AM UTC

    We are continuing to work on a fix for this issue.

  3. identified Oct 11, 2024, 03:35 PM UTC

    We are preparing a new release fixing the issue.

  4. resolved Oct 11, 2024, 05:36 PM UTC

    A new generation `Camunda 8.6.0+gen2`, containing Zeebe 8.6.2 with the fix, is available. We recommend to update clusters running `Camunda 8.6.0+gen1` to `Camunda 8.6.0+gen2` asap. We will trigger automatic updates to `Camunda 8.6.0+gen2` for all clusters that have automatic updates enabled. We apologize for any inconvenience caused and thank you for your patience.