Camunda incident

Cluster creation and updates slow in northamerica-northeast2

Notice Resolved View vendor source →

Camunda experienced a notice incident on February 29, 2024, lasting 36m. The incident has been resolved; the full update timeline is below.

Started
Feb 29, 2024, 04:53 PM UTC
Resolved
Feb 29, 2024, 05:30 PM UTC
Duration
36m
Detected by Pingoru
Feb 29, 2024, 04:53 PM UTC

Update timeline

  1. investigating Feb 29, 2024, 04:53 PM UTC

    We are currently investigating slow Camunda cluster creation and updates. Running workloads are unaffected. Our Kubernetes Control Plane has slow response times to update requests.

  2. identified Feb 29, 2024, 05:26 PM UTC

    We've identified an erroneous mutating and validating webhook configuration that created many requests against the Control Plane.

  3. monitoring Feb 29, 2024, 05:27 PM UTC

    Cluster creation and updates are back to normal after we fixed the configuration of the validating and mutating webhooks.

  4. resolved Mar 01, 2024, 07:57 AM UTC

    This incident has been resolved.