AlertOps incident

AlertOps Delivery Thread Cancellation Issue

Minor Resolved View vendor source →

AlertOps experienced a minor incident on June 4, 2025, lasting —. The incident has been resolved; the full update timeline is below.

Started
Jun 04, 2025, 07:30 AM UTC
Resolved
Jun 04, 2025, 07:30 AM UTC
Duration
Detected by Pingoru
Jun 04, 2025, 07:30 AM UTC

Update timeline

  1. resolved Jun 04, 2025, 08:55 PM UTC

    During the impacted window, alerts that should have exited their escalation workflows continued notifying users up the escalation chain—even after being resolved—causing unnecessary alert noise and operational disruption.

  2. postmortem Jun 04, 2025, 08:58 PM UTC

    On June 4, AlertOps experienced a system-wide issue causing alert escalations to persist even after alerts were **assigned** or **closed**. As a result, users continued to receive escalation notifications despite taking action on those alerts. This issue was caused by a **code regression** that disrupted the logic responsible for halting escalation threads when alerts are updated with a resolved or assigned state.