Datadog Govcloud incident

Delayed Monitors Notifications

Minor Resolved View vendor source →

Datadog Govcloud experienced a minor incident on September 20, 2024 affecting Monitors, lasting 13h 51m. The incident has been resolved; the full update timeline is below.

Started
Sep 20, 2024, 08:58 PM UTC
Resolved
Sep 21, 2024, 10:49 AM UTC
Duration
13h 51m
Detected by Pingoru
Sep 20, 2024, 08:58 PM UTC

Affected components

Monitors

Update timeline

  1. investigating Sep 20, 2024, 08:58 PM UTC

    We are investigating delays in Monitors Notifications for metric queries with time windows longer than 1 hour, which began at 7:35 PM UTC.

  2. monitoring Sep 20, 2024, 10:21 PM UTC

    We have deployed a fix and are currently monitoring the results. Further updates will be provided as we continue to make progress toward full recovery.

  3. monitoring Sep 20, 2024, 11:57 PM UTC

    The issue causing delays in Monitor Notifications for monitors with metric queries over time windows longer than one hour has been largely resolved. The impact is now limited to a small percentage of monitors affecting a subset of customers. We are closely monitoring the situation and will provide a final update once the issue is fully resolved.

  4. resolved Sep 21, 2024, 10:49 AM UTC

    This incident has been resolved.