Datadog US3 incident

Delayed events for Logs, Synthetics and Container

Major Resolved View vendor source →

Datadog US3 experienced a major incident on July 2, 2024 affecting APM and Continuous Profiler and 1 more component, lasting 1h 3m. The incident has been resolved; the full update timeline is below.

Started
Jul 02, 2024, 03:56 PM UTC
Resolved
Jul 02, 2024, 05:00 PM UTC
Duration
1h 3m
Detected by Pingoru
Jul 02, 2024, 03:56 PM UTC

Affected components

APMContinuous ProfilerLog ManagementMetrics and Infra MonitoringRUMSynthetics

Update timeline

  1. investigating Jul 02, 2024, 03:56 PM UTC

    We are investigating increased latency processing Logs, Synthetics Test Results and Container updates. As a result of this issue, some users may see delays or gaps for data in their logs queries and synthetics tests status. To prevent spurious alerts, we have temporarily disabled monitors based on this data.

  2. identified Jul 02, 2024, 04:03 PM UTC

    The issue has been identified and a fix is being implemented.

  3. investigating Jul 02, 2024, 04:05 PM UTC

    We are continuing to investigate this issue.

  4. identified Jul 02, 2024, 04:05 PM UTC

    The issue has been identified and a fix is being implemented.

  5. monitoring Jul 02, 2024, 04:22 PM UTC

    A fix has been implemented and we are monitoring the results.

  6. resolved Jul 02, 2024, 05:00 PM UTC

    This incident has been resolved.