Datadog US1 Outage History

This incident has been resolved.

This incident has been resolved.

Apr 15, 20:37 EDT Resolved - This incident has been resolved. Apr 15, 20:32 EDT Monitoring - We have resolved the lag in distribution metrics and we are monitoring the results. We will provide another update once the issue is fully resolved. Apr 15, 19:54 EDT Update - We are still investigating increased latency processing Distribution Metrics, delays have gone down but have not fully recovered. Apr 15, 19:16 EDT Update - We are investigating increased latency processing Distribution Metrics. As a result of this issue, some users may see delays or gaps for distribution metrics on graphs. To prevent false monitor alerts due to delayed data, monitors affected by the delay will not notify and will automatically resume once current data is available. All other monitors will operate normally. Apr 15, 18:38 EDT Investigating - We are investigating increased latency processing Metrics. As a result of this issue, some users may see delays or gaps for metrics on graphs. To prevent false monitor alerts due to delayed data, monitors affected by the delay will not notify and will automatically resume once current data is available. All other monitors will operate normally.

Apr 10, 16:59 EDT Resolved - Between approximately 9:42 AM and 10:38 AM ET on April 10, we observed delivery failures for customers in the US1, AP1, and AP2 regions. During this window, notifications to the following integrations were delayed or temporarily undelivered: • PagerDuty • Slack • Microsoft Teams • Webhooks • Jira • ServiceNow • OpsGenie • VictorOps • BigPanda • Zendesk • Sumologic All delayed notifications have been replayed with the following exceptions: PagerDuty pages and Slack notifications queued during the outage window were not replayed, as they were no longer actionable at the time of recovery. Webhook notifications will be reprocessed and delivered at a later date. No monitoring data was lost during this incident. Monitor evaluations continued normally — only the delivery of triggered notifications was affected. Thank you for your patience. Apr 10, 16:13 EDT Update - Webhooks and Slack notifications are still catching up. Apr 10, 15:44 EDT Update - Webhooks and Slack notifications are still catching up. Apr 10, 15:02 EDT Update - Webhooks and Slack notifications are still catching up. Apr 10, 14:33 EDT Update - Notifications for ServiceNow and Microsoft Teams are now caught up. Webhooks and Slack notifications are still catching up. Apr 10, 13:37 EDT Update - Notifications for JIRA are now caught up. Apr 10, 13:12 EDT Update - Notifications for Opsgenie, VictorOps, BigPanda, Zendesk, and Sumologic are caught up. Webhooks, Microsoft Teams, Slack, JIRA, and ServiceNow are still catching up. Apr 10, 12:15 EDT Update - We are still catching-up on past notifications that may have been delayed. Apr 10, 11:59 EDT Update - We are still catching-up on past notifications that may have been delayed. Apr 10, 11:42 EDT Monitoring - New notifications are now successfully delivered, and we are catching-up on past notifications that may have been delayed Apr 10, 11:42 EDT Update - New notifications are now successfully delivered, and we are catching-up on past notifications that may have been delayed. Apr 10, 11:23 EDT Update - We are continuing to work on a fix for this issue. Apr 10, 11:00 EDT Identified - The issue has been identified and a fix is being implemented. Apr 10, 10:54 EDT Investigating - We are investigating an issue to send notifications to Pagerduty, VictorOps, Slack, Webhook, Microsoft Teams and OpsGenie.

Mar 27, 15:30 EDT Resolved - This incident has been resolved. Mar 27, 15:21 EDT Monitoring - A fix has been implemented and we are monitoring the results. Mar 27, 15:17 EDT Identified - The issue has been identified and a fix is being implemented. Mar 27, 14:57 EDT Investigating - We are investigating increased latency in processing and storing Traces in APM. As a result of this issue, some users may see missing or delayed traces in APM Trace Search since 5pm UTC. They may also experience delay in APM trace-based monitors.

Mar 26, 15:56 EDT Resolved - This incident has been resolved. Mar 26, 15:42 EDT Monitoring - A fix has been implemented and we are monitoring the results. Mar 26, 15:07 EDT Update - We are continuing to work on a fix for this issue. Mar 26, 14:38 EDT Update - We are continuing to work on a fix for this issue. Mar 26, 13:28 EDT Update - We are continuing to work on a fix for this issue. Mar 26, 12:48 EDT Identified - The issue has been identified and a fix is being implemented. Mar 26, 12:11 EDT Update - We are continuing to investigate delays in monitor notifications. Mar 26, 11:32 EDT Update - We are still continuing investigating delays in monitor notifications that started at 12:00 UTC. We've identified the root cause and are working on a fix. Mar 26, 10:55 EDT Update - We are continuing investigating delays in monitor notifications that started at 12:00 UTC. Mar 26, 10:24 EDT Investigating - We are investigating delays in monitor notifications, which began at 12:00 UTC.

Mar 18, 22:53 EDT Resolved - This incident has been resolved. Mar 18, 22:39 EDT Monitoring - A fix has been implemented and we are monitoring the results. Mar 18, 22:04 EDT Update - We are continuing to investigate the issue and will provide updates as available. Mar 18, 21:27 EDT Update - We are continuing to investigate the issue and will provide updates as available. Mar 18, 20:54 EDT Investigating - We are investigating an issue causing some metric monitors to intermittently report "No Data" status. This primarily affects monitors based on distribution metrics. Monitor alerting may be unreliable during this time. We are actively investigating and will provide updates as available.

Mar 13, 06:30 EDT Resolved - This incident has been resolved. Mar 13, 06:05 EDT Monitoring - A fix has been implemented and we are monitoring the results. Mar 13, 06:00 EDT Identified - The issue has been identified and a fix is being implemented. Mar 13, 05:52 EDT Update - We are continuing to investigate this issue. Mar 13, 05:32 EDT Update - We are continuing to investigate this issue. Mar 13, 05:27 EDT Update - We are continuing to investigate this issue. Mar 13, 05:26 EDT Investigating - We are investigating loading issues on our web application. As a result, some users might be getting errors when loading the web application.

Feb 24, 02:08 EST Resolved - This incident has been resolved. Feb 24, 00:52 EST Monitoring - We have deployed a fix and we are monitoring the results. We will provide another update once the issue is fully resolved. Feb 24, 00:29 EST Identified - We have identified the underlying issue and are working on a fix. It is important to note that no data has been lost, and it will be backfilled and available once the service is operational again. Feb 23, 23:31 EST Investigating - We are investigating increased latency processing RUM data. As a result of this issue, some users may see gaps or delays in RUM graphs as well as empty or partial query results on RUM Sessions, RUM Analytics, RUM Application, and Error Tracking pages.

Feb 18, 15:37 EST Resolved - This incident has been resolved. Feb 18, 14:30 EST Monitoring - We are investigating delays in a subset of Distribution Metrics and Monitor Evaluations, which began at 18:33 UTC.

Feb 11, 21:08 EST Resolved - This incident has been resolved. Feb 11, 20:49 EST Update - The remaining impact is limited to AWS Streams metrics. We are continuing to monitor for any further issues. Feb 11, 20:32 EST Monitoring - A fix has been implemented and we are monitoring the results. Feb 11, 20:20 EST Identified - The issue has been identified and a fix is being implemented. Feb 11, 19:57 EST Investigating - We are investigating increased latency processing Traces in APM Trace Search and AWS streams metrics. As a result of this issue, some users may see missing or delayed traces in APM Trace Search, and delayed AWS stream metrics since 00:13 UTC.

Feb 5, 12:43 EST Resolved - This incident has been resolved. Feb 5, 12:23 EST Monitoring - We have observed full recovery of monitors and will continue to monitor Feb 5, 12:01 EST Update - We are observing recovery for the vast majority of monitors and are continuing to work on a full fix for this issue. Feb 5, 11:50 EST Update - We are continuing to work on a fix for this issue. Feb 5, 11:31 EST Identified - The issue has been identified and a fix is being implemented. Feb 5, 11:28 EST Update - We are continuing to investigate this issue. Feb 5, 11:11 EST Update - We are continuing to investigate this issue. Feb 5, 11:10 EST Update - We are continuing to investigate this issue. Feb 5, 11:04 EST Update - We are continuing to investigate this issue. Feb 5, 10:24 EST Investigating - We are currently investigating this issue.