Charts and Detectors are Intermittently Failing to Load

Update timeline

1. monitoring Mar 19, 2026, 07:37 PM UTC

   Mar 19, 12:37 PDT Resolved - This incident has been resolved. Mar 19, 12:32 PDT Update - We have successfully reverted the rate limits to their previous settings and continue to monitor for any issues. Mar 19, 11:57 PDT Update - We continue to monitor as we slowly reduce the implemented rate limits that partially mitigated the impact to ensure the environment remains stable. Mar 19, 11:25 PDT Monitoring - The fix has been implemented and we are monitoring the results. Mar 19, 11:08 PDT Update - Engineering teams continue to apply the following mitigation actions: Fine tuning internal query routing to healthy MTS Store Availability Zones with capacity and limiting expensive and high volume customer queries in order to bring general stability and support the success of scale-out operations. Mar 19, 10:28 PDT Identified - Charts and Detectors are intermittently failing. Infrastructure monitoring might also see impact. Datapoint Ingest is not impacted. The issue has been identified and a mitigation fix is being implemented. Mar 19, 09:52 PDT Investigating - Charts and Detectors are intermittently failing. Datapoint ingest is not affected. We are currently investigating.