Grafana incident

Metrics Alert Rule Evaluation Failures in AWS Australia (prod-au-southeast-1)

Resolved after 1 hour 48 minutes, affecting AWS Australia - prod-au-southeast-1: Rule Evaluation, on November 2, 2026.

resolved Nov 02, 2026, 11:08 AM UTC

Between approximately 08:00 and 09:40 UTC (about 1 hour 40 minutes), some customers in AWS Australia (prod-au-southeast-1) experienced intermittent failures when evaluating alert and recording rules, which may have caused missed or delayed alert notifications and gaps in recorded series. This has been fully resolved and rule evaluation is operating normally.

Started
Nov 02, 2026, 09:20 AM UTC
Resolved
Nov 02, 2026, 11:08 AM UTC
Duration
1h 48m
Detected by Pingoru
Nov 02, 2026, 09:20 AM UTC

Affected components

AWS Australia - prod-au-southeast-1: Rule Evaluation

Update timeline

  1. investigating Nov 02, 2026, 09:20 AM UTC

    We are investigating an elevated rate of failures when evaluating Metrics alert and recording rules in AWS Australia (prod-au-southeast-1). Some customers may see rules missed or delayed rule evaluations, which can result in missed or delayed alert notifications and gaps in recorded series. Next update by 10:20 UTC.

  2. investigating Nov 02, 2026, 10:17 AM UTC

    We continue to investigate intermittent failures when evaluating alert and recording rules in AWS Australia (prod-au-southeast-1), first observed at approximately 08:00 UTC. Some customers may see rules fail to evaluate or be delayed, which can result in missed or delayed alert notifications and gaps in recorded series. We are continuing to investigate. Next update by 11:15 UTC.

  3. resolved Nov 02, 2026, 11:08 AM UTC

    Between approximately 08:00 and 09:40 UTC (about 1 hour 40 minutes), some customers in AWS Australia (prod-au-southeast-1) experienced intermittent failures when evaluating alert and recording rules, which may have caused missed or delayed alert notifications and gaps in recorded series. This has been fully resolved and rule evaluation is operating normally.

Your product runs on more than Grafana

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →