Grafana incident

Alerting expressions pipeline failing when recovery settings

Resolved on August 8, 2026, affecting AWS Australia - prod-au-southeast-1: Rule Evaluation and AWS Australia - prod-au-southeast-1: Alertmanager and Rules Configuration API and 1 more.

resolved Aug 08, 2026, 07:48 PM UTC

Due to a software bug the evaluation of some alert rules (primarily ones that have recovery threshold setting) were failing to be evaluated starting 14:00 UTC to 19:30 UTC today.

Started
Aug 08, 2026, 07:48 PM UTC
Resolved
Aug 08, 2026, 07:48 PM UTC
Duration
–
Detected by Pingoru
Aug 08, 2026, 07:48 PM UTC

Affected components

AWS Australia - prod-au-southeast-1: Rule EvaluationAWS Australia - prod-au-southeast-1: Alertmanager and Rules Configuration APIAWS Australia - prod-au-southeast-1: AlertmanagerAWS Brazil - prod-sa-east-1: Rule EvaluationAWS Brazil - prod-sa-east-1: Alertmanager and Rules Configuration APIAWS Brazil - prod-sa-east-1: AlertmanagerAWS Canada - prod-ca-east-0: Rule EvaluationAWS Canada - prod-ca-east-0: Alertmanager and Rules Configuration APIAWS Canada - prod-ca-east-0: AlertmanagerAWS Germany - prod-eu-west-2: Rule Evaluation

Update timeline

  1. resolved Aug 08, 2026, 07:48 PM UTC

    Due to a software bug the evaluation of some alert rules (primarily ones that have recovery threshold setting) were failing to be evaluated starting 14:00 UTC to 19:30 UTC today.

Your product runs on more than Grafana

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →