Grafana incident

Trace querying issue in all Tempo clusters

Minor Resolved View vendor source →

Grafana experienced a minor incident on February 27, 2026 affecting AWS Australia - prod-ap-southeast-2: Querying and AWS Brazil - prod-sa-east-1: Querying and 1 more component, lasting 9h 51m. The incident has been resolved; the full update timeline is below.

Started
Feb 27, 2026, 01:46 PM UTC
Resolved
Feb 27, 2026, 11:38 PM UTC
Duration
9h 51m
Detected by Pingoru
Feb 27, 2026, 01:46 PM UTC

Affected components

AWS Australia - prod-ap-southeast-2: QueryingAWS Brazil - prod-sa-east-1: QueryingAWS Canada - prod-ca-east-0: QueryingAWS Germany - prod-eu-west-2: QueryingAWS India - prod-ap-south-1: QueryingAWS Japan - prod-ap-northeast-0: QueryingAWS Singapore - prod-ap-southeast-1: QueryingAWS Sweden - prod-eu-north-0: QueryingAWS US East - prod-us-east-0: QueryingAWS US West - prod-us-west-0: Querying

Update timeline

  1. investigating Feb 27, 2026, 01:46 PM UTC

    We're currently working on an issue where portions of data may be temporarily unretrievable, affecting a small percentage of tenants in all Tempo clusters.

  2. identified Feb 27, 2026, 07:27 PM UTC

    Our team has identified the issue, and are in the process of testing a fix.

  3. resolved Feb 27, 2026, 11:38 PM UTC

    This incident has been resolved.