Grafana incident
Prometheus writes in prod-eu-west-3 are degraded
Grafana experienced a critical incident on March 25, 2026 affecting Azure Netherlands - prod-eu-west-3: Ingestion, lasting 29d 5h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Mar 25, 2026, 02:11 PM UTC
The metric writes issue reported in https://status.grafana.com/incidents/gfshj17lxj5z is still ongoing. Our Engineering team is actively investigating this and we will provide further updates as our investigation progresses.
- investigating Mar 25, 2026, 09:35 PM UTC
We are continuing to investigate this issue.
- monitoring Mar 26, 2026, 12:04 PM UTC
A fix has been implemented and we are monitoring the results.
- monitoring Mar 26, 2026, 05:45 PM UTC
We are continuing to monitor the previously impacted environments.
- monitoring Mar 27, 2026, 09:05 PM UTC
We are continuing to monitor this through the weekend.
- monitoring Apr 02, 2026, 09:38 PM UTC
We are continuing to monitor for any further issues.
- monitoring Apr 08, 2026, 08:32 PM UTC
We are still seeing intermittent issues and continue to seek a resolution
- monitoring Apr 14, 2026, 08:11 PM UTC
We have deployed mitigation and seen improvement in write failures over the past week. We are still seeing intermittent spikes in latency and continue to monitor.
- monitoring Apr 20, 2026, 03:08 PM UTC
We are continuing to monitor for any further issues.
- resolved Apr 23, 2026, 08:07 PM UTC
This incident has been resolved. Thank you for your patience.