Resolved after 1 hour 1 minute, affecting API Queries, on September 11, 2026.
# **Root Cause Analysis: Query Latency on September 11, 2026** Region: Azure West Europe Cluster: prod01-eu-west-1 **Incident report: query latency in Azure West Europe, September 11, 2026** On the morning of September 11, some customers in Azure West Europe saw slower than usual query responses. Read latency was elevated and there were occasional query timeouts. Performance returned to normal at 08:52 UTC. We're sorry for the disruption. **What happened** On September 10, a single workload ran a sustained volume of deletes. That activity left a large backlog of delete markers on every storage partition in the region. Delete markers are normally cleared by a scheduled delete-compaction process. Until that process ran, each hourly retention pass had to work through the full backlog. Passes that usually take about three minutes were taking 10 to 15 minutes across the cluster. While a partition was in one of these longer passes it responded more slowly, and queries that spanned it were delayed. At 07:55 UTC on September 11, one storage partition entered a self-recovery restart. Its companion replica was in the middle of a retention pass at the time, so for a short period that partition had no healthy replica available to serve queries. This was not the root cause, but the overlap added to the impact on queries touching that partition. Alerts on query queue depth fired at 08:14 and we opened an incident. An operator restarted the companion replica at 08:25. The impaired partition completed its recovery at 08:47. The query queue then drained without further intervention, and by 08:52 it was back to normal and the alerts had cleared. **Timeline** \(UTC\) Time Event Sep 10 - Sustained high delete volume from a single workload Sep 11, 07:55 - Storage partition restarted and entered extended recovery 08:14 - Query queue-depth alerts triggered; incident opened 08:25 - Companion replica restarted 08:47 - Impaired partition completed recovery 08:52 - Query queue returned to normal; alerts cleared **Changes we are making** There are four pieces of work under way. None of them is complete yet, and we won't consider them validated until we have confirmed the effect in production. * Retention enforcement is moving to an off-peak window. Longer passes should then fall outside business-hours query load. * Full compactions will run more frequently, so a delete backlog can clear between scheduled runs instead of accumulating. * Per-partition compaction throughput is being increased. * New alerting and telemetry will tell us whether the changes above are having the intended effect.
Affected components
Update timeline
- identified Sep 11, 2026, 08:45 AM UTC
The issue has been identified and a fix is being implemented.
- monitoring Sep 11, 2026, 09:27 AM UTC
A fix has been implemented and we are monitoring the results.
- resolved Sep 11, 2026, 09:46 AM UTC
This incident has been resolved.
- postmortem Sep 28, 2026, 01:14 PM UTC
# **Root Cause Analysis: Query Latency on September 11, 2026** Region: Azure West Europe Cluster: prod01-eu-west-1 **Incident report: query latency in Azure West Europe, September 11, 2026** On the morning of September 11, some customers in Azure West Europe saw slower than usual query responses. Read latency was elevated and there were occasional query timeouts. Performance returned to normal at 08:52 UTC. We're sorry for the disruption. **What happened** On September 10, a single workload ran a sustained volume of deletes. That activity left a large backlog of delete markers on every storage partition in the region. Delete markers are normally cleared by a scheduled delete-compaction process. Until that process ran, each hourly retention pass had to work through the full backlog. Passes that usually take about three minutes were taking 10 to 15 minutes across the cluster. While a partition was in one of these longer passes it responded more slowly, and queries that spanned it were delayed. At 07:55 UTC on September 11, one storage partition entered a self-recovery restart. Its companion replica was in the middle of a retention pass at the time, so for a short period that partition had no healthy replica available to serve queries. This was not the root cause, but the overlap added to the impact on queries touching that partition. Alerts on query queue depth fired at 08:14 and we opened an incident. An operator restarted the companion replica at 08:25. The impaired partition completed its recovery at 08:47. The query queue then drained without further intervention, and by 08:52 it was back to normal and the alerts had cleared. **Timeline** \(UTC\) Time Event Sep 10 - Sustained high delete volume from a single workload Sep 11, 07:55 - Storage partition restarted and entered extended recovery 08:14 - Query queue-depth alerts triggered; incident opened 08:25 - Companion replica restarted 08:47 - Impaired partition completed recovery 08:52 - Query queue returned to normal; alerts cleared **Changes we are making** There are four pieces of work under way. None of them is complete yet, and we won't consider them validated until we have confirmed the effect in production. * Retention enforcement is moving to an off-peak window. Longer passes should then fall outside business-hours query load. * Full compactions will run more frequently, so a delete backlog can clear between scheduled runs instead of accumulating. * Per-partition compaction throughput is being increased. * New alerting and telemetry will tell us whether the changes above are having the intended effect.
Your product runs on more than InfluxData
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services