InfluxData Outage History

Started Nov 20, 2025, 09:03 PM UTC· resolved Nov 20, 2025, 09:39 PM UTC

1. investigating Nov 20, 2025, 09:03 PM UTC

   We are investigating an issue in the Serverless UI that is preventing measurements from being displayed in the dropdown list, or queried.

2. investigating Nov 20, 2025, 09:33 PM UTC

   We are continuing to investigate this issue.

3. investigating Nov 20, 2025, 09:37 PM UTC

   We are continuing to investigate this issue.

4. identified Nov 20, 2025, 09:38 PM UTC

   The issue has been identified and a fix is being implemented.

5. resolved Nov 20, 2025, 09:39 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Nov 20, 2025, 08:52 AM UTC· resolved Nov 20, 2025, 02:07 PM UTC

1. investigating Nov 20, 2025, 08:52 AM UTC

   We have been made aware of issues with Google OAuth client, we are working with provider to solve the issue.

2. investigating Nov 20, 2025, 08:54 AM UTC

   We are continuing to investigate this issue.

3. identified Nov 20, 2025, 09:33 AM UTC

   We have tested extensively and found that Cloud dedicated clusters are still functional but we are aware of issues being able to perform certain admin api calls via influxctl

4. identified Nov 20, 2025, 09:36 AM UTC

   We have tested extensively and found that Cloud dedicated clusters and Cloud 2 cluster are still functional, we can log in using credentials not via Goggle OAUTH. We are aware of issues being where certain admin api calls via influxctl are not able to be performed

5. identified Nov 20, 2025, 10:14 AM UTC

   We are continuing to work on a fix for this issue.

6. monitoring Nov 20, 2025, 12:50 PM UTC

   A fix has been implemented and we are monitoring the results.

7. resolved Nov 20, 2025, 02:07 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Nov 18, 2025, 07:48 PM UTC· resolved Nov 19, 2025, 01:47 AM UTC

1. investigating Nov 18, 2025, 07:48 PM UTC

   We are investigating increased query errors within the region

2. investigating Nov 18, 2025, 08:11 PM UTC

   We are continuing to investigate this issue.

3. identified Nov 18, 2025, 09:30 PM UTC

   The issue has been identified. We are adding capacity to the cluster, to help clear the backlog more quickly

4. monitoring Nov 18, 2025, 11:57 PM UTC

   A fix has been implemented and we are monitoring the results.

5. resolved Nov 19, 2025, 01:47 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Nov 11, 2025, 02:46 AM UTC· resolved Nov 12, 2025, 04:17 AM UTC

1. investigating Nov 11, 2025, 02:46 AM UTC

   Investigating fix.

2. identified Nov 11, 2025, 03:20 AM UTC

   A fix has been implemented. We are monitoring whether read performance has improved.

3. identified Nov 11, 2025, 05:49 AM UTC

   We are still seeing stability issues. Investigating other approaches.

4. identified Nov 11, 2025, 06:15 AM UTC

   We are still working on the resolution. Access to the InfluxDB Cloud UI and API will result in internal errors.

5. identified Nov 11, 2025, 10:35 AM UTC

   We are continuing to work on a fix for this issue.

6. identified Nov 11, 2025, 02:36 PM UTC

   We are still working on the resolution.

7. identified Nov 11, 2025, 02:45 PM UTC

   We are continuing to work on a fix for this issue.

8. identified Nov 11, 2025, 03:13 PM UTC

   We have identified the issue and continuing to work on a fix, we are working on applying a fix.

9. identified Nov 11, 2025, 04:03 PM UTC

   We have identified the issue and continuing to work on a fix.

10. identified Nov 11, 2025, 05:19 PM UTC

    We have identified the issue and continuing to work on a fix.

11. identified Nov 11, 2025, 07:02 PM UTC

    We have implemented a fix that is resulting in steady system recovery.

12. identified Nov 11, 2025, 07:25 PM UTC

    Recovery is continuing to progress.

13. identified Nov 11, 2025, 09:01 PM UTC

    Recovery is continuing, with setbacks.

14. identified Nov 11, 2025, 10:56 PM UTC

    We apologize for the length of time that it is taking to bring this cluster back into service. The team is working diligently to restore normal operation.

15. identified Nov 11, 2025, 10:57 PM UTC

    We apologize for the length of time that it is taking to bring this cluster back into service. The team is working diligently to restore normal operation

16. monitoring Nov 12, 2025, 01:13 AM UTC

    We have increased the cluster capacity, and it’s currently recovering. However, due to a large backlog of data, the TTBR remains elevated for now. Our team continues to monitor this issue closely.

17. monitoring Nov 12, 2025, 02:46 AM UTC

    The cluster has been scaled up, and system performance has returned to normal. Writes and queries are now processing as expected, and TTBR has stabilized. We sincerely apologize for the disruption to service over the past 24 hours and will share a detailed root cause analysis (RCA) once it is complete.

18. resolved Nov 12, 2025, 04:17 AM UTC

    This incident has been resolved.

19. postmortem Nov 14, 2025, 06:33 PM UTC

    RCA for Cloud 2 prod01-us-east-1 outage on Nov 11, 2025 # Summary The incident began during a planned migration of storage pods to a newer, more performant node class in Azure. The root cause was insufficient capacity in the new Azure node pool. When additional capacity could not be obtained, the team had to spend significant time rebalancing workloads across existing node pools to find a configuration capable of sustaining the write and query load. ‌ Clear communication was difficult during the incident because multiple mitigation paths were being explored in parallel, and we did not have confidence in any single fix until late in the recovery. A contributing factor to the duration of the outage was the amount of time Azure requires to apply node pool adjustments, which slowed our ability to test and iterate on potential solutions. Once sufficient capacity was available and the storage pods were redistributed appropriately, the cluster recovered. ‌ Background As part of an ongoing initiative to improve performance across Cloud 2, we are upgrading clusters to newer virtual machine types that were not available when the platform originally launched. These newer node classes provide higher throughput and allow us to right-size workloads more efficiently. This upgrade process has been completed successfully and without downtime in our AWS and GCP environments, as well as in our Azure staging cluster. ‌ As with all planned maintenance, an on-call engineering team was monitoring the migration and prepared to intervene if unexpected behavior occurred. ‌ Incident Details Cloud 2 clusters consist of 128 storage pods \(64 primary and 64 secondary\). Normal operation requires all 64 pods to be available \(either primary or secondary\). Our standard migration sequence moves secondary pods first, monitors stability, and then transitions primary pods once the new node pool has demonstrated adequate capacity. ‌ During the migration, secondary pods were successfully moved to the new Azure node pool and appeared stable under their normal load. However, when we initiated the transition of primary pods, the increased traffic on the secondaries exposed that the new pool did not have enough capacity to sustain the full workload. As soon as the capacity issue became apparent, engineers began working through multiple mitigation paths in parallel, including alternative node-pool configurations and fallback options. While the primaries were active, they were absorbing a significant share of read and write traffic, masking the fact that the secondaries were running close to their limits. When we attempted to scale the new node pool to provide the required capacity, Azure reported that no additional nodes of that type were available in the region. This prevented in-place scaling and removed the most direct mitigation path. To restore service, the team had to evaluate multiple fallback configurations using other node pools still available to us. ‌ Each adjustment required Azure to perform a full node-pool update cycle, which includes provisioning a temporary pool, shifting workloads, and then applying changes to the original pool. These operations take a significant amount of time in Azure, which slowed our ability to test and validate alternative configurations quickly. ‌ The combination of these factors \(an under-capacity new node pool, limited regional availability for scaling, and long Azure node-pool update times\) extended the overall duration of the outage. Once additional capacity using the prior node class was provisioned and storage pods were redistributed across the updated pool layout, the cluster stabilized and normal performance resumed. # Communication Communication during the incident did not meet expectations. Although we issued updates on our status page \([status.influxdata.com](http://status.influxdata.com)\), they lacked clarity because the team was evaluating several mitigation paths concurrently and did not have confidence that any single approach would resolve the issue. We were also slow to communicate full recovery. Additionally, an internal miscommunication led to inaccurate information being shared in our community Slack, suggesting the issue was not being addressed due to the U.S. holiday. This was incorrect; engineers were actively working throughout the incident. We recognize the impact this had on customer trust and have identified this as a process issue. # Future Mitigations 1. Strengthened Capacity and Quota Planning We will incorporate earlier and stricter quota and capacity validation into the migration process. Future node-pool changes will require confirming that available Azure quota is at least double the projected capacity needed for a safe migration. ‌ 2\. Revised Migration Procedure We will update our migration approach to move smaller slices of primaries and secondaries together. This will surface capacity issues earlier and prevent scenarios where a node pool appears healthy until the final migration step. ‌ 3\. Improved Internal Communication During Incidents We are refining our internal communication channels to ensure all customer-facing employees receive timely, authoritative information during outages. We are also reviewing expectations for status-page updates to ensure clarity and consistency.

Read the full incident report →

Started Nov 05, 2025, 06:17 PM UTC· resolved Nov 05, 2025, 09:54 PM UTC

1. investigating Nov 05, 2025, 06:17 PM UTC

   Writes may be degraded in us-east-1, and we're investigating

2. investigating Nov 05, 2025, 07:01 PM UTC

   We are continuing to investigate this issue.

3. monitoring Nov 05, 2025, 07:53 PM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved Nov 05, 2025, 09:54 PM UTC

   The incident has been resolved, and operations are back to normal

Read the full incident report →

Started Oct 28, 2025, 09:49 AM UTC· resolved Oct 28, 2025, 04:06 PM UTC

1. investigating Oct 28, 2025, 09:49 AM UTC

   We are seeing degraded performance in the us-east-1 region for queries. The team in continuing to investigate and will keep you updated

2. investigating Oct 28, 2025, 10:05 AM UTC

   We are continuing to investigate this issue.

3. investigating Oct 28, 2025, 10:46 AM UTC

   We are continuing to investigate this issue.

4. monitoring Oct 28, 2025, 01:04 PM UTC

   A fix has been applied and error rates are on decline. We will continue to monitor the issue.

5. monitoring Oct 28, 2025, 02:37 PM UTC

   Services continue to be healthy. We are continuing to monitor for any further issues.

6. resolved Oct 28, 2025, 04:06 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 28, 2025, 09:23 AM UTC· resolved Oct 28, 2025, 04:05 PM UTC

1. investigating Oct 28, 2025, 09:23 AM UTC

   We are still seeing Degraded performance in the region and some queries are failing. We are continuing to investigate and will post further updates.

2. investigating Oct 28, 2025, 09:27 AM UTC

   We are continuing to investigate this issue.

3. investigating Oct 28, 2025, 10:03 AM UTC

   We are working on the issue, we can see writes are starting to fail.

4. investigating Oct 28, 2025, 10:44 AM UTC

   Writes in this cluster has been restored, we are working on further to restore the read issues.

5. investigating Oct 28, 2025, 01:05 PM UTC

   A fix has been applied and error rates are on decline. We will continue to monitor the issue.

6. investigating Oct 28, 2025, 02:41 PM UTC

   Services continue to look healthy. We are continuing to monitor for further issues.

7. resolved Oct 28, 2025, 04:05 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 20, 2025, 06:45 PM UTC· resolved Oct 21, 2025, 12:15 AM UTC

1. monitoring Oct 20, 2025, 06:45 PM UTC

   While service has been restored, we do see some intermittent latency issues in AWS us-east-1, due to the continuing AWS outage.

2. resolved Oct 21, 2025, 12:15 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 20, 2025, 10:44 AM UTC· resolved Oct 20, 2025, 03:27 PM UTC

1. monitoring Oct 20, 2025, 10:44 AM UTC

   We are experiencing intermittent issues with API access in AWS us-east-1 due to upstream provider availability.

2. monitoring Oct 20, 2025, 11:55 AM UTC

   We are continuing to monitor for any further issues.

3. monitoring Oct 20, 2025, 02:49 PM UTC

   We are experiencing intermittent issues with API access in AWS us-east-1 due to upstream provider availability.

4. resolved Oct 20, 2025, 03:27 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 17, 2025, 05:20 PM UTC· resolved Oct 17, 2025, 06:56 PM UTC

1. investigating Oct 17, 2025, 05:20 PM UTC

   We are investigating an issue causing an increase in TTBR for some IOx queries in AWS us-east-1.

2. monitoring Oct 17, 2025, 05:46 PM UTC

   A fix has been implemented and we are monitoring the results.

3. resolved Oct 17, 2025, 06:56 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 15, 2025, 06:53 PM UTC· resolved Oct 15, 2025, 10:05 PM UTC

1. investigating Oct 15, 2025, 06:53 PM UTC

   We are investigating an issue with writes and an increased error rate for SQL queries in the EU-Central-1.

2. investigating Oct 15, 2025, 07:43 PM UTC

   We are continuing to investigate this issue.

3. monitoring Oct 15, 2025, 08:37 PM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved Oct 15, 2025, 10:05 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 03, 2025, 03:11 PM UTC· resolved Oct 03, 2025, 05:15 PM UTC

1. investigating Oct 03, 2025, 03:11 PM UTC

   We are currently seeing issues affecting database queries in the eu-central-1 region of AWS.

2. investigating Oct 03, 2025, 05:14 PM UTC

   We are continuing to investigate this issue.

3. resolved Oct 03, 2025, 05:15 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Sep 24, 2025, 04:00 PM UTC· resolved Sep 24, 2025, 07:21 PM UTC

1. investigating Sep 24, 2025, 04:00 PM UTC

   We are investigating an issue which may cause intermittent 504 responses to some queries.

2. monitoring Sep 24, 2025, 05:10 PM UTC

   Increased error rates have cleared. We are continuing to monitor.

3. monitoring Sep 24, 2025, 06:10 PM UTC

   We are continuing to monitor for any further issues.

4. resolved Sep 24, 2025, 07:21 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Sep 17, 2025, 11:00 PM UTC· resolved Sep 18, 2025, 01:02 AM UTC

1. investigating Sep 17, 2025, 11:00 PM UTC

   We are investigating the issue.

2. investigating Sep 17, 2025, 11:04 PM UTC

   We are investigating the issue.

3. monitoring Sep 18, 2025, 12:06 AM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved Sep 18, 2025, 01:02 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Sep 04, 2025, 10:30 PM UTC· resolved Sep 04, 2025, 10:33 PM UTC

1. identified Sep 04, 2025, 10:30 PM UTC

   We are currently experiencing issues with the Quartz Signup page. Users may see errors or the page stuck loading. Our engineering team has identified the cause and is working on a fix.

2. resolved Sep 04, 2025, 10:33 PM UTC

   A fix has been implemented and service to the Quartz Signup page should now be restored.

Read the full incident report →

Started Aug 01, 2025, 12:19 AM UTC· resolved Aug 01, 2025, 01:51 AM UTC

1. investigating Aug 01, 2025, 12:19 AM UTC

   We are currently investigating this issue.

2. identified Aug 01, 2025, 12:48 AM UTC

   The issue has been identified and a fix is being implemented.

3. monitoring Aug 01, 2025, 01:09 AM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved Aug 01, 2025, 01:51 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Jul 31, 2025, 09:53 AM UTC· resolved Jul 31, 2025, 02:02 PM UTC

1. investigating Jul 31, 2025, 09:53 AM UTC

   We have identified some FlightSQL errors in Serverless AWS eu-central-1 region for certain customers. Our team has taken some remedial action and is continuing to investigate this. We will provide more updates soon.

2. monitoring Jul 31, 2025, 11:03 AM UTC

   A fix has been implemented and we are monitoring the results.

3. resolved Jul 31, 2025, 02:02 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Jul 10, 2025, 06:42 PM UTC· resolved Jul 10, 2025, 11:21 PM UTC

1. investigating Jul 10, 2025, 06:42 PM UTC

   The Cloud dedicated table management API is currently unavailable impacting the ability to list tables

2. identified Jul 10, 2025, 07:35 PM UTC

   We believe we have identified the cause of the problem and are implementing a fix to revert. This incident has been updated to reflect that only the ListTables endpoint is impacted.

3. identified Jul 10, 2025, 08:27 PM UTC

   We have identified a fix and are testing it internally

4. identified Jul 10, 2025, 09:47 PM UTC

   We have identified a fix and are pushing it out to affected customer clusters

5. monitoring Jul 10, 2025, 10:11 PM UTC

   We have finished deploying a fix to all affected customer clusters.

6. resolved Jul 10, 2025, 11:21 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Jun 30, 2025, 10:10 AM UTC· resolved Jun 30, 2025, 12:53 PM UTC

1. investigating Jun 30, 2025, 10:10 AM UTC

   We are currently investigating this issue.

2. investigating Jun 30, 2025, 11:11 AM UTC

   We are continuing to investigate this issue.

3. monitoring Jun 30, 2025, 11:16 AM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved Jun 30, 2025, 12:53 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Jun 13, 2025, 04:13 PM UTC· resolved Jun 13, 2025, 10:14 PM UTC

1. investigating Jun 13, 2025, 04:13 PM UTC

   We have observed an elevated query error rate and are investigating.

2. identified Jun 13, 2025, 05:06 PM UTC

   The issue has been identified and a fix is being put into place.

3. monitoring Jun 13, 2025, 05:39 PM UTC

   A fix has been implemented and deployed. We are monitoring at this time.

4. monitoring Jun 13, 2025, 07:09 PM UTC

   We are continuing to closely monitor the health of the query API

5. monitoring Jun 13, 2025, 08:36 PM UTC

   We are continuing to monitor this region for any further degradation

6. resolved Jun 13, 2025, 10:14 PM UTC

   This incident has been resolved.

7. postmortem Jul 17, 2025, 08:26 PM UTC

   In a multi-tenant environment, such as Cloud 2, there can be unpredictable changes in workload, as different customers on the shared platform may increase or decrease their activity on the cluster with no advance notice. While we try to shelter customers from one another, via rate limits and some partitioning, there is no way to fully protect customers from the impact of a "noisy neighbor". We maintain excess capacity in the cluster to provide some amount of headroom for increased workloads, and we also have rate limits to ensure that one customer does not overwhelm the cluster \(via writes, deletes or queries\), but the rate limits are not very fine-grained, and at times, a single customer can stress the cluster. When that occurs, the team is automatically notified, and we add additional resources to accommodate the extra workload. It can sometimes take time for the extra capacity to be fully deployed, leading to temporary performance problems. This is what occurred in the Azure eu-west cluster on June 13th. We were notified that query performance was degraded and the team increased the storage pods to provide extra capacity. Once the storage pods limits were fully applied, the query TTBR recovered. We apologize for the inconvenience.

Read the full incident report →

Started Jun 12, 2025, 06:37 PM UTC· resolved Jun 12, 2025, 09:53 PM UTC

1. investigating Jun 12, 2025, 06:37 PM UTC

   We are currently investigating query availability in GCP regions.

2. investigating Jun 12, 2025, 06:43 PM UTC

   We are continuing to investigate this issue.

3. investigating Jun 12, 2025, 07:18 PM UTC

   We are continuing to investigate this issue.

4. identified Jun 12, 2025, 07:26 PM UTC

   Google Cloud Platform is experiencing a major incident. More details on this incident can be found at https://status.cloud.google.com/

5. identified Jun 12, 2025, 08:19 PM UTC

   Our team is awaiting recovery of the underlying GCP infrastructure and is prepared to bring all services online upon recovery. More incident related information can be found at https://status.cloud.google.com/

6. identified Jun 12, 2025, 08:49 PM UTC

   Services are beginning to recover as underlying infrastructure becomes available.

7. monitoring Jun 12, 2025, 08:50 PM UTC

   A fix has been implemented and we are monitoring the results.

8. resolved Jun 12, 2025, 09:53 PM UTC

   All GCP services have been recovered.

9. postmortem Jun 20, 2025, 10:33 PM UTC

   **RCA - Service Outage on June 12, 2025** ‌ **Summary** On June 12, 2025, at 17:51 UTC, Google Cloud Platform \(GCP\) experienced a global outage that disrupted a wide range of cloud services, including core infrastructure on which InfluxDB Cloud relies. As a result, all InfluxDB Cloud services hosted on GCP became temporarily unavailable. ‌ While most services recovered automatically once Google services were restored, manual intervention was required to fully resume Flux task processing. ‌ **Customer Impact** Writes and queries to InfluxDB Cloud began to fail across GCP-hosted regions on June 12, at 17:51 UTC and remained unavailable until InfluxData’s cluster became operational on June 12, at 20:30 UTC. Google announced full service restoration on June 13, at 01:18 UTC. ‌ We received a report of Flux tasks not processing on June 12, 23:20 UTC. Flux task processing was restored on June 12, at approximately 23:50 UTC. ‌ **Cause of the Incident** See [details](https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW) on the GCP incident timeframe and root cause analysis. Flux task processing was impacted due to service recovery issues. ‌ **Investigation and Recovery** We actively monitored GCP service restoration via email and its status page. For InfluxDB Cloud services, the majority of services were restored without manual intervention. However, manual intervention was needed to restore Flux task processing. This intervention effectively restarted impacted services, resulting in the full restoration of Flux task processing.

Read the full incident report →