CloudAMQP Outage History

Started Oct 21, 2026, 04:39 PM UTC· resolved Oct 21, 2026, 05:13 PM UTC

1. investigating Oct 21, 2026, 04:39 PM UTC

   We're investigating connectivity issues affecting some servers in Azure West Europe. We'll post an update as soon as we know more.

2. resolved Oct 21, 2026, 05:13 PM UTC

   Connectivity to servers in southeast Asia has been restored.

Read the full incident report →

Started Oct 20, 2026, 07:21 AM UTC· resolved Oct 20, 2026, 01:32 PM UTC

1. investigating Oct 20, 2026, 07:21 AM UTC

   We're investigating connectivity issues affecting some servers in Google europe-west2. We'll post an update as soon as we know more.

2. investigating Oct 20, 2026, 07:22 AM UTC

   We are continuing to investigate this issue.

3. investigating Oct 20, 2026, 07:29 AM UTC

   Incident is limited to servers in europe-west2-b

4. monitoring Oct 20, 2026, 09:00 AM UTC

   GCE is still reporting issues, but we are seeing some API calls succeed.

5. monitoring Oct 20, 2026, 12:13 PM UTC

   GCE has confirmed that a power outage in europe-west2-b caused the disruption. The power issue has been mitigated, and full recovery of affected workloads is tentatively expected by 13:00 UTC today. We continue to monitor the recovery.

6. monitoring Oct 20, 2026, 12:33 PM UTC

   We are continuing to monitor for any further issues.

7. resolved Oct 20, 2026, 01:32 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 09, 2026, 01:28 PM UTC· resolved Oct 09, 2026, 07:27 PM UTC

1. investigating Oct 09, 2026, 01:28 PM UTC

   We're investigating connectivity issues affecting some servers in Azure West Europe. We'll post an update as soon as we know more.

2. investigating Oct 09, 2026, 02:39 PM UTC

   Azure is aware of this issue and are actively investigating. We'll post another update as soon as we know more.

3. identified Oct 09, 2026, 03:08 PM UTC

   Azure has identified this issue. They have narrowed the issue to storage accessibility affecting the region. Their teams is actively performing recovery actions on the affected infrastructure while continuing to investigate the underlying cause. We are monitoring recovery progress and working to restore affected resources.

4. monitoring Oct 09, 2026, 05:07 PM UTC

   We have been able to establish communication with instances in Azure West Europe. Majority of the instances are back up and running. Azure still has some VMs to resolve.

5. resolved Oct 09, 2026, 07:27 PM UTC

   All CloudAMQP instances are up and running in Azure West Europe.

6. postmortem Oct 12, 2026, 02:51 PM UTC

   **Summary** CloudAMQP lost connectivity with a subset of clusters in Azure west Europe availability zone 2 on October 9th. Single node clusters in AZ2 were most impacted by the incident. ‌ **Impact** Customers were unable to access data or mange nodes impacted during the incident. Timeline of events from Azure. * 12:25 UTC on 09 October 2026 – We identified this issue via our internal service monitoring alerts. * 12:29 UTC on 09 October 2026 – Customer impact began, with VM disk failures detected in the affected West Europe storage scale unit. * 2:30 UTC on 09 October 2026 – The investigation identified loss of heartbeat across multiple storage nodes and fault domains, resulting in the storage scale unit to go in distress mode. * 15:32 UTC on 09 October 2026 - We took the storage scale unit out of distress mode, isolating the disks on the impacted storage servers to enable recovery of the remaining disks. At this stage, most of the affected customers began seeing recovery. * 21:00 UTC on 09 October 2026 - Datacenter and Storage engineers continued executing the recovery plan on multiple storage nodes. * 23:27 UTC on 09 October 2026 – Efforts to recover the affected storage nodes were successful, and work to recover the virtual disks continued. Subsequent recovery actions included node-level recovery and restoration of affected storage resources. * 03:00 UTC on 10 October 2026 – Service functionality recovered, and customer impact was mitigated. **Root cause** Azure experienced a platform issue resulted in an impact to Azure Virtual Machines storage disks in the West Europe region AZ 2. Impacted customers may have experienced connectivity failures, unexpected restarts, or Virtual Machines entering a failed state in the region. ‌ **Resolution** Machine access was restored by Azure in about 3 hours. CloudAMQP was able to restore communication with the impacted nodes. We continued to monitor updates from Azure to ensure nodes were no longer impacted.

Read the full incident report →

Started Oct 06, 2026, 02:37 PM UTC· resolved Oct 06, 2026, 03:48 PM UTC

1. identified Oct 06, 2026, 02:37 PM UTC

   We are seeing degraded network connectivity between some non-Azure networks and Azure UAE North (azure-arm::uaenorth), caused by a routing problem at an upstream transit provider. All servers in the region are up and processing messages, and monitoring, metrics and logs are working as normal. Customers whose clients connect from networks affected by this routing problem may also see connection timeouts. We have reported the problem to the provider and are routing our own management traffic around it.

2. monitoring Oct 06, 2026, 03:44 PM UTC

   Routing is back to normal.

3. resolved Oct 06, 2026, 03:48 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Aug 26, 2026, 01:27 PM UTC· resolved Aug 26, 2026, 01:57 PM UTC

1. investigating Aug 26, 2026, 01:27 PM UTC

   We are currently investigating this issue.

2. identified Aug 26, 2026, 01:28 PM UTC

   The issue has been identified and a fix is being implemented.

3. monitoring Aug 26, 2026, 01:41 PM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved Aug 26, 2026, 01:57 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Aug 24, 2026, 11:52 PM UTC· resolved Aug 25, 2026, 11:58 PM UTC

1. monitoring Aug 24, 2026, 11:52 PM UTC

   DigitalOcean is currently experiencing issues with its API and control panel and you may run into errors and higher latency while the issue persists

2. resolved Aug 25, 2026, 11:58 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Jul 23, 2026, 03:33 PM UTC· resolved Jul 23, 2026, 09:48 PM UTC

1. investigating Jul 23, 2026, 03:33 PM UTC

   We've received various alert notifying us about couple of servers deployed on Azure West US not being reachable. Apparently this seems to be related specific to this region impacting 25% of the servers.

2. monitoring Jul 23, 2026, 07:50 PM UTC

   Microsoft Azure confirmed they were having network issues on West US region. For now, most of the servers are operational. We'll keep monitoring for the following hours.

3. resolved Jul 23, 2026, 09:48 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Jun 25, 2026, 02:10 PM UTC· resolved Jun 26, 2026, 02:44 PM UTC

1. investigating Jun 25, 2026, 02:10 PM UTC

   We are currently investigating this issue.

2. identified Jun 25, 2026, 02:21 PM UTC

   The issue has been identified and a fix is being implemented.

3. investigating Jun 26, 2026, 08:38 AM UTC

   We are currently investigating this issue.

4. identified Jun 26, 2026, 09:17 AM UTC

   The issue has been identified and a fix is being implemented.

5. monitoring Jun 26, 2026, 02:22 PM UTC

   A fix has been implemented and we are monitoring the results.

6. resolved Jun 26, 2026, 02:44 PM UTC

   This incident has been resolved.

7. postmortem Jun 29, 2026, 11:22 AM UTC

   **Summary** Between June 25 and June 26, our monitoring system incorrectly reported some dedicated servers as unreachable and sent "server down" notifications for servers that were in fact operating normally. This was a monitoring-side issue only — no customer servers or services were actually down, and no data or message delivery was affected. **Impact** Affected customers received one or more alerts indicating their dedicated server was down. These alerts were false positives. The servers themselves remained healthy and fully available throughout the incident. **Root cause** Before connecting to a server, our monitoring system performs a network authorization step. A defect in how one component handled a transient network hiccup could leave a monitoring process unable to complete that step, after which it failed to connect to the servers it was responsible for checking. The servers themselves stayed healthy and available throughout — the monitoring process had simply lost its ability to reach them, and reported them as down. **Resolution** We identified the affected monitoring processes, confirmed the root cause in production, and deployed a fix that makes this authorization step resilient to such transient failures and able to recover automatically. After deploying, we monitored the system to confirm that notifications returned to normal. We apologize for the confusion these false notifications may have caused.

Read the full incident report →

Started May 30, 2026, 10:25 PM UTC· resolved May 31, 2026, 11:29 AM UTC

1. monitoring May 30, 2026, 10:25 PM UTC

   Between 17:30 and 20:00 UTC on 2026-05-30, our server monitoring system generated a large number of false "server_unreachable" alerts. Clusters were operational and healthy throughout this period — no customer data or message delivery was affected. The alerts were caused by a rolling restart of our monitoring service, during which individual monitoring workers went offline mid-cycle and lost state. This caused the workers to report connection timeouts for servers they could no longer reach — even though those servers were fully healthy. We apologize for any concern this may have caused.

2. resolved May 31, 2026, 11:29 AM UTC

   This incident has been resolved.

Read the full incident report →

Started May 22, 2026, 03:16 AM UTC· resolved May 22, 2026, 04:50 AM UTC

1. investigating May 22, 2026, 03:16 AM UTC

   Since around 8pm UTC last night, some metrics are not being forwarded to legacy integration. Prometheus integrations not affected. We are investingating.

2. monitoring May 22, 2026, 04:08 AM UTC

   A fix has been implemented and we are monitoring the results.

3. resolved May 22, 2026, 04:50 AM UTC

   This incident has been resolved.

4. postmortem May 22, 2026, 01:52 PM UTC

   # Legacy host metrics integrations degraded Incident window: 2026-05-21 18:37 UTC – 2026-05-22 04:05 UTC \(9h 28m\) Affected service: Legacy metric integrations \(CloudWatch, Datadog, Librato, New Relic, Splunk, Stackdriver\). Host metrics affected, not broker metrics, nor was internal monitoring or console graphs. ## Summary For ~9.5 hours, the service that collects host-level metrics \(CPU, memory, disk, network\) for legacy third-party integrations entered a crash loop and stopped shipping data. ## Root Cause An internal credential-signing service was migrated to a new container runtime the afternoon before. Due to a bug with environment variables the collector could not renew it's credentials. The collector's existing credential remained valid for several hours, so the failure only surfaced when it tried to renew. ## Resolution On-call staff detected the failure early morning, developers helped restore the service and collector recovered at 04:05 UTC. ## Prevention * All services, both signing and collector report failed authentication attempts earlier and with higher severity * Metrics pipeline alarm thresholds was tightened so a similar drop in data triggers alarms faster.

Read the full incident report →

Started May 19, 2026, 01:21 PM UTC· resolved May 19, 2026, 01:50 PM UTC

1. investigating May 19, 2026, 01:21 PM UTC

   We are currently experiencing issues with our backend services handling account and server creation. This does not affect any running customer servers but might delay provisioning new servers.

2. identified May 19, 2026, 01:36 PM UTC

   The issue has been identified and a fix is being implemented.

3. monitoring May 19, 2026, 01:41 PM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved May 19, 2026, 01:50 PM UTC

   This incident has been resolved.

Read the full incident report →

Started May 03, 2026, 07:11 PM UTC· resolved May 04, 2026, 01:44 AM UTC

1. investigating May 03, 2026, 07:11 PM UTC

   We are currently investigating this issue.

2. identified May 03, 2026, 07:14 PM UTC

   Servers in Azure West Europe can experience connection issues due to underlying issues at provider. Latest update from Azure "Current Status: We continue to investigate an availability issue caused by a subset of unhealthy backend storage infrastructure supporting Azure Virtual Machines and Azure Direct Drive in West Europe."

3. resolved May 04, 2026, 01:44 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Apr 28, 2026, 10:27 AM UTC· resolved Apr 28, 2026, 11:05 AM UTC

1. investigating Apr 28, 2026, 10:27 AM UTC

   We are currently experiencing issues with our backend services handling account and server creation. This does not affect any running customer servers but might delay provisioning new servers.

2. identified Apr 28, 2026, 10:49 AM UTC

   The issue has been identified and a fix is being implemented.

3. monitoring Apr 28, 2026, 10:53 AM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved Apr 28, 2026, 11:05 AM UTC

   We had an abnormal increase in traffic causing timeouts and slow responses for customer.cloudamqp.com, we increased capacity and are back to normal. This did not affect any running clusters, just provisioning and configuration updates.

Read the full incident report →

Started Apr 24, 2026, 09:36 AM UTC· resolved Apr 24, 2026, 10:05 AM UTC

1. investigating Apr 24, 2026, 09:36 AM UTC

   We are getting indications of instances being down in Azure region westus. We are investigating the issue.

2. resolved Apr 24, 2026, 11:12 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Apr 23, 2026, 10:10 AM UTC· resolved Apr 23, 2026, 10:56 AM UTC

1. investigating Apr 23, 2026, 10:10 AM UTC

   Servers in Azure West US 3 can experience connection issues due to underlying issues at provider. We monitor and will update when we know more.

2. resolved Apr 23, 2026, 10:56 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Apr 23, 2026, 03:30 AM UTC· resolved Apr 23, 2026, 06:00 AM UTC

1. investigating Apr 23, 2026, 04:33 AM UTC

   There appears to be an issue regarding sending metrics to datadog. We are currently investigating the issue

2. monitoring Apr 23, 2026, 06:53 AM UTC

   A fix has been implemented and we are monitoring the results.

3. resolved Apr 23, 2026, 11:32 AM UTC

   This incident has been resolved.

4. postmortem Apr 24, 2026, 08:38 AM UTC

   Broker metrics and metrics integrations degradation Window: 2026-04-23 03:25 – 05:35 UTC \(2h 10m\) ## Impact Broker metrics \(connections, channels, queues, consumers, message rates, node and netsplit state\) were degraded, progressively dropping until recovery at 05:35 UTC. Downstream effects: * Queue, Consumer, Connection, Channel, Connection flow, and Netsplit alarms: threshold breaches during the window may not have triggered, or triggered late. Alarms already tripped before 03:25 UTC kept their state. * Metrics Integrations \(Datadog, CloudWatch, New Relic, Splunk, Dynatrace, etc.\): delivery of broker metrics was reduced for the duration of the window. Alarms configured on the receiving side may likewise have missed or lagged. Not affected: server metrics \(CPU, memory, disk\) and their corresponding alarms, Notice alarms, broker availability, and the metric graphs shown in the CloudAMQP console \(these use a separate data path and continued to update normally\). ## Timeline \(UTC\) * 03:25 — broker-metrics collection begins degrading * 03:35 — ~30% of broker-metric samples missing * 03:50 — ~50% plateau * 05:25 — collector workers cycled and re-initialised * 05:35 — full throughput restored ## Root cause Broker metrics are polled from each cluster's management HTTP API by a pool of collector workers and published to an internal message bus that both the Alarms service and Metrics Integrations consume from. At 03:25 UTC this service became unresponsive without raising an error. Affected workers silently stopped polling while remaining alive to the platform, so automatic restart did not trigger. On-call staff began investigating around 05:00 UTC; force-restarting the service restored metric flow. The exact trigger has not been identified. Our focus is on ensuring the condition is detected and handled promptly if it recurs. ## What we are changing * Added metrics and alarms specific to this service, with high-urgency paging for on-call. * Reliability improvements in the service itself to detect and restart stalled work. ## What you can do CloudAMQP offers a new generation of metrics integrations based on Prometheus, with a re-engineered pipeline that does not rely on these centralised services — each server forwards data directly to your endpoint. These have been running in production for some time and have proved very reliable. * [https://www.cloudamqp.com/blog/prometheus-metrics-integrations.html](https://www.cloudamqp.com/blog/prometheus-metrics-integrations.html) * [https://www.cloudamqp.com/docs/monitoring\_metrics\_datadog\_v3.html](https://www.cloudamqp.com/docs/monitoring_metrics_datadog_v3.html) * Background: [https://www.cloudamqp.com/blog/decentralized-observability-with-open-telemetry-part-1.html](https://www.cloudamqp.com/blog/decentralized-observability-with-open-telemetry-part-1.html)

Read the full incident report →

Started Apr 15, 2026, 02:00 PM UTC· resolved Apr 15, 2026, 02:00 PM UTC

1. resolved Apr 15, 2026, 03:32 PM UTC

   One of our backend components suffered a small outage due to an unexpected full services restart. Connectivity with the servers were broken and service triggered alarms for many clusters. We apologize for the inconvenience. All clusters were operational and healthy within the incident/alarms timespan.

Read the full incident report →

Started Apr 02, 2026, 02:41 PM UTC· resolved Apr 02, 2026, 03:12 PM UTC

1. identified Apr 02, 2026, 02:41 PM UTC

   Customers are experiencing Management UI being unavailable with 502 Bad Gateway. Timeline: - 14:16 UTC -> The issue was identified - 14:30 UTC -> Recent code changes were reverted. No relationship with incident noted.

2. identified Apr 02, 2026, 02:52 PM UTC

   The Management UI and brokers are serving traffic normally. You can still use the Management UI by using the Broker URL and properly typing User and Password. The impact is related to our backend service logs into Management UI via CloudAMQP SSO.

3. identified Apr 02, 2026, 03:07 PM UTC

   We are continuing to work on a fix for this issue.

4. resolved Apr 02, 2026, 03:12 PM UTC

   The Management UI SSO shortcut is back to service. Impacted Clusters: All customers with clusters created more than 6 months ago Cause: Some code changes unexpectedly changed the communication schema between our backend services and the SSO component on the clusters. The proxy configuration conflicted with those changes causing the 502 Bad Gateway error to show up. Impact: Only the SSO via CloudAMQP Console was impacted given it required communication with our backend components. The AMQP Clusters were healthy and servicing traffic via both AMQP and HTTP on Management UI and API. Logging into the Management UI via Credentials and all Authentications Backends was functioning normally.

Read the full incident report →

Started Mar 25, 2026, 05:45 PM UTC· resolved Mar 25, 2026, 06:03 PM UTC

1. identified Mar 25, 2026, 05:45 PM UTC

   We're currently experiencing issues with metrics delivery, this affects 12% of the API based v1/v2 integrations. We've added additional capacity and expect have handled the backlog within 20 minutes.

2. resolved Mar 25, 2026, 06:03 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Mar 18, 2026, 09:25 PM UTC· resolved Mar 19, 2026, 03:50 AM UTC

1. identified Mar 18, 2026, 09:25 PM UTC

   We noticed an issue with our queue metrics being sent to metrics integrations. there is currently a delay in processing, but we have identified an issue and expect the delay to be fixed shortly.

2. monitoring Mar 18, 2026, 10:16 PM UTC

   A fix has been implemented and we are monitoring the results.

3. resolved Mar 19, 2026, 03:50 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Mar 03, 2026, 09:11 PM UTC· resolved Mar 04, 2026, 02:14 PM UTC

1. investigating Mar 03, 2026, 09:11 PM UTC

   We're currently experiencing issues with Coralogix log delivery.

2. monitoring Mar 04, 2026, 11:16 AM UTC

   A fix has been implemented and we are monitoring the results.

3. resolved Mar 04, 2026, 02:14 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Mar 03, 2026, 06:28 PM UTC· resolved Mar 03, 2026, 08:51 PM UTC

1. investigating Mar 03, 2026, 06:28 PM UTC

   We're currently experiencing issues with metrics delivery, this affects v3 (Prometheus based) metrics.

2. identified Mar 03, 2026, 06:47 PM UTC

   The issue has been identified and a fix is being implemented.

3. monitoring Mar 03, 2026, 07:30 PM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved Mar 03, 2026, 08:51 PM UTC

   This incident has been resolved.

5. postmortem Mar 04, 2026, 02:14 PM UTC

   ### Summary On March 3, an internal monitoring metric stopped being reported for approximately 14 hours. No customer metrics delivery was affected, all integrations continued operating normally. The missing metric caused our dashboards to display false failure rates, which was incorrectly reported as a metrics delivery outage. ### Impact None. Metrics continued to be delivered to all customer endpoints throughout the incident. ### Resolution The monitoring metric was restored and we have updated our alerting queries to be resilient to missing data, preventing false positives in the future.

Read the full incident report →

Started Mar 02, 2026, 05:35 AM UTC· resolved May 05, 2026, 06:55 AM UTC

1. monitoring Mar 02, 2026, 05:35 AM UTC

   AWS has an ongoing issue with multiple services in region me-south-1. Shared server hummingbird-01 affected.

2. monitoring Mar 02, 2026, 08:19 PM UTC

   We are still experiencing issues with this region. If your services are impacted, we recommend you create a new cluster in a different region. When creating, use the copy config option, but do not copy plugin settings as this requires the source cluster to be running. Instead, enable once cluster is configured.

3. resolved May 05, 2026, 06:55 AM UTC

   We are suspending active updates on this status page regarding the AWS Middle East (Bahrain/UAE) outages. We will start supporting those regions again when they have been fully recovered. Reach out to [email protected] if you have any questions.

Read the full incident report →

Started Mar 01, 2026, 01:20 PM UTC· resolved May 05, 2026, 06:55 AM UTC

1. monitoring Mar 01, 2026, 01:20 PM UTC

   AWS has an ongoing issue with multiple services in region me-central-1.

2. monitoring Mar 01, 2026, 01:22 PM UTC

   We are continuing to monitor for any further issues.

3. monitoring Mar 01, 2026, 01:23 PM UTC

   We are continuing to monitor for any further issues.

4. monitoring Mar 02, 2026, 01:46 AM UTC

   We are continuing to monitor. The outage is in regards to the AZ mec1-az2

5. monitoring Mar 02, 2026, 08:19 PM UTC

   We are still experiencing issues with all AZ's in this region. If your services are impacted, we recommend you create a new cluster in a different region. When creating, use the copy config option, but do not copy plugin settings as this requires the source cluster to be running. Instead, enable once cluster is configured.

6. resolved May 05, 2026, 06:55 AM UTC

   We are suspending active updates on this status page regarding the AWS Middle East (Bahrain/UAE) outages. We will start supporting those regions again when they have been fully recovered. Reach out to [email protected] if you have any questions.

Read the full incident report →

Started Feb 17, 2026, 07:31 PM UTC· resolved Feb 18, 2026, 03:30 PM UTC

1. investigating Feb 17, 2026, 07:31 PM UTC

   We are currently investigating this issue.

2. monitoring Feb 17, 2026, 09:44 PM UTC

   A fix has been implemented and we are monitoring the results.

3. resolved Feb 18, 2026, 03:30 PM UTC

   This incident has been resolved.

Read the full incident report →