Amplify Fusion US Control Plane Disruption on November 4, 2026

Update timeline

1. resolved Nov 04, 2026, 04:15 PM UTC

   On November 4, 2026, between 16:40 and 19:54 UTC (~3h15m), some customers saw "Data Planes 0" or "Failed to get cluster" errors in the LIVE console for the US Production control plane (us-east-1). A sharp spike in audit event volume overloaded the storage behind our Hazelcast service, causing control plane services to restart repeatedly. Data planes were not affected and continued to process traffic throughout. We resolved the issue by moving Hazelcast to faster storage. Audit events queued during the incident window may be delayed or missing from audit reports; we are reviewing recovery options. A full Root Cause Analysis is provided below.

2. postmortem Nov 11, 2026, 05:18 PM UTC

   **Incident date**: 2026-11-04 **Customer impact window**: 16:40 UTC - 19:54 UTC \(~3h15m\) **Environment affected**: US Production control plane \(us-east-1\) **SUMMARY** Audit service streams runtime audit events into Hazelcast to manage persistence and load balancing. On November 4, 2026, a sharp increase in runtime audit event volume caused the persistent volume backing our Hazelcast service to become IO bound. This slowed the control plane services that depend on Hazelcast, and they were restarted repeatedly by the platform. As a result, some customers saw "Data Planes 0" or "Failed to get cluster" in their LIVE consoles. To mitigate this, we moved the Hazelcast persistent storage to faster, lower latency volumes. Normal operation was restored at 19:54 UTC. The data planes themselves remained healthy and continued to run throughout the incident. **TIMELINE \(UTC\)** 16:15 - Runtime audit event volume starts rising to about 100 times its normal level 16:40 - Control plane services \(Integration Manager and Audit service\) begin restarting after failing health checks 17:04 - Customers report "Data Planes 0 / Failed to get cluster" in LIVE consoles; support case opened 17:14 - Investigation finds the services are waiting on responses from Hazelcast 17:44 - Hazelcast confirmed as saturated by the audit write load 19:14 - Incoming audit traffic temporarily paused to relieve pressure 19:51 - Hazelcast persistent storage migrated to faster, lower latency volumes 19:54 - Audit traffic restored; services stable **WHAT HAPPENED** Integration Manager and Audit service rely on Hazelcast for audit event queuing and for retrieving cluster information. Starting around 16:15 UTC, the volume of audit events rose far above normal. The persistent volume used by Hazelcast to store the audit queue could not keep up with the rate of writes, and Hazelcast responses slowed down significantly. While waiting on these responses, the control plane services failed their health checks and were restarted by the platform. This repeated until the underlying storage issue was addressed. During that time the console could not retrieve cluster information, which is why customers saw errors, even though their data planes were running normally. After the cause was identified, we paused incoming audit traffic, moved the Hazelcast persistent storage to faster volumes, and restored traffic. The affected services have not restarted since the change. **ROOT CAUSE** The Hazelcast persistent volume became IO bound under a spike in audit event load. The storage configuration in place could not sustain that write rate, which delayed requests from the control plane services and led to repeated service restarts. **CUSTOMER IMPACT** For approximately 3 hours and 15 minutes \(16:40 UTC to 19:54 UTC\), the control plane was intermittently unavailable, and some customers saw errors in the LIVE console when viewing their data planes. Data planes were not affected and continued to process traffic. Audit events that were queued on the previous storage during the incident window may be delayed or missing from audit reports. We are reviewing options for recovering them and will update this document with the outcome. **CORRECTIVE AND PREVENTIVE ACTIONS** We also acknowledge that this issue was first reported to us by customers rather than detected by our own monitoring. We are making the following changes: 1. Persistent storage configuration \[COMPLETED\] Make the faster Hazelcast storage part of our automated deployment configuration across all environments, so it is retained in future deployments. Owner: Axway Operations 2. Audit event handling \[IN PROGRESS\] Review how audit events are queued and persisted under high load, including protection against sudden volume spikes from a single source. Owner: Axway R&D 3. Detection \[COMPLETED\] Add monitoring and alerting for data plane disconnection and control plane health, so issues are detected before they are visible to customers. Owner: Axway Operations **CLOSING** We apologize for the disruption this caused and for the time it took to resolve. If you have any questions about this analysis or the actions above, please reach out to us through your Support Case or your usual Axway contact.