Exalate experienced a major incident on December 12, 2025 affecting connect.exalate.net (mapper) and Exalate Console and 1 more component, lasting 12h 10m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Dec 12, 2025, 04:15 PM UTC
There is an outage on one of the clusters in Exalate cloud that might affect access to the nodes temporarily. We are currently investigating the issue to ensure that service is restored as soon as possible. A new update will be provided within 1 hour
- identified Dec 12, 2025, 04:31 PM UTC
We have root caused the issue and the restoration process is underway already. The next update will be provided in 30mins.
- identified Dec 12, 2025, 04:58 PM UTC
The recovery process is still ongoing. We will provide an update within an hour.
- identified Dec 12, 2025, 06:12 PM UTC
We continue to work on the restoration of the affected nodes. We will be providing a new update in one hour.
- identified Dec 12, 2025, 07:11 PM UTC
We continue to work on the restoration of the affected nodes. We will be providing a new update in one hour.
- identified Dec 12, 2025, 08:20 PM UTC
Nodes are being restored now, most nodes are now online, a full operational message will be posted once all nodes are back online.
- identified Dec 12, 2025, 11:11 PM UTC
Most nodes are back on operational status, a full operational message will be posted once all nodes are back.
- identified Dec 13, 2025, 01:51 AM UTC
Most nodes are back on operational status, a full operational message will be posted once all nodes are back.
- monitoring Dec 13, 2025, 02:56 AM UTC
All nodes are back to operational status now. We continue to monitor the stability closely.
- resolved Dec 13, 2025, 04:26 AM UTC
All Exalate nodes are stable and working normally. A post-mortem for the incident will be published in due course.
- postmortem Dec 22, 2025, 04:43 AM UTC
**Incident: Service Outage - December 12, 2025** **Duration:** 16:35 CET - 03:01 CET \(approximately 10 hours\) **Impact:** A significant number of Exalate Cloud nodes experienced service unavailability, resulting in temporary disruption to data synchronization services. No data was lost during this incident. **Summary:** On December 12, 2025, our infrastructure monitoring detected a service outage affecting Exalate Cloud customers on our production clusters. The root cause was identified as a storage system failure triggered by network latency on a legacy infrastructure component, which caused storage connectivity issues for customer workloads. **Timeline:** * 16:35 CET - Issue detected by monitoring systems * 17:31 CET - Root cause identified, restoration initiated * 21:20 CET - Majority of nodes restored to service * 03:01 CET - Full service restoration confirmed **Root Cause:** A storage management component became unresponsive due to elevated network latency caused by a legacy networking layer. This resulted in storage disconnection for customer workloads. **Resolution:** The storage system was restored, all affected workloads were rescheduled, and the external dependency was updated to a supported version. All nodes were returned to full operation with no data loss. **Preventive Measures:** * Deploying patches to address the storage component stability issue * Accelerating migration away from the legacy networking layer to modern infrastructure * Implementing internal mirroring of critical external dependencies to eliminate reliance on third-party availability * Enhancing operational safeguards for infrastructure management procedures We sincerely apologize for the disruption this incident caused. Our team remains committed to improving the reliability and resilience of the Exalate Cloud platform.