IONOS Cloud experienced a minor incident on March 2, 2026 affecting Managed Kubernetes, lasting 2h 52m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Mar 02, 2026, 12:03 PM UTC
We are currently experiencing instability affecting Kubernetes clusters hosted in Frankfurt. Some clusters are stuck in an updating/deploying state, which is causing intermittent communication issues between control plane components and worker nodes. The issue may cause brief interruptions or degraded performance across impacted services. Our engineering teams are actively investigating and working to restore full stability as quickly as possible.
- resolved Mar 02, 2026, 02:55 PM UTC
This incident has been resolved.
- postmortem Mar 06, 2026, 04:11 PM UTC
**Incident Summary** On March 2, 2026, some Managed Kubernetes clusters in our Frankfurt \(DE/FRA\) region became stuck in an `UPDATING` or `DEPLOYING` state. This caused intermittent communication issues between control plane components and worker nodes. **Root Cause** The instability was caused by our control plane management system running out of memory and crashing under high load. When the system restarted, it was unable to properly resume its interrupted tasks, which left several clusters stuck mid-update. **Resolution** Our engineering teams quickly stabilized the system by increasing its memory allocation. Once stable, engineers manually recovered the remaining clusters that were stuck to restore full service. **Prevention** To prevent this from happening again, we are working to address an upstream software bug related to how the system handles memory-related crashes. Additionally, we are fixing an internal alerting issue that failed to notify our on-call team, ensuring a much faster response should a similar issue arise in the future.