Ion Interactive experienced a critical incident on April 9, 2024 affecting ion interactive Platform, lasting 2h 29m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Apr 09, 2024, 05:09 PM UTC
We are experiencing an incident affecting some customers, our team is already investigating it, and we'll keep you updated as soon as we have more details.
- monitoring Apr 09, 2024, 05:30 PM UTC
Consoles are working as expected now, we'll keep them under close monitoring. Also, we'll conclude the investigation and share more details soon.
- resolved Apr 09, 2024, 07:38 PM UTC
Our Cloud Platform detected an underlying problem with the hardware hosting on one of our services, leading to a service disruption - the issue is reported as a hostError. To address the issue swiftly, our systems are configured to automatically restart servers and perform a live host migration. Unfortunately, due to the severity of the hardware issue, our live migration feature, which normally helps prevent such disruptions by seamlessly transferring VMs to healthy hardware, was unable to intervene effectively. We are actively collaborating with the cloud platform to investigate the root cause of the hardware/software issue and explore strategies to prevent its recurrence. Additionally, we are probing into why the live migration feature did not function as expected and whether defining a maintenance window for such restarts could mitigate similar incidents in the future. Rest assured, we are committed to implementing additional measures to enhance our platform's resilience and ensure smoother operations moving forward.