LUMASERV incident
Degraded management of VMs running on ls-ds-34 hypervisor
LUMASERV experienced a minor incident on November 11, 2020 affecting Cloud server hosts, lasting 3d 20h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Nov 11, 2020, 10:32 PM UTC
Since today 6am CET the management of VMs running on hypervisor ls-ds-34 is degraded. This incident affects only the management via our customer interface and our API, not the processes running in the VM or the reachability. If you need to control your VM via our interface, please feel free to contact our customer support. We can find an individual solution for you until the general problem with the hypervisor is solved. For a permanent solution we will schedule one further maintenance where we will have to do a controlled reboot.
- identified Nov 14, 2020, 07:36 PM UTC
In order to regain the full manageability of all VMs running on this host we will do a scheduled reboot tomorrow (Sunday, 15th november) between 2 and 2.30am CET. To solve this problem generally we will replace the disk caused the problem and add a further redundancy to avoid eventually further problems. In our customer interface you can look at the VM detail page if you are affected by this maintenance. For questions please feel free to contact our support. We apologize for the inconvenience.
- resolved Nov 15, 2020, 05:28 AM UTC
The hypervisor has been rebooted between 2.25am and 2.27am CET. All VMs were up and running again after about three minutes. Furthermore we have implemented the announced further redundancy to prevent issues like this in future.