Adeptcore experienced a major incident on September 18, 2020 affecting ACP - Cloud-Hosted and ACP - Nodes, lasting 1d 8h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Sep 18, 2020, 07:30 PM UTC
We just received alerts of NODE06 going offline. Adeptcloud Engineers and INAP engineers are investigating the issue.
- identified Sep 18, 2020, 07:44 PM UTC
We have identified, the issue at this time, DCOPS is currently working on the server. We will provide an update in 15 minutes on status and restoration.
- identified Sep 18, 2020, 08:11 PM UTC
Affected host is currently being rebooted as ESXI crash occurred, current ETA for restoration is 30 minutes.
- monitoring Sep 18, 2020, 08:35 PM UTC
Server is back online now we are slowly bringing up all clients affected by this.
- monitoring Sep 18, 2020, 08:49 PM UTC
All virtual machines are coming back online now. We are confirming accessibility and functionality of the virtual machines that are coming online. We have a preliminary diagnosis of a Kernel Panic that occurred due to one of the CPU's. We are currently monitoring the environment and will be sending out a full synopsis once the environment is confirmed to be fully functional.
- monitoring Sep 18, 2020, 09:33 PM UTC
All systems appear to be operational at this time and we will continue monitoring performance throughout the weekend. We are still investigating the exact cause of the server lock up.
- resolved Sep 25, 2020, 02:00 PM UTC
This incident has been resolved.