Adeptcore incident

PHX Datacenter outage

Critical Resolved View vendor source →

Adeptcore experienced a critical incident on April 26, 2023 affecting ACP - Storage, lasting 20d 22h. The incident has been resolved; the full update timeline is below.

Started
Apr 26, 2023, 10:27 PM UTC
Resolved
May 17, 2023, 08:54 PM UTC
Duration
20d 22h
Detected by Pingoru
Apr 26, 2023, 10:27 PM UTC

Affected components

ACP - Storage

Update timeline

  1. investigating Apr 26, 2023, 10:27 PM UTC

    We are aware of reports of VM's not responding and showing offline at our PHX Datacenter. We are investigating this and will let you know once we have more information

  2. identified Apr 26, 2023, 10:43 PM UTC

    The PHX Datacenter engineers have informed us that the issue is a networking issue affecting their SAN network. We are actively engaged with datacenter Engineers to resolve this issue. We will update this as more information and updates become available.

  3. monitoring Apr 26, 2023, 11:00 PM UTC

    We are seeing networking being restored and are working on ensuring everything is brought back up. We will post another update once everything appears to be back up and running.

  4. monitoring Apr 27, 2023, 12:02 AM UTC

    We have finished booting up the last of the servers. At this time, all servers should be online or coming online within the next couple of minutes. If you continue to see your servers offline or have troubles logging in, please submit an email to [email protected] for issues that require immediate action or submit a ticket to [email protected] for issues that can be resolved during normal business hours.

  5. resolved May 17, 2023, 08:54 PM UTC

    This incident has been resolved.