Adeptcore incident

Elk Datacenter Network Issue

Major Resolved View vendor source →

Adeptcore experienced a major incident on February 16, 2023 affecting ACP - Network, lasting 35d. The incident has been resolved; the full update timeline is below.

Started
Feb 16, 2023, 04:53 PM UTC
Resolved
Mar 23, 2023, 04:59 PM UTC
Duration
35d
Detected by Pingoru
Feb 16, 2023, 04:53 PM UTC

Affected components

ACP - Network

Update timeline

  1. identified Feb 16, 2023, 04:53 PM UTC

    We are aware of network issues regarding one of the firewalls at our Elk Datacenter. It appears that the primary firewall went down and failed over to the secondary firewall as expected. We are already seeing devices come back online with full connectivity restored. We are currently investigating what caused the primary firewall to go offline and will update this as we get more information.

  2. monitoring Feb 16, 2023, 08:33 PM UTC

    At this time we have confirmed that all services are back up and running from the outage. We are continuing to investigate the cause and will update once we have identified and resolved the issue.

  3. resolved Mar 23, 2023, 04:59 PM UTC

    issue was due to a kernel panic on the primary switch. Failover worked as expected to bring services back online as quickly as possible.