Endless Group incident

Dead dedicated Server

Major Resolved View vendor source →

Endless Group experienced a major incident on August 8, 2019 affecting DirectAdmin and Homepage/Signups and 1 more component, lasting 2d. The incident has been resolved; the full update timeline is below.

Started
Aug 08, 2019, 10:40 PM UTC
Resolved
Aug 10, 2019, 10:49 PM UTC
Duration
2d
Detected by Pingoru
Aug 08, 2019, 10:40 PM UTC

Affected components

DirectAdminHomepage/SignupsCloudflare Authoritative DNSApacheHelpdeskFTPEmail ServerMySQLEximBilling

Update timeline

  1. identified Aug 08, 2019, 09:16 PM UTC

    Our Dedicated Server has died completely. We are trying to come up with solutions on how to handle this if our Dedi does not come back online in the Datacenter. ( Don't worry, we have a Backup of everything)

  2. identified Aug 08, 2019, 10:40 PM UTC

    Edit: The staff mailserver never went offline, the client one is still down. (Exim = client email)

  3. monitoring Aug 10, 2019, 04:15 AM UTC

    We're running off a backup node for now so expect some degraded performance, but sites are currently back online. We do not have an ETA yet for our main node coming back online, but we will continue to update here as soon as we know more.

  4. monitoring Aug 10, 2019, 10:48 PM UTC

    We're back online, as of about 4 hours ago! Isn't this an efficient statuspage!!

  5. resolved Aug 10, 2019, 10:49 PM UTC

    [closing, see previous update]