Funio incident

Critical issue with servers s071 - s104

Critical Resolved View vendor source →

Funio experienced a critical incident on January 8, 2023 affecting Shared Hosting, lasting 5h 16m. The incident has been resolved; the full update timeline is below.

Started
Jan 08, 2023, 02:11 PM UTC
Resolved
Jan 08, 2023, 07:27 PM UTC
Duration
5h 16m
Detected by Pingoru
Jan 08, 2023, 02:11 PM UTC

Affected components

Shared Hosting

Update timeline

  1. identified Jan 08, 2023, 02:11 PM UTC

    After the maintenance was completed yesterday afternoon, we noticed slowness on some servers, which we attributed to a consistency check running on the RAID after the maintenance. This task typically increases the I/O load. After the consistency check was completed this morning, the servers' load was still higher than usual. After further investigation, we have concluded that the updated software was causing the problem on specific servers running a particular hardware configuration. We have started a new maintenance to fix the issues by reverting what was updated yesterday. We apologize for any inconvenience this may cause you.

  2. identified Jan 08, 2023, 04:30 PM UTC

    Servers s091 to s104 are now back up and accessible. We continue the maintenance for servers s071 to s084.

  3. monitoring Jan 08, 2023, 06:07 PM UTC

    Servers s071 to s084 are now back up and accessible.

  4. identified Jan 08, 2023, 06:10 PM UTC

    Every servers are accessible now.

  5. monitoring Jan 08, 2023, 06:14 PM UTC

    A fix has been implemented and we are monitoring the results.

  6. resolved Jan 08, 2023, 07:27 PM UTC

    All servers and services are operational. Thank you for your patience and understanding.