US Central - v.2 Platform - Single Physical Server Issue

Update timeline

1. investigating May 02, 2022, 01:28 PM UTC

   Our technical team are investigating performance impacts to a single physical server in our US Central data center at the moment. As soon as we know more we will update this incident.

2. identified May 02, 2022, 02:04 PM UTC

   We have identified a critical issue in the storage system of this single physical server. We have placed the storage system in read-only mode to help protect data while we migrate each client over to a different physical server and restore service to each as the migration of data completes. We will update this incident as the migration process continues.

3. identified May 02, 2022, 02:52 PM UTC

   Migrations are moving along smoothly, but data migration of course does take time depending on the amount of storage each client's VPS server contains. We will update again as this process progresses further.

4. identified May 02, 2022, 03:21 PM UTC

   The process is proceeding smoothly, ~15-20% of clients have been restored, but the process time solely depends on the amount of data that needs to migrate between systems per client. We will update more as the progress continues.

5. identified May 02, 2022, 03:42 PM UTC

   The data migration process is going well, we are at about 40% restoration of service for impacted clients at this point.

6. identified May 02, 2022, 03:55 PM UTC

   Restoration is now at about 60% of impacted clients being back online.

7. identified May 02, 2022, 04:16 PM UTC

   Right at the 80% restoration of service mark.

8. identified May 02, 2022, 04:38 PM UTC

   We are at just about 95% restoration. The remaining ones are just waiting for a few really large storage usage systems in front of them to finish up the data transfers.

9. resolved May 02, 2022, 05:04 PM UTC

   This incident has been resolved.