Arcustech incident

US West - Single Physical Server Issues

Resolved after 1 hour 56 minutes, affecting US - West, on May 9, 2020.

resolved May 09, 2020, 11:47 PM UTC

All client systems are back online related to this incident, and we are monitoring things through the evening too. We do still have a maintenance window going on as well.

Started
May 09, 2020, 09:51 PM UTC
Resolved
May 09, 2020, 11:47 PM UTC
Duration
1h 56m
Detected by Pingoru
May 09, 2020, 09:51 PM UTC

Affected components

US - West

Update timeline

  1. investigating May 09, 2020, 09:51 PM UTC

    Staff are investigating an issue with a single physical server that is impacting clients currently. This is not related to network maintenance that is currently going on, initial investigation is showing a storage system issue. We will update this incident as we know more.

  2. identified May 09, 2020, 10:31 PM UTC

    The issue on this host is currently pointing to some storage issues. We are working on pinning down if this is a correctable issue, or if we will need replace components and restore client machines from full system snapshots and/or add-on file level backups for clients who purchase that add-on. We will update this as we have more details.

  3. identified May 09, 2020, 10:58 PM UTC

    False alarm, with the help of Ubuntu Advantage support we were able to isolate the issue, and client VPS servers will be coming back online shortly with no data loss. Sorry for the scare there for a bit, and sorry for the downtime this has caused today. We will update this to Monitoring once all client VPSs are back online.

  4. identified May 09, 2020, 11:21 PM UTC

    We are more than half way through verifying and restarting each client's VPS server. The rest will be online short.

  5. resolved May 09, 2020, 11:47 PM UTC

    All client systems are back online related to this incident, and we are monitoring things through the evening too. We do still have a maintenance window going on as well.

Your product runs on more than Arcustech

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →