Umbrellar experienced a major incident on June 11, 2019 affecting Christchurch, lasting 22h 13m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jun 11, 2019, 11:20 PM UTC
We are currently investigating an incident with one of the Storage Nodes within the Umbrellar Managed Compute cluster in Christchurch. Customer workloads may experience increased latency and/or storage issues. Our engineers are actively working on the issue and we will provide updates as they become available. If you have any questions, please let us know by email to [email protected].
- investigating Jun 11, 2019, 11:20 PM UTC
We are continuing to investigate this issue.
- identified Jun 12, 2019, 01:12 AM UTC
We have identified the root cause to be hardware failure, We are actively working with our vendor on a replacement part and continue to monitor the incident. Performance is back to normal. Where necessary, the effected machines have been rebooted. If you are experiencing further issues please get in touch by email to [email protected]
- monitoring Jun 12, 2019, 01:13 AM UTC
We have failed over to other nodes within the cluster and continue to monitor the incident.
- resolved Jun 12, 2019, 09:33 PM UTC
We have been monitoring the situation, there is no further impact to any workload on the effected storage node. If you have any questions, please let us know by email to [email protected].