Hyperstack incident

Storage Service Outage in CA1

Major Resolved View vendor source →

Hyperstack experienced a major incident on August 7, 2025 affecting Canada 1 Datacentre, lasting 1d 1h. The incident has been resolved; the full update timeline is below.

Started
Aug 07, 2025, 01:45 AM UTC
Resolved
Aug 08, 2025, 03:00 AM UTC
Duration
1d 1h
Detected by Pingoru
Aug 07, 2025, 01:45 AM UTC

Affected components

Canada 1 Datacentre

Update timeline

  1. investigating Aug 07, 2025, 01:45 AM UTC

    We're currently investigating a storage-related issue in our CA1 region. A subset of storage nodes is unavailable, which is affecting performance and availability for some virtual machines and operations that rely on block storage. - You may experience delays launching, hibernating, or snapshotting instances. - Instances booting from volume may be temporarily unavailable or exhibit degraded I/O performance. - General instance operation is unaffected if you are using root disk or ephemeral. Our engineering team is working with our hardware partner to restore full service as soon as possible. We’ll continue to provide updates here as more information becomes available.

  2. identified Aug 07, 2025, 02:20 AM UTC

    Our CA1 data centre experienced a power disruption. While critical infrastructure responded as expected, some backend systems remain affected and we are actively working with our partners to restore full service.

  3. monitoring Aug 07, 2025, 03:20 AM UTC

    All services are now restored in CA1. However, clients might still expected IOs degradation when our storage nodes are being rebalanced.

  4. resolved Aug 08, 2025, 03:00 AM UTC

    This incident has been resolved.