SENKO incident

Emergency disk replacement on hel-9454p-zerda

Resolved after 43 minutes, affecting FI-EPYC ( 🇫🇮 HT [9454P] zerda) and FI-EPYC (🇫🇮 HT [9454P] zerda - VM Status), on November 8, 2026.

resolved Nov 08, 2026, 06:04 AM UTC

The affected node is back online and virtual machines are now powering back on. Services should return to normal over the next 10 minutes as each VM completes boot.

Started
Nov 08, 2026, 05:21 AM UTC
Resolved
Nov 08, 2026, 06:04 AM UTC
Duration
43m
Detected by Pingoru
Nov 08, 2026, 05:21 AM UTC

Affected components

FI-EPYC ( 🇫🇮 HT [9454P] zerda)FI-EPYC (🇫🇮 HT [9454P] zerda - VM Status)

Update timeline

  1. identified Nov 08, 2026, 05:21 AM UTC

    We are currently performing an emergency replacement of a failed NVMe drive from one of the RAID10 mirrors. All customer data is safe and intact. Affected VMs will be back online as soon as the disk replacement is complete and the node has been booted back into the OS. We'll post an update once service is restored.

  2. resolved Nov 08, 2026, 06:04 AM UTC

    The affected node is back online and virtual machines are now powering back on. Services should return to normal over the next 10 minutes as each VM completes boot.

Your product runs on more than SENKO

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →