SENKO incident

Node Outage

Major Resolved View vendor source →

SENKO experienced a major incident on January 23, 2026, lasting 15m. The incident has been resolved; the full update timeline is below.

Started
Jan 23, 2026, 04:00 AM UTC
Resolved
Jan 23, 2026, 04:16 AM UTC
Duration
15m
Detected by Pingoru
Jan 23, 2026, 04:00 AM UTC

Update timeline

  1. investigating Jan 23, 2026, 03:50 AM UTC

    Unfortunately, due to an unexpected issue with the node's NIC, network connectivity became unavailable. As the IPMI console was also inaccessible, we had no alternative but to request Remote Hands support from the data center. The data center engineers are currently inspecting the server and will resolve the issue as soon as possible.

  2. investigating Jan 23, 2026, 04:06 AM UTC

    We have now regained direct access to the node via IPMI. As the server became unavailable during the outage, a hard reboot was required to restore access. Since the server had been running continuously for a prolonged period, the motherboard has initiated RAM training. This is expected behavior on DDR5-based systems and may take up to 15-20 minutes. We will provide an update as soon as the node is back up online.

  3. resolved Jan 23, 2026, 04:16 AM UTC

    The node is now back online, and the issue has been fully resolved.