dataforest incident

Power Feed Outage in Rack X10

Minor Resolved View vendor source →

dataforest experienced a minor incident on October 16, 2025 affecting Dedicated Servers, lasting 23h 38m. The incident has been resolved; the full update timeline is below.

Started
Oct 16, 2025, 11:52 PM UTC
Resolved
Oct 17, 2025, 11:30 PM UTC
Duration
23h 38m
Detected by Pingoru
Oct 16, 2025, 11:52 PM UTC

Affected components

Dedicated Servers

Update timeline

  1. identified Oct 17, 2025, 12:07 AM UTC

    One of the power feeds in X10 (containing only dedicated servers) has failed, while the other remains operational. Servers equipped with redundant power supplies are unaffected. Systems with only a single PSU may be impacted, depending on which feed they are connected to, resulting in roughly a 50% chance of downtime. If your server is currently unreachable, please rest assured that we are already addressing the issue. There is no need to contact our support team.

  2. identified Oct 17, 2025, 12:48 AM UTC

    We unplugged all servers from the affected PDU, reinserted the fuse, and reconnected the servers one by one to identify which system caused the fuse to trip. Since none of the servers powered on, it became clear that the PDU itself is faulty. We are currently moving the server to another feed to boot the servers. We are also aware that this happend the second time. We are already in contact with the datacenter and the PDU manufacturer to permanently resolve the issue.

  3. monitoring Oct 17, 2025, 01:02 AM UTC

    Affected servers are currently booting with the first ones already being up.

  4. monitoring Oct 17, 2025, 01:18 AM UTC

    All servers are back on power. Should you still see any issues with your server, please check it via IPMI before reaching out to our support. The technicans will replace the defective PDU during the day and we need to move the servers back to the correct feed. We will keep you updated.

  5. monitoring Oct 17, 2025, 05:14 PM UTC

    The PDU has been replaced and we will shut down the affected servers at 18.10.2025 - 01:00 CET, reconnect them to the correct feed, and then restart the servers.

  6. resolved Oct 17, 2025, 11:30 PM UTC

    This incident has been resolved.