h8lio Outage History

h8lio is up right now

h8lio had 12 outages in the last 2 years totaling 226h 18m of downtime, averaging 0.5 incidents per month.

There were 12 h8lio outages since October 30, 2024 totaling 226h 18m of downtime. Each is summarised below. Incident details, duration, and resolution information.

Source: h8lio.statuspage.io

Minor May 22, 2026

node21 - hardware issue

Detected by Pingoru
May 22, 2026, 10:26 AM UTC
Resolved
May 22, 2026, 01:49 PM UTC
Duration
3h 23m
Affected: kube.h8l.iomonitoring.h8l.io
Timeline · 4 updates
  1. identified May 22, 2026, 10:26 AM UTC

    The hosting provider has planed an intervention soon. In the same time, we have drained the nodes' pods and scripting the Ceph volume attachement release for pods with mounted volumes.

  2. identified May 22, 2026, 11:08 AM UTC

    Problem identified by the hosting provider : https://bare-metal-servers.status-ovhcloud.com/incidents/lncbjp1744g2 On our side, the Ceph volume attachments cleaning process is in progress. You should soon see your pods with stuck mounted volumes rescheduled.

  3. monitoring May 22, 2026, 12:22 PM UTC

    A fix has been implemented and we are monitoring the results.

  4. resolved May 22, 2026, 01:49 PM UTC

    This incident has been resolved.

Read the full incident report →

Notice November 25, 2025

node03 down

Detected by Pingoru
Nov 25, 2025, 12:14 PM UTC
Resolved
Nov 25, 2025, 01:13 PM UTC
Duration
59m
Affected: kube.h8l.io
Timeline · 2 updates
  1. monitoring Nov 25, 2025, 12:14 PM UTC

    Node03 is currently down. It will be excluded from the cluster. ⚠️ Services running on this node may experience a short interruption during rescheduling to another node.

  2. resolved Nov 25, 2025, 01:13 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor November 3, 2025

node12 failure

Detected by Pingoru
Nov 03, 2025, 07:46 AM UTC
Resolved
Nov 03, 2025, 08:50 AM UTC
Duration
1h 3m
Affected: sources.h8l.iokube.h8l.iomonitoring.h8l.io
Timeline · 5 updates
  1. identified Nov 03, 2025, 07:46 AM UTC

    node12 is failing to respond. The hosting provider has planned an intervention in the next few minutes.

  2. identified Nov 03, 2025, 07:46 AM UTC

    We are continuing to work on a fix for this issue.

  3. identified Nov 03, 2025, 08:30 AM UTC

    We are continuing to work on a fix for this issue.

  4. monitoring Nov 03, 2025, 08:37 AM UTC

    The incident has been resolved. The power supply unit (PSU) has been replaced and the node is now operational.

  5. resolved Nov 03, 2025, 08:50 AM UTC

    This incident has been resolved.

Read the full incident report →

Notice October 31, 2025

node47 removed

Detected by Pingoru
Oct 31, 2025, 09:00 AM UTC
Resolved
Oct 31, 2025, 09:00 AM UTC
Duration
–
Timeline · 1 update
  1. resolved Oct 31, 2025, 09:53 AM UTC

    the node47 has been removed.

Read the full incident report →

Minor September 23, 2025

Electric issue on 3 nodes

Detected by Pingoru
Sep 23, 2025, 12:00 PM UTC
Resolved
Sep 23, 2025, 02:30 PM UTC
Duration
2h 29m
Affected: h8l.iosources.h8l.iokube.h8l.ioci.h8l.ioregistry.h8l.iocd.h8l.ioapps.h8l.io
Timeline · 3 updates
  1. identified Sep 23, 2025, 12:00 PM UTC

    We experienced an electric issue on the node4, node6 and node9. The nodes have been excluded temporarily from the Kubernetes cluster. You may experience a service interruption while the pods are rescheduled on another node.

  2. monitoring Sep 23, 2025, 12:11 PM UTC

    A fix has been implemented and we are monitoring the results.

  3. resolved Sep 23, 2025, 02:30 PM UTC

    This incident has been resolved.

Read the full incident report →

Major July 23, 2025

nodes not ready

Detected by Pingoru
Jul 23, 2025, 05:23 AM UTC
Resolved
Jul 23, 2025, 08:04 AM UTC
Duration
2h 40m
Affected: h8l.iokube.h8l.io
Timeline · 6 updates
  1. identified Jul 23, 2025, 05:23 AM UTC

    The node23 is not ready. We identified the issue and the resolution is in progress.

  2. monitoring Jul 23, 2025, 06:09 AM UTC

    A fix has been implemented and we are monitoring the results.

  3. identified Jul 23, 2025, 06:35 AM UTC

    Several nodes are not ready (31,32,33) in the same region. We are investigating the issue and check if there is any problem is reported on the hosting provider side.

  4. investigating Jul 23, 2025, 06:36 AM UTC

    We are currently investigating this issue.

  5. monitoring Jul 23, 2025, 07:02 AM UTC

    The region issue is resolved, we are monitoring it

  6. resolved Jul 23, 2025, 08:04 AM UTC

    This incident has been resolved.

Read the full incident report →

Notice March 24, 2025

osd.7 down

Detected by Pingoru
Mar 24, 2025, 11:30 PM UTC
Resolved
Mar 24, 2025, 11:30 PM UTC
Duration
–
Timeline · 1 update
  1. resolved Mar 25, 2025, 06:58 AM UTC

    The failing Ceph disk osd.7 has been replaced.

Read the full incident report →

Major February 28, 2025

[node06] unscheduled reboot

Detected by Pingoru
Feb 28, 2025, 05:00 PM UTC
Resolved
Feb 28, 2025, 05:00 PM UTC
Duration
–
Timeline · 1 update
  1. resolved Mar 01, 2025, 12:55 AM UTC

    Node node06 has been rebooted by the hosting provider due to no response to ping. We are investigating the reason for this reboot and plan to replace this node to prevent further issues.

Read the full incident report →

Major February 14, 2025

[kubernetes] dashboard not available

Detected by Pingoru
Feb 14, 2025, 09:04 AM UTC
Resolved
Feb 14, 2025, 11:09 AM UTC
Duration
2h 5m
Affected: kube.h8l.io
Timeline · 3 updates
  1. investigating Feb 14, 2025, 09:04 AM UTC

    The address of the Kubernetes dashboard kube.h8l.io is not available. The other services are not impacted by this issue and you still can use the command line kubectl.

  2. monitoring Feb 14, 2025, 09:14 AM UTC

    We forced the kube.h8l.io DNS address onto the h8lio's edge router IP. The Kubernetes Dashboard should be available now (may be affected by the DNS record propagation time).

  3. resolved Feb 14, 2025, 11:09 AM UTC

    This incident has been resolved.

Read the full incident report →

Notice October 30, 2024

Backbone incident

Detected by Pingoru
Oct 30, 2024, 02:03 PM UTC
Resolved
Oct 30, 2024, 03:01 PM UTC
Duration
57m
Affected: h8l.ioedge.h8l.io
Timeline · 3 updates
  1. monitoring Oct 30, 2024, 02:03 PM UTC

    You might experienced instabilities reaching our network & services. Reference: https://network.status-ovhcloud.com/incidents/qgb1ynp8x0c4

  2. resolved Oct 30, 2024, 03:01 PM UTC

    This incident has been resolved.

  3. postmortem Oct 31, 2024, 06:12 AM UTC

    The reason of this incident appears to be the missing RPKI configuration on a backbone peer located in the USA. The peer has been updated with worldwide RPKI configuration.

Read the full incident report →