Helium incident

Degraded Data Transfer in U.S.

Major Resolved View vendor source →

Helium experienced a major incident on June 26, 2023 affecting Helium Packet Router, lasting 4h 10m. The incident has been resolved; the full update timeline is below.

Started
Jun 26, 2023, 04:37 PM UTC
Resolved
Jun 26, 2023, 08:47 PM UTC
Duration
4h 10m
Detected by Pingoru
Jun 26, 2023, 04:37 PM UTC

Affected components

Helium Packet Router

Update timeline

  1. investigating Jun 26, 2023, 04:37 PM UTC

    Due to a high amount of packet rate, the data transfer in U.S. may be experiencing degraded service. The core team is on it and we will provides updates here.

  2. investigating Jun 26, 2023, 04:37 PM UTC

    We are continuing to investigate this issue.

  3. investigating Jun 26, 2023, 05:11 PM UTC

    The issue was caused by the packet routers in U.S. running out of space due to multi-buy service logs. The core team is on it and we will provides updates here.

  4. identified Jun 26, 2023, 06:06 PM UTC

    The core team is going to patch the packet routers in U.S. with disk space optimization, and we will monitor the performance.

  5. identified Jun 26, 2023, 07:34 PM UTC

    The core team has also upgraded the U.S. packet routers, we will continue monitoring the performance.

  6. resolved Jun 26, 2023, 08:47 PM UTC

    Based on the monitoring, all packet routers have fully stabilized.