Helium Mobile experienced a critical incident on August 29, 2023, lasting 3h 44m. The incident has been resolved; the full update timeline is below.
Update timeline
- investigating Aug 29, 2023, 05:36 PM UTC
The Nova Labs team received alerts early this morning indicating irregular load on the Helium 5G core network. This is resulting in failed CBRS data sessions for users. In addition, radios may intermittently go offline and come back. This issue only affects the CBRS side of Helium Mobile and subscribers are not impacted. The team is analyzing the source of the issue.
- monitoring Aug 29, 2023, 07:32 PM UTC
The Nova Labs core team has stabilized the core network. Some updates that were deployed yesterday were triggered into a bad state early today. The team has rolled back the updates and the system has stabilized. Radio owners do not need to do any actions. CBRS users may need to airplane mode on/off if they have data connection issues. A post mortem will be performed to understand how the bad state was entered and how it can be prevented in the future. We'll update after monitoring for a few hours and mark resolved when the system has demonstrated prolonged stability after the rollback.
- resolved Aug 29, 2023, 09:20 PM UTC
This incident has been resolved.