Resolved after 3 hours 44 minutes, on August 29, 2023.
The Nova Labs core team has stabilized the core network. Some updates that were deployed yesterday were triggered into a bad state early today. The team has rolled back the updates and the system has stabilized. Radio owners do not need to do any actions. CBRS users may need to airplane mode on/off if they have data connection issues. A post mortem will be performed to understand how the bad state was entered and how it can be prevented in the future. We'll update after monitoring for a few hours and mark resolved when the system has demonstrated prolonged stability after the rollback.
Update timeline
- investigating Aug 29, 2023, 05:36 PM UTC
The Nova Labs team received alerts early this morning indicating irregular load on the Helium 5G core network. This is resulting in failed CBRS data sessions for users. In addition, radios may intermittently go offline and come back. This issue only affects the CBRS side of Helium Mobile and subscribers are not impacted. The team is analyzing the source of the issue.
- monitoring Aug 29, 2023, 07:32 PM UTC
The Nova Labs core team has stabilized the core network. Some updates that were deployed yesterday were triggered into a bad state early today. The team has rolled back the updates and the system has stabilized. Radio owners do not need to do any actions. CBRS users may need to airplane mode on/off if they have data connection issues. A post mortem will be performed to understand how the bad state was entered and how it can be prevented in the future. We'll update after monitoring for a few hours and mark resolved when the system has demonstrated prolonged stability after the rollback.
- resolved Aug 29, 2023, 09:20 PM UTC
This incident has been resolved.
Your product runs on more than Helium Mobile
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services