Helium Mobile experienced a major incident on September 26, 2024 affecting Wi-Fi Hotspots, lasting 3d 14h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Sep 26, 2024, 10:44 PM UTC
We're currently investigating an issue with coverage planner failures originating from an upstream service. This afftect coverage plans, and hotspot onboarding and location assertions that reset the coverage object for a hotspot.
- identified Sep 27, 2024, 12:09 AM UTC
The upstream modeled coverage service appears to have ceased working completely. A fallback is being deployed that sets outdoor WiFi hotspots to template coverage. CBRS radios will continue to have issues with coverage objects until a workaround is implemented.
- monitoring Sep 27, 2024, 01:12 AM UTC
Template outdoor WiFi coverage fallback has been deployed and activated and appears to be working. We'll continue to monitor the coverage service. The CBRS fallback for this service is being worked on and will likely get deployed tomorrow. CBRS will continue to be affected on add, re-assert or planner requests.
- monitoring Sep 27, 2024, 01:15 AM UTC
We are continuing to monitor for any further issues.
- resolved Sep 30, 2024, 12:59 PM UTC
CBRS template coverage fallvack was deployed on Suday, Sept 29th and monitored. Systems are operational again using templated coverage.