Hero Internet experienced a notice incident on March 1, 2023 affecting Voice, lasting 16h 24m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Mar 01, 2023, 05:54 PM UTC
Upstream transit carrier for Spark inbound calls is currently having a network wide outage and this is affecting incoming calls to our network from Spark callers. Engineers at the upstream carrier are working to restore. No ETA at this stage on resolution. Updates will follow.
- identified Mar 01, 2023, 08:11 PM UTC
Upstream carrier advises that their systems are restored and they expect to have their SBCs operational again shortly. No specific ETA on this but they are prioritising the SBC system restoration and expect service to be restored shortly. An update will be posted once service is restored.
- monitoring Mar 01, 2023, 09:36 PM UTC
Service appears to have been partially restored from our upstream carrier and calls starting to route to our network again from Spark. We are continuing to monitor the situation but we believe the calling issues should be resolved shortly as service is restored.
- monitoring Mar 01, 2023, 09:43 PM UTC
We are seeing around 30% of normal call volumes from Spark which indicates that a large number of calls are still failing from the Spark network so the current situation is 'partially' restored and we will continue to seek updates on the remaining call failures
- monitoring Mar 01, 2023, 09:54 PM UTC
Spark are working with upstream carrier to reset SS7 links which are in a 'timeout' state due to earlier issues. Once Spark have reset these links service should be restored for all callers
- monitoring Mar 01, 2023, 11:31 PM UTC
The Spark issues are ongoing due to a resource issue at Spark to work on the fault. We have started to reroute calls via an alternate path and have already moved a couple hundred high volume/priority numbers to get them back in service. Please contact our support team on 092420001 if you have priority numbers that need to be urgently rerouted and we can action that for you in a few minutes. Meanwhile we will continue rerouting affected numbers based on call volumes.
- monitoring Mar 02, 2023, 01:41 AM UTC
Engineers are still working hard on this issue, if you still have any priority numbers that need to be urgently rerouted please email "[email protected]" and we will do them ASAP.
- monitoring Mar 02, 2023, 10:16 AM UTC
We are continuing to monitor for any further issues.
- resolved Mar 02, 2023, 10:18 AM UTC
We believe that all inbound Spark calling issues are resolved. Please contact our support team if you are still having any issuesl.