SimpleVoIP experienced a major incident on November 2, 2022 affecting Core Voice Services and SimpleVoIP Admin Portal, lasting 19h 29m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Nov 02, 2022, 07:51 PM UTC
We are investigating reports of some calls to US-East and US-West not completing on our system. We will provide updates as soon as we have identified the cause and impact of this event.
- identified Nov 02, 2022, 08:01 PM UTC
Our engineers have routed traffic away from our US-West cluster, and preliminary metrics indicate this has restored normal call behavior across all 3 zones. We will continue to investigate the underlying cause of the failed calls on both US-West and US-East, but at this time we believe that all calls will complete without issue.
- monitoring Nov 02, 2022, 09:07 PM UTC
Once traffic had been routed away from our servers in US-West, our engineers were able to restart services on the affected servers. After testing all US-West services to confirm that the restart resolved the issue, we have restored traffic to default routing. All traffic is now completing normally over standard routes. We will continue to keep a close eye on these servers over the course of the evening to ensure that all services remain stable.
- resolved Nov 03, 2022, 03:20 PM UTC
Services have been stable since restoring default routing yesterday. This incident is now resolved.