Ooma experienced a major incident on December 17, 2018 affecting Vancouver POP, lasting 1d 4h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Dec 17, 2018, 07:16 PM UTC
We are aware of an issue impacting phones phones registered to our Vancouver POP. Customer phones will failover to our backup POPs; this process takes up to 10 minutes in some cases. We are working on having this issue resolved as quickly as possible.
- identified Dec 17, 2018, 07:24 PM UTC
We are continuing to work on solving the issue as quickly as possible. Some phones will have already failed back to VAN. Inbound calls are being delivered intermittently.
- monitoring Dec 17, 2018, 07:26 PM UTC
Services have been restored. Phones that failed over to backup POPs will reconnect to VAN automatically as their registrations refresh. Inbound and Outbound calls are being delivered. We will continue to monitor the situation and provide further updates as needed.
- identified Dec 17, 2018, 09:30 PM UTC
We have seen instability return to one of our routers in our VAN POP. We are investigating and will have the issue remediated as soon as possible.
- monitoring Dec 17, 2018, 09:46 PM UTC
All services in VAN have been restored. We will continue to monitor the situation an post updates here as needed.
- monitoring Dec 17, 2018, 09:51 PM UTC
We are continuing to monitor for any further issues.
- identified Dec 18, 2018, 06:52 PM UTC
We are aware of an issue impacting phones phones registered to our Vancouver POP. Customer phones will failover to our backup POPs; this process takes up to 10 minutes in some cases. We are working on having this issue resolved as quickly as possible.
- identified Dec 18, 2018, 07:08 PM UTC
We are continuing to work on solving the issue as quickly as possible. Some phones will have already failed back to VAN. Inbound calls are being delivered intermittently.
- monitoring Dec 18, 2018, 07:12 PM UTC
Services have been restored. Phones that failed over to backup POPs will reconnect to VAN automatically as their registrations refresh. Inbound and Outbound calls are being delivered. We will continue to monitor the situation and provide further updates as needed.
- resolved Dec 18, 2018, 11:49 PM UTC
We have returned to normal operations. Our team is still investigating the root cause of the issue and we will provide a post mortem via this page when it is available.