Ooma incident
Upstream internet routing issue identified on San Jose POP - Degraded Performance
Ooma experienced a major incident on August 3, 2020 affecting San Jose POP, lasting 20h 35m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Aug 03, 2020, 08:56 PM UTC
An intermittent upstream internet routing issue has been identified in San Jose. This is causing intermittent call quality, call control, and dropped calls to San Jose. This issue is out of our direct control, however our engineers are attempting to adjust routing on our end to mitigate the impact this is having on our services.
- identified Aug 03, 2020, 10:19 PM UTC
We have identified an upstream network loop within GTT/NTT networks. Our upstream vendors are attempting to work around these networks on their end.
- monitoring Aug 04, 2020, 01:54 PM UTC
Routing issues on our SJC POP have been resolved at our data center after they removed the bad link yesterday evening. Our Network team will continue to monitor today.
- resolved Aug 04, 2020, 05:31 PM UTC
The routing issue at our SJC POP is now resolved. Our team will keep monitoring for the rest of the day to be sure, but all systems are currently fully operational