Ziron experienced a minor incident on December 8, 2018 affecting Outbound Voice and Number Lookup, lasting 1h 26m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Dec 08, 2018, 12:33 AM UTC
We are investigating an IP network outage at our primary London location. Services have been failed over to our secondary site.
- investigating Dec 08, 2018, 12:39 AM UTC
Services currently affected: outbound voice, number lookup (get_country and hashed_imsi), and London colocation customers. Inbound voice has failed over to our DR site and is unaffected. SMS, other lookups, dashboard and API are also unaffected.
- identified Dec 08, 2018, 12:45 AM UTC
Services are being restored. The issue appears to initially be around the inter-site VPLS links, but we are waiting for further details.
- monitoring Dec 08, 2018, 01:07 AM UTC
Services have been stable since 00:43 UTC. We are continuing to monitor and will provide another update by 01:30 UTC.
- monitoring Dec 08, 2018, 01:28 AM UTC
Our network team have identified the causes of the VPLS issues earlier, and are now confident that this issue is resolved. We will shortly perform a failover back to the primary London site, and expect to see a few seconds of potential disruption while external routing re-converges.
- monitoring Dec 08, 2018, 01:42 AM UTC
The restoration of our primary London site has been successful. We will continue to monitor stability before restoring inbound voice to this site.
- resolved Dec 08, 2018, 02:00 AM UTC
Service has now been stable and we are marking this incident as resolved. We apologise to those customers affected by this outage, an RFO will follow early next week