Ziron incident
[lookups] Issues with get_country and hashed_imsi services
Ziron experienced a minor incident on June 19, 2018 affecting Number Lookup, lasting 57m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jun 19, 2018, 02:29 PM UTC
A connectivity issue between Ziron and an operator partner is currently affecting the following lookup services: get_country hashed_imsi
- identified Jun 19, 2018, 03:06 PM UTC
Investigation by Ziron has confirmed that this is not a connectivity issue, but instead appears to be an outage at an upstream operator partner.
- identified Jun 19, 2018, 03:11 PM UTC
The operator partner concerned have issued a major service outage notification and confirmed that engineers are working on the issue. They have advised the next update will be at 16:00 UTC.
- resolved Jun 19, 2018, 03:26 PM UTC
This incident has been resolved.
- postmortem Aug 01, 2018, 07:36 PM UTC
Our operator partner have advised that a bug in their firewall - in particular within the Cisco Adaptive Security Device Manager (ASDM) software on the device - resulted in Ziron being unable to access services within their network via our established VPN connections. The problem was detected by Ziron at 14:22 UTC, and by the operator at 14:30 UTC. Ziron saw resolution of the issue at 15:17 UTC, and the operator resolving this issue with all affected VPN connections issued at 15:19 UTC. The operator partner have scheduled an upgrade of the ASDM software by July 14th. We also understand that any further VPN configuration will be deployed via CLI until the ASDM software has been upgraded.