Ziron incident

[voice] Call quality issues

Minor Resolved View vendor source →

Ziron experienced a minor incident on February 6, 2018, lasting 5h 17m. The incident has been resolved; the full update timeline is below.

Started
Feb 06, 2018, 12:06 PM UTC
Resolved
Feb 06, 2018, 05:24 PM UTC
Duration
5h 17m
Detected by Pingoru
Feb 06, 2018, 12:06 PM UTC

Update timeline

  1. investigating Feb 06, 2018, 12:06 PM UTC

    We are investigating reports of call quality issues on some inbound and outbound call traffic in our London network. Details to follow.

  2. identified Feb 06, 2018, 12:26 PM UTC

    This issue appears to be a reoccurrence of a similar fault yesterday with a network suppliers core link. Traffic is in the process of being rerouted, and we are seeing an overall improvement in call quality.

  3. monitoring Feb 06, 2018, 01:48 PM UTC

    Call quality has remained at normal levels since the last update. We are continuing to monitor this issue.

  4. identified Feb 06, 2018, 01:58 PM UTC

    We are seeing a repeat of this problem, and have escalated with our network supplier.

  5. identified Feb 06, 2018, 02:47 PM UTC

    We have taken steps to reduce impact of these issues by moving the majority of our inbound call traffic to a backup data centre, run by an alternative supplier. We continue to work with our primary service supplier to resolve this issue.

  6. identified Feb 06, 2018, 04:02 PM UTC

    Senior engineers at our primary network supplier continue to investigate the issue, and have re-routed our LINX and LONAP public peering traffic via alternate paths. We continue to investigate, but overall are seeing an improvement in call quality.

  7. monitoring Feb 06, 2018, 04:22 PM UTC

    Our re-routed LINX and LONAP public peering traffic is stable, so we are now in the process of bring the previously rerouted call traffic back to our primary data centres. No current calls will be lost during this process.

  8. resolved Feb 06, 2018, 05:24 PM UTC

    The connectivity between our two primary London data centres and our LINX and LONAP peering has been rerouted, and we have not seen any reoccurrence of quality issues. We will provide a full RFO in the next few days, and apologise for the inconvenience this issue may have caused.