I.T Communications Limited incident

Issue affecting all VoIP Services

Notice Resolved View vendor source →

I.T Communications Limited experienced a notice incident on June 5, 2018, lasting 13h 10m. The incident has been resolved; the full update timeline is below.

Started
Jun 05, 2018, 03:02 PM UTC
Resolved
Jun 06, 2018, 04:12 AM UTC
Duration
13h 10m
Detected by Pingoru
Jun 05, 2018, 03:02 PM UTC

Update timeline

  1. investigating Jun 05, 2018, 03:02 PM UTC

    We are aware of a issue at the moment which is causing issues with VoIP Services. We are trying to find the cause, it would appear a network fault somewhere. We do not believe the fault to be on our network or our carriers. So external to our network. We are currently running trace routes to see where the issue is.

  2. investigating Jun 05, 2018, 03:38 PM UTC

    We are still investigating the cause of the issue. We are seeing issues with the London Internet Exchange as all routes to and from Linx is getting packet loss.

  3. investigating Jun 05, 2018, 04:00 PM UTC

    We are now shutting down our connection to London Linx 1 - to hopefully workaround the issue.

  4. identified Jun 05, 2018, 05:58 PM UTC

    We have been informed that a 3rd party Router Line Card is the cause of the issues this afternoon. The line card has now been removed from service and a replacement is being fitted this evening. The Line card is responsible for delivering Linx Lon1 traffic to and from Volta Data Centre. Traffic that was routing to Lon1 will now go via Lon2 and we are seeing traffic levels return to normal.

  5. resolved Jun 06, 2018, 04:12 AM UTC

    We are pleased to confirm that the faulty line card on the 3rd party network core router was replaced at 3am this morning. We have restored London Linx 1 and can see traffic is routing as normal and therefore happy this issue is now fully resolved.