Is Elite UK down?

Type: Incident Duration: 7 hours and 18 minutes Affected Components: Wavelength Services, Colocation Services May 25, 00:46:00 GMT+0 - Investigating - We are investigating a loss of resilience affecting our IXN POP. At 01:46 BST, one side of the IXN DWDM connectivity path reported both associated interfaces down. Services remain operational at this stage, but the site is operating with reduced resilience. Further updates will be provided as the investigation progresses. May 25, 01:39:00 GMT+0 - Identified - The issue has been identified as a loss of both DWDM connectivity paths into the IXN POP. At 02:39 BST, the remaining DWDM path also went down, causing a loss of connectivity to the IXN POP. Initial optical diagnostics indicate receive signal loss on the affected DWDM interfaces. We are aware of unauthorised activity by a data centre engineer in our rack involving cable movement, which is considered the most likely cause. A case has been raised with DC Techs to investigate and restore the affected connectivity and outstanding resilience as soon as possible. May 25, 01:51:00 GMT+0 - Monitoring - Connectivity to the IXN POP was restored at 02:51 BST. The confirmed full outage window was from 02:39:35 BST to 02:51:02 BST, with a total impact duration of 11 minutes and 27 seconds. Services are operational again; however, the POP remains in a reduced-resilience state while DC Techs continue work to restore the outstanding DWDM path. We are continuing to monitor the site until full resilience is restored. May 25, 04:51:00 GMT+0 - Monitoring - We are continuing to monitor the IXN POP following the earlier restoration of connectivity. Services remain operational; however, the POP is still operating in a reduced-resilience state while we await DC Techs restoring the outstanding affected DWDM path. This has been chased with DC Techs on multiple occasions. We will also be raising this incident, along with previous issues observed at this POP, with the account manager for further review and escalation. May 25, 07:04:00 GMT+0 - Monitoring - The outstanding IXN DWDM resilience issue appears to have been restored. At 08:04 BST, the previously affected DWDM interfaces returned to an UP state following work by data centre technicians. This indicates the downed DWDM link has been repatched/restored. IXN POP connectivity and path resilience are currently operational again. We are continuing to monitor for stability before marking this incident as resolved. May 25, 08:04:00 GMT+0 - Resolved - IXN POP connectivity and DWDM path resilience have remained stable following restoration of the affected DWDM link at 08:04 BST. This incident is now resolved. The confirmed full service outage lasted from 02:39:35 BST to 02:51:02 BST, with a customer-impacting duration of 11 minutes and 27 seconds. The reduced-resilience period lasted from 01:46:44 BST to 08:04:51 BST, with a total duration of 6 hours, 18 minutes and 7 seconds. We will be escalating this incident with the POP account manager and requesting a full Reason for Outage report from the data centre regarding the unauthorised rack access and disconnection/disturbance of patch leads within our rack.

We are investigating a loss of resilience affecting our IXN POP. At 01:46 BST, one side of the IXN DWDM connectivity path reported both associated interfaces down. Services remain operational at this stage, but the site is operating with reduced resilience. Further updates will be provided as the investigation progresses.

The issue has been identified as a loss of both DWDM connectivity paths into the IXN POP. At 02:39 BST, the remaining DWDM path also went down, causing a loss of connectivity to the IXN POP. Initial optical diagnostics indicate receive signal loss on the affected DWDM interfaces. We are aware of unauthorised activity by a data centre engineer in our rack involving cable movement, which is considered the most likely cause. A case has been raised with DC Techs to investigate and restore the affected connectivity and outstanding resilience as soon as possible.

Connectivity to the IXN POP was restored at 02:51 BST. The confirmed full outage window was from 02:39:35 BST to 02:51:02 BST, with a total impact duration of 11 minutes and 27 seconds. Services are operational again; however, the POP remains in a reduced-resilience state while DC Techs continue work to restore the outstanding DWDM path. We are continuing to monitor the site until full resilience is restored.

We are continuing to monitor the IXN POP following the earlier restoration of connectivity. Services remain operational; however, the POP is still operating in a reduced-resilience state while we await DC Techs restoring the outstanding affected DWDM path. This has been chased with DC Techs on multiple occasions. We will also be raising this incident, along with previous issues observed at this POP, with the account manager for further review and escalation.

The outstanding IXN DWDM resilience issue appears to have been restored. At 08:04 BST, the previously affected DWDM interfaces returned to an UP state following work by data centre technicians. This indicates the downed DWDM link has been repatched/restored. IXN POP connectivity and path resilience are currently operational again. We are continuing to monitor for stability before marking this incident as resolved.

IXN POP connectivity and DWDM path resilience have remained stable following restoration of the affected DWDM link at 08:04 BST. This incident is now resolved. The confirmed full service outage lasted from 02:39:35 BST to 02:51:02 BST, with a customer-impacting duration of 11 minutes and 27 seconds. The reduced-resilience period lasted from 01:46:44 BST to 08:04:51 BST, with a total duration of 6 hours, 18 minutes and 7 seconds. We will be escalating this incident with the POP account manager and requesting a full Reason for Outage report from the data centre regarding the unauthorised rack access and disconnection/disturbance of patch leads within our rack.

Type: Maintenance Duration: 5 hours and 30 minutes Affected Components: Wavelength Services, Carrier Interconnects May 22, 18:00:00 GMT+0 - Identified - A planned maintenance will take place from **19:00 to 21:00** for DWDM wavelength services and associated hardware upgrades. During this window: * Some wavelength services will be **fully unavailable** for periods within the duration of the maintenance. * Some NNI services will be **fully unavailable** for periods within the maintenance window. * **Dual-homed services** will operate with **reduced resiliency** and may experience **minor service disruption**. We will work to minimise impact and restore full service promptly following completion. May 22, 18:00:01 GMT+0 - Identified - Maintenance is now in progress May 22, 19:47:21 GMT+0 - Identified - Unfortunately, we have had to extend the maintenance window by an additional 2 hours due to an issue currently affecting the DCI services. At present, only a single wavelength service is impacted. Our on-site engineers are currently working with the data centre technicians to investigate and resolve issues relating to the pre-cabling within the facility. May 22, 21:43:34 GMT+0 - Identified - We have now confirmed with the data centre technicians that the issue relates to an incorrect patching matrix within the data centre pre-cabling, and is not caused by Elite equipment or services. The data centre team is currently correcting the patching, and we are awaiting confirmation before proceeding further with the planned maintenance activities. May 22, 22:06:43 GMT+0 - Identified - Please disregard any previous completion notice; this maintenance is still open and has been extended further. At present, only a single-wavelength service is still impacted. We are still awaiting confirmation from the data centre before proceeding further with the planned maintenance activities. May 22, 22:24:12 GMT+0 - Identified - The data centre technicians have now correctly patched the affected wavelength service and successfully restored service. They are currently verifying the remaining patching before we proceed with the next stage of the planned migrations. May 22, 23:30:00 GMT+0 - Completed - Maintenance has completed successfully.