RX Frame Errors on Fibre port at Volta DC on Site to Site 100G Link

Update timeline

1. investigating Feb 06, 2025, 06:15 PM UTC

   One of our 100G Fibre Wavelengths Between Volta Data Centre, 36-43 Great Sutton St, London EC1V 0AB and Sovereign House, 227 Marsh Wall, London E14 9SD Data Centres is triggering alarms on our monitoring systems for RX Frame Errors. As a precaution, we attended both locations today and swapped out the Fibre SFP Modules, to see if that cleared the errors. During the SFP Swap out, some active phone calls may have dropped / disconnected depending on how customers reach our network. There would have been two incidents where calls may have disconnected. Around 15:40 and 17:14 today. Unfortunately, the errors are still showing. We have therefore reported this to our wavelength provider to investigate.

2. identified Feb 06, 2025, 07:37 PM UTC

   Closer investigation has revealed that it seems likely our Wavelength provider "SSE Group" has a defective LR4 optic facing our equipment in Volta One (and only one) of the mux lanes has been transmitting gradually less and less power. We are now awaiting our Wavelength provider "SSE Group" to send someone out to Volta to replace a suspected LR4 Fibre Module failure. They have now agreed this is the likely cause and looking to dispatch an engineer to replace the LR4 Fibre Module.

3. identified Feb 06, 2025, 08:03 PM UTC

   Update from Our Wavelength provider part of the SSE Group. Field Engineer & part due onsite at Volta DC tonight at 23:00

4. resolved Feb 06, 2025, 11:19 PM UTC

   Our Wavelength provider part of the SSE Group has now replaced the faulty hardware and alerts on our monitoring is now clear.