Exascale experienced a notice incident on March 14, 2021 affecting WLV0 - Exascale, Wolverhampton, UK and MCR0 - Datum, Manchester, UK and 1 more component, lasting 1d 2h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Mar 14, 2021, 05:05 AM UTC
We’re continuing to observe instability in our Cisco IOS XR 9000 series routing platform. We have taken measures to minimise disruption while we implement a tactical fix.
- investigating Mar 14, 2021, 09:29 AM UTC
We've successfully resolved the bug in the RSPs in Wolverhampton. We're going to observe the situation then move on to the Telford router, we will then divert traffic away from Telford to go via Wolverhampton. Then reload Manchester RSPs later this evening.
- investigating Mar 14, 2021, 02:12 PM UTC
We are continuing to investigate this issue.
- monitoring Mar 14, 2021, 06:09 PM UTC
A fix has been implemented and we are monitoring the results.
- monitoring Mar 14, 2021, 06:13 PM UTC
We have stabilised the routing platform, we’re now monitoring the situation.
- resolved Mar 15, 2021, 07:31 AM UTC
We have seen over 12 hours of stability within the platform. Any further remedial actions will be carried out under a planned maintenance window.