Arcustech experienced a minor incident on June 25, 2014, lasting 4d 4h. The incident has been resolved; the full update timeline is below.
Update timeline
- scheduled Jun 25, 2014, 01:07 AM UTC
We will be carrying out hardware replacements in our core switch configuration that links us to our upstream providers. This is a rescheduled maintenance window from May 10th, 2014. There have been no recent issues with this core switch setup but we still feel the full replacement option is the best for long term stability, and to better postmortem the existing switches for full root cause analysis. This maintenance window WILL have outage periods, but the whole window will not be an outage, only periods of time within the maintenance window. The length of time for each step will vary depending on a few factors so we are simply going to say at this point there WILL be times of unavailability as we carry out the work. Staff will keep this status notice updated every ~15 minutes even if its to just say we are still working. If you have any questions at all about this needed work please open a support request with our support team at [email protected]. Work Summary: As mentioned in the Postmortem - http://status.arcustech.com/incidents/dmjq0thdvsnd we have had a few issues with this switch configuration and are not going to risk another relapse with this setup so we are fully replacing this switch configuration with new, fully tested systems. We will then run full evaluation on the removed switch configuration and if possible will fully production rate them again as cold-spares. If you have any questions at all, please let the Support Team know. Thank you. -Nevin Lyne -Founder & Director of Technology
- scheduled Jun 29, 2014, 02:00 AM UTC
Maintenance will begin as scheduled in 60 minutes.
- in progress Jun 29, 2014, 03:00 AM UTC
Scheduled maintenance is currently in progress. We will provide updates as necessary.
- monitoring Jun 29, 2014, 05:17 AM UTC
Work in this maintenance window has been completed but staff is going over everything, checking logs, remote monitoring and global remote client site availability as a verification step. Once we are satisfied with the verification process we will mark this as completed, but we are setting the Network Infrastructure back to fully operational from "degraded". As always if you have any questions at all please let our Support Team know. Thank you! -Nevin Lyne -Founder and Director of Technology
- completed Jun 29, 2014, 06:00 AM UTC
The scheduled maintenance has been completed.