Umbrellar experienced a minor incident on March 9, 2019 affecting Auckland, lasting 1d. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Mar 09, 2019, 11:05 PM UTC
A core network gateway has failed. Due to redundancy the second gateway has taken over the functionality. There is no direct impact for our customers but a degraded service until the gateway has been replaced. Our team is currently investigating the failure.
- investigating Mar 10, 2019, 06:24 AM UTC
The investigation so far has surfaced the nature of the failure. Remediation work is underway by our team.
- identified Mar 10, 2019, 08:31 AM UTC
The issue with the switch has been identified as a software fault causing it to reboot and prevent from starting up cleanly. The config on the switch has been verified and restarted in isolation, clearing it of any issues. Our team are working now to restore the switch into the network.
- monitoring Mar 10, 2019, 08:59 AM UTC
Work has been completed to re-introduce the switch into topology, functionality has been tested and under monitoring since 21.30hrs.
- resolved Mar 11, 2019, 12:05 AM UTC
This incident has been resolved.