M.D.G. IT incident

Storage outage in SY1

Notice Resolved View vendor source →

M.D.G. IT experienced a notice incident on November 8, 2017, lasting 59m. The incident has been resolved; the full update timeline is below.

Started
Nov 08, 2017, 05:10 AM UTC
Resolved
Nov 08, 2017, 06:09 AM UTC
Duration
59m
Detected by Pingoru
Nov 08, 2017, 05:10 AM UTC

Update timeline

  1. investigating Nov 08, 2017, 05:10 AM UTC

    We're investigating a storage outage in SY1, more details to follow at 4:15pm AEDT

  2. investigating Nov 08, 2017, 05:20 AM UTC

    A storage network failure has been detected, affected services have failed over to a functional network path. Some services will require a reboot, which is being initiated by M.D.G. IT systems administrators. Full availability to all services should be restored by 4:45pm AEDT, with the majority of services restored by 4:25pm AEDT.

  3. identified Nov 08, 2017, 05:43 AM UTC

    Server availability to most services has now been restored, with approximately 30 servers still requiring a reboot and / or manual fsck commands to be run. These should be restored in the next 5-10 minutes.

  4. resolved Nov 08, 2017, 06:09 AM UTC

    All services should now be restored. If you are encountering any errors please open a support ticket and our support team will address the issue immediately.