Real World Technology Solutions incident

Vocus Data Centre Failure

Minor Resolved View vendor source →

Real World Technology Solutions experienced a minor incident on March 12, 2023 affecting NBN and Intercapital Capacity and 1 more component, lasting 12h 10m. The incident has been resolved; the full update timeline is below.

Started
Mar 12, 2023, 10:35 PM UTC
Resolved
Mar 13, 2023, 10:45 AM UTC
Duration
12h 10m
Detected by Pingoru
Mar 12, 2023, 10:35 PM UTC

Affected components

NBNIntercapital CapacityPrivate WAN (MPLS VPN)

Update timeline

  1. identified Mar 12, 2023, 10:35 PM UTC

    A mass service disruption has been caused by the failure of power within a Vocus data centre in Melbourne affecting a number of network services, providers and content sources. While we do not have any equipment in this facility, some of our intercapital paths transit through it. One of our network authentication providers also utilises this facility to house some of their equipment. Customers may presently be unable to view customer account details, and authentication may also be impacted. We have seen services begin restoration at approximately 9:28am.

  2. identified Mar 12, 2023, 10:43 PM UTC

    Intercapital services were restored at 9:21am. Network authentication services are still affected.

  3. identified Mar 12, 2023, 11:22 PM UTC

    Power has been lost within the facility again. Vocus have provided a status notice for this incident at https://support.vocus.com.au/s/article/Current-Outages

  4. identified Mar 12, 2023, 11:22 PM UTC

    We are continuing to work on a fix for this issue.

  5. monitoring Mar 13, 2023, 01:06 AM UTC

    Authentication and API services have been restored, and inter capital services have been stable for approximately 1.5hrs. We continue to monitor the status of this event.

  6. monitoring Mar 13, 2023, 07:32 AM UTC

    Vocus have advised they will be performing further works to restore the remaining power rail this evening. We do not expect any impact to our services as a result of this action.

  7. resolved Mar 13, 2023, 10:45 AM UTC

    Power has now been restored to both the A and B feeds within the data centre facility. No further issues arising from this incident are expected.