Root incident

AWS Cape Town outage affecting dashboard and API

Critical Resolved View vendor source →

Root experienced a critical incident on September 14, 2022 affecting API and Dashboard, lasting 2h 5m. The incident has been resolved; the full update timeline is below.

Started
Sep 14, 2022, 10:57 AM UTC
Resolved
Sep 14, 2022, 01:03 PM UTC
Duration
2h 5m
Detected by Pingoru
Sep 14, 2022, 10:57 AM UTC

Affected components

APIDashboard

Update timeline

  1. investigating Sep 14, 2022, 10:57 AM UTC

    Our service provider AWS is experiencing issues. We are monitoring closely.

  2. investigating Sep 14, 2022, 11:02 AM UTC

    Currently seems to have only affected our web dashboard - the API seems to be operational. We are monitoring closely.

  3. monitoring Sep 14, 2022, 11:46 AM UTC

    AWS has confirmed that they are experiencing elevated API errors and latency in the AF-SOUTH-1 region, which impacts services used by our dashboard, API & database.

  4. monitoring Sep 14, 2022, 12:39 PM UTC

    AWS services are starting to resolve themselves and our dashboard is back up. The API seems to be mostly operational as well, we are still monitoring closely to confirm that all the related services are back up and running.

  5. resolved Sep 14, 2022, 01:03 PM UTC

    All AWS services are operational again and our dashboard & API are back to normal.