Elevio incident

Major AWS outages affecting our main API and event processing

Major Resolved View vendor source →

Elevio experienced a major incident on November 25, 2020, lasting 19h 7m. The incident has been resolved; the full update timeline is below.

Started
Nov 25, 2020, 02:30 PM UTC
Resolved
Nov 26, 2020, 09:37 AM UTC
Duration
19h 7m
Detected by Pingoru
Nov 25, 2020, 02:30 PM UTC

Update timeline

  1. investigating Nov 25, 2020, 08:23 PM UTC

    We are currently investigating downtime with a number of AWS services, we will report back when we have further information

  2. investigating Nov 25, 2020, 08:44 PM UTC

    The main issues are stemming from major AWS outages, which is bringing down a large portion of the internet at the moment. More information on the current status of AWS can be seen here: https://status.aws.amazon.com/ More details from other major applications being down: https://www.theverge.com/2020/11/25/21719396/amazon-web-services-aws-outage-down-internet

  3. investigating Nov 25, 2020, 11:40 PM UTC

    Service is slowly being restored. However, full recovery may take up to a few hours.

  4. monitoring Nov 26, 2020, 12:16 AM UTC

    A fix has been implemented and we are monitoring the results.

  5. investigating Nov 26, 2020, 12:32 AM UTC

    Our main API powering the Assistant has now recovered, however interaction events (article views, searches, module clicks, etc) aren't currently being processed. We continue to investigate the latter.

  6. monitoring Nov 26, 2020, 04:03 AM UTC

    Our event processing systems are gradually recovering. However, full recovery may take up to a few hours.

  7. resolved Nov 26, 2020, 09:37 AM UTC

    All systems are now operating normally.