Panoply incident

Issues with starting jobs

Major Resolved View vendor source →

Panoply experienced a major incident on November 25, 2020 affecting Job System, lasting 3h 40m. The incident has been resolved; the full update timeline is below.

Started
Nov 25, 2020, 03:26 PM UTC
Resolved
Nov 25, 2020, 07:06 PM UTC
Duration
3h 40m
Detected by Pingoru
Nov 25, 2020, 03:26 PM UTC

Affected components

Job System

Update timeline

  1. monitoring Nov 25, 2020, 03:26 PM UTC

    We are encountering issues due to a major outage in AWS itself. We are in contact with AWS representatives and are working to resolve the issue.

  2. monitoring Nov 25, 2020, 05:38 PM UTC

    Jobs are returning to their normal scheduling behavior. We are continuing to monitor the situation to verify that everything works smooth.

  3. monitoring Nov 25, 2020, 07:06 PM UTC

    We are continuing to monitor for any further issues.

  4. resolved Nov 25, 2020, 07:06 PM UTC

    This incident has been resolved.