Panoply incident

Platform UI degraded performance

Major Resolved View vendor source →

Panoply experienced a major incident on July 24, 2018 affecting Data Sources and Platform UI, lasting 8h 36m. The incident has been resolved; the full update timeline is below.

Started
Jul 24, 2018, 01:46 PM UTC
Resolved
Jul 24, 2018, 10:23 PM UTC
Duration
8h 36m
Detected by Pingoru
Jul 24, 2018, 01:46 PM UTC

Affected components

Data SourcesPlatform UI

Update timeline

  1. investigating Jul 24, 2018, 01:46 PM UTC

    Some customers are experiencing degraded performance while using the platform UI. We are currently investigating these issues.

  2. identified Jul 24, 2018, 03:38 PM UTC

    We have identified the issue and are currently working on restoring the platform to a fully operational status.

  3. identified Jul 24, 2018, 08:31 PM UTC

    The current issue is affecting the following: * sources page * jobs page * running jobs We’re working on resolving the issues and all jobs will be rescheduled

  4. monitoring Jul 24, 2018, 09:09 PM UTC

    We've fixed the issues and the data sources and UI components are now fully operational. The jobs that failed will be retried and all the pending jobs will start working shortly as we will start to ingest the jobs backlog. We are currently monitoring the system to make sure it is stable.

  5. resolved Jul 24, 2018, 10:23 PM UTC

    Monitoring as completed with no outstanding issues. The issue is resolved. The system is fully operational