Skylight incident

Data Processing Delay

Major Resolved View vendor source →

Skylight experienced a major incident on June 11, 2019 affecting Application, lasting 7h 2m. The incident has been resolved; the full update timeline is below.

Started
Jun 11, 2019, 08:12 PM UTC
Resolved
Jun 12, 2019, 03:14 AM UTC
Duration
7h 2m
Detected by Pingoru
Jun 11, 2019, 08:12 PM UTC

Affected components

Application

Update timeline

  1. investigating Jun 11, 2019, 08:12 PM UTC

    One of our workers is not making progress, we are investigating. Around 20% customer apps are affected.

  2. identified Jun 11, 2019, 08:24 PM UTC

    The worker is stuck due to ingesting malformed data, we are working on a patch.

  3. monitoring Jun 11, 2019, 09:02 PM UTC

    We have unblocked the worker.

  4. resolved Jun 12, 2019, 03:14 AM UTC

    The issue was resolved as of 14:30 Pacific Time.