Scrutinizer incident

Intermittent Unavailability of Web Frontend

Resolved after 10 hours 13 minutes, affecting Web, on October 25, 2019.

monitoring Oct 25, 2019, 01:15 PM UTC

The recovery process is still running, and we expect it to take another 2 hours before completing. Certain git operations are still degraded during this time. We will continuing to post status updates here.

Started
Oct 25, 2019, 06:01 AM UTC
Resolved
Oct 25, 2019, 04:14 PM UTC
Duration
10h 13m
Detected by Pingoru
Oct 25, 2019, 06:01 AM UTC

Affected components

Web

Update timeline

  1. investigating Oct 25, 2019, 06:01 AM UTC

    We are currently investigating this issue.

  2. identified Oct 25, 2019, 06:32 AM UTC

    The issue has been identified and a fix is being implemented.

  3. identified Oct 25, 2019, 06:44 AM UTC

    We have identified a degradation in the storage that serves git data for various parts of the platform. A recovery process has been triggered and is currently running.

  4. monitoring Oct 25, 2019, 08:28 AM UTC

    The recovery process is still running. All data is already available, but at a reduced redundancy. Due to the ongoing recovery process and restoring of full data redundancy, we are currently seeing git operations to take longer than usual. This can affect individual pages which display source code or access other git information.

  5. monitoring Oct 25, 2019, 09:25 AM UTC

    We are continuing to monitor for any further issues.

  6. monitoring Oct 25, 2019, 01:15 PM UTC

    The recovery process is still running, and we expect it to take another 2 hours before completing. Certain git operations are still degraded during this time. We will continuing to post status updates here.

  7. resolved Oct 25, 2019, 04:14 PM UTC

    This incident has been resolved.

Your product runs on more than Scrutinizer

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →