Gemfury incident

Elevated errors

Minor Resolved View vendor source →

Gemfury experienced a minor incident on March 16, 2020 affecting Repo endpoints and Uploads, lasting 1h 51m. The incident has been resolved; the full update timeline is below.

Started
Mar 16, 2020, 04:18 PM UTC
Resolved
Mar 16, 2020, 06:09 PM UTC
Duration
1h 51m
Detected by Pingoru
Mar 16, 2020, 04:18 PM UTC

Affected components

Repo endpointsUploads

Update timeline

  1. investigating Mar 16, 2020, 04:18 PM UTC

    Investigating elevated errors and reports of delayed upload processing.

  2. investigating Mar 16, 2020, 04:18 PM UTC

    We are continuing to investigate this issue.

  3. investigating Mar 16, 2020, 04:42 PM UTC

    Index rebuild queue has run away. Pausing new index rebuilds to allow workers to catch up.

  4. investigating Mar 16, 2020, 04:51 PM UTC

    Update from our platform provider: platform issues starting around 16:22 UTC

  5. investigating Mar 16, 2020, 05:07 PM UTC

    Queue has caught up. Indexing recent uploads now.

  6. investigating Mar 16, 2020, 05:18 PM UTC

    Uploads queue has been processed, and back to normal now. Still seeing some elevated latencies on repo endpoints.

  7. investigating Mar 16, 2020, 05:42 PM UTC

    Platform provider incident resolved upstream at 17:31 UTC

  8. resolved Mar 16, 2020, 06:09 PM UTC

    This incident has been resolved.