Qualified incident

An issue with our underlying Redis provider is causing background processing to be delayed

Minor Resolved View vendor source →

Qualified experienced a minor incident on September 19, 2019 affecting Standard Queue, lasting 49m. The incident has been resolved; the full update timeline is below.

Started
Sep 19, 2019, 04:55 PM UTC
Resolved
Sep 19, 2019, 05:45 PM UTC
Duration
49m
Detected by Pingoru
Sep 19, 2019, 04:55 PM UTC

Affected components

Standard Queue

Update timeline

  1. investigating Sep 19, 2019, 04:55 PM UTC

    We are currently investigating this issue.

  2. resolved Sep 19, 2019, 05:45 PM UTC

    This incident has been resolved.

  3. postmortem Sep 19, 2019, 05:45 PM UTC

    Our underlying Redis cloud provider experience a full outage. While this provider has fixed their issues, we have also already migrated to a new provider. Some items queued for processing may have been lost during this incident. Our system is designed to auto-detect key data that is out of state and re-enqueue processing, so nothing major should be affected.