Tideways incident

Processing queue is currently stopped due to failed deployment

Minor Resolved View vendor source →

Tideways experienced a minor incident on January 20, 2022 affecting Ingest / Data Collection, lasting 15m. The incident has been resolved; the full update timeline is below.

Started
Jan 20, 2022, 04:44 PM UTC
Resolved
Jan 20, 2022, 04:59 PM UTC
Duration
15m
Detected by Pingoru
Jan 20, 2022, 04:44 PM UTC

Affected components

Ingest / Data Collection

Update timeline

  1. investigating Jan 20, 2022, 04:44 PM UTC

    Due to a deployment failure the processing of queues is currently stopped and we are looking to fix the problem and restart processing in a few minutes.

  2. monitoring Jan 20, 2022, 04:52 PM UTC

    We have released a fix for the failed deployment that has lead to all workers restarting. We are now monitoring the backlog of tasks and hope it to complete within the next 5-10 minutes.

  3. resolved Jan 20, 2022, 04:59 PM UTC

    Monitoring, performance, trace and error tracking data is up to date now.