Divio incident

RabbitMQ cluster degraded

Minor Resolved View vendor source →

Divio experienced a minor incident on March 29, 2021 affecting Hosting, lasting 32m. The incident has been resolved; the full update timeline is below.

Started
Mar 29, 2021, 10:04 AM UTC
Resolved
Mar 29, 2021, 10:36 AM UTC
Duration
32m
Detected by Pingoru
Mar 29, 2021, 10:04 AM UTC

Affected components

Hosting

Update timeline

  1. investigating Mar 29, 2021, 10:04 AM UTC

    We are investigating an issue with a RabbitMQ cluster serving sites in our EU region not responding in a timely manner. Messages ingest and delivery may be impacted.

  2. identified Mar 29, 2021, 10:20 AM UTC

    We have implemented a fix and are checking for any cleanup tasks that need to be performed.

  3. monitoring Mar 29, 2021, 10:29 AM UTC

    A fix has been implemented and we are monitoring the results.

  4. resolved Mar 29, 2021, 10:36 AM UTC

    This incident has been resolved. Applications running on the EU region using an outdated version of celery not automatically reconnecting to the message broker may need to be redeployed by reusing the current Docker image.