Courier incident

Automation Delay Processing Issues

Minor Resolved View vendor source →

Courier experienced a minor incident on January 11, 2025 affecting Automations, lasting 3h 2m. The incident has been resolved; the full update timeline is below.

Started
Jan 11, 2025, 01:23 AM UTC
Resolved
Jan 11, 2025, 04:25 AM UTC
Duration
3h 2m
Detected by Pingoru
Jan 11, 2025, 01:23 AM UTC

Affected components

Automations

Update timeline

  1. identified Jan 11, 2025, 01:23 AM UTC

    We are currently experiencing an issue affecting automation workflows that include delay steps. Some customers may encounter failures when attempting to execute automations with scheduled delays. Our engineering team has identified the root cause and is implementing a fix. We expect the service to be fully restored in a short while.

  2. monitoring Jan 11, 2025, 02:19 AM UTC

    We have deployed a fix for the earlier issue affecting automation workflows with delay steps. New automation workflows are now executing successfully. However, automations that failed during the incident period (approximately 3:20 PM PT - 6:05 PM PT) have not yet been automatically retried. Our engineering team is monitoring the recovery and evaluating a plan to process these backlogged automations.

  3. resolved Jan 11, 2025, 04:25 AM UTC

    The service issue affecting automation workflows with delay steps has been partially resolved. New automation workflows started after 6:05 PM PT are executing successfully. However, we have determined that automations that failed during the incident period (approximately 3:20 PM PT - 6:05 PM PT) cannot be automatically retried at this time due to technical limitations. If your business was impacted by failed automation runs during this incident, please contact our support team and we will work directly with you to address your concerns. We sincerely apologize for this disruption to your workflows.