Courier experienced a major incident on October 24, 2024 affecting API, lasting 4h 10m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 24, 2024, 10:38 PM UTC
Courier has identified a delay in message processing for the messages API. We are currently investigating the root cause and will update you periodically.
- investigating Oct 24, 2024, 11:01 PM UTC
We are continuing to monitor the root cause of the delayed message processing.
- investigating Oct 24, 2024, 11:29 PM UTC
The team is still investigating and discussing the root cause internally. Messages are experiencing longer than normal processing times.
- identified Oct 25, 2024, 12:00 AM UTC
The team has identified the issue and is rolling out a fix for the message delays.
- identified Oct 25, 2024, 12:40 AM UTC
Fix has been deployed and will be live in around 1hr.
- identified Oct 25, 2024, 01:31 AM UTC
The team identified an issue with the latest fix and reverted it. A new fix has been published to mitigate the message delay issue. ETA to land in production ~45 minutes.
- monitoring Oct 25, 2024, 02:43 AM UTC
A fix has been implemented and delivery times are beginning to returning to expected levels. The team will continue to monitor.
- resolved Oct 25, 2024, 02:48 AM UTC
This incident has been resolved