Courier experienced a minor incident on September 6, 2024 affecting API and Observability, lasting 23m. The incident has been resolved; the full update timeline is below.
Affected components
APIObservability
Update timeline
- identified Sep 06, 2024, 04:35 PM UTC
The Courier team has identified an issue with the send pipeline and event status updates causing queued messages and delayed webhook events. The Courier team will monitor the bottleneck and adjust message batching necessary to flow messages normally.
- identified Sep 06, 2024, 04:39 PM UTC
The team has increased our processing to help with the bottleneck.
- monitoring Sep 06, 2024, 04:44 PM UTC
The team has increased the send pipeline worker and messages are clearing the bottleneck.
- monitoring Sep 06, 2024, 04:52 PM UTC
Corrective actions have cleared the bottleneck and messages and events should be flowing normally.
- resolved Sep 06, 2024, 04:59 PM UTC
Bottleneck has cleared for messages and event statuses.