Courier incident

Courier Send and Message Event Slowdown

Minor Resolved View vendor source →

Courier experienced a minor incident on February 13, 2025 affecting API, lasting 36m. The incident has been resolved; the full update timeline is below.

Started
Feb 13, 2025, 10:32 PM UTC
Resolved
Feb 13, 2025, 11:09 PM UTC
Duration
36m
Detected by Pingoru
Feb 13, 2025, 10:32 PM UTC

Affected components

API

Update timeline

  1. identified Feb 13, 2025, 10:32 PM UTC

    The Courier team has identified a slowdown in sending notifications. This issue has been identified by our team and is closely monitoring. We do not expect any messages to drop. We will continue to monitor and update accordingly.

  2. monitoring Feb 13, 2025, 10:51 PM UTC

    Our team has identified a bottleneck in our system caused by timeouts on our sendworker. Our exponential backoff has caught up and messages are starting to go through.

  3. resolved Feb 13, 2025, 11:09 PM UTC

    The team has identified the issue related to network connections on AWS which resulted in a backup of messages on our pipeline. Messages are flowing through as expected, and the stream has caught up.