ClearlyIP incident

Calling issues with US East trunks

Resolved after 1 hour 21 minutes, affecting US East, on November 3, 2025.

postmortem Nov 04, 2025, 05:13 PM UTC

At approximately 11:27AM ET on November 3, ClearlyIP's NOC team began to receive system alerts indicating excess session activity within its US-East trunking systems. The team's investigation identified unauthenticated INVITE messages being received from unknown IPs as the source of the excess traffic, as well as a customer generating unexpected levels of outbound calls. Steps were taken to mitigate the traffic's impact, including restarting affected services to clear sessions and restore normal status. By approximately 11:55AM ET, the US-East situation was stabilized and session traffic had returned to normal levels. The team has identified a number of improvements and traffic management strategies to help avoid similar incidents which will be implemented in future maintenance events.

Started
Nov 03, 2025, 04:49 PM UTC
Resolved
Nov 03, 2025, 06:10 PM UTC
Duration
1h 21m
Detected by Pingoru
Nov 03, 2025, 04:49 PM UTC

Affected components

US East

Update timeline

  1. investigating Nov 03, 2025, 04:49 PM UTC

    We are currently investigating this issue.

  2. identified Nov 03, 2025, 04:57 PM UTC

    The issue has been identified and a fix is being implemented.

  3. monitoring Nov 03, 2025, 05:29 PM UTC

    A fix has been implemented and we are monitoring the results.

  4. resolved Nov 03, 2025, 06:10 PM UTC

    This incident has been resolved.

  5. postmortem Nov 04, 2025, 05:13 PM UTC

    At approximately 11:27AM ET on November 3, ClearlyIP's NOC team began to receive system alerts indicating excess session activity within its US-East trunking systems. The team's investigation identified unauthenticated INVITE messages being received from unknown IPs as the source of the excess traffic, as well as a customer generating unexpected levels of outbound calls. Steps were taken to mitigate the traffic's impact, including restarting affected services to clear sessions and restore normal status. By approximately 11:55AM ET, the US-East situation was stabilized and session traffic had returned to normal levels. The team has identified a number of improvements and traffic management strategies to help avoid similar incidents which will be implemented in future maintenance events.

Your product runs on more than ClearlyIP

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →