Toll-Free Call Failures Due to Unrecognized Carrier Source IPs

Update timeline

1. resolved Mar 10, 2026, 07:14 PM UTC

   Toll-Free Call Failures Due to Unrecognized Carrier Source IPs Start Time: March 6, 2026 - 10:16 PM ET End Time: March 7, 2026 - 11:27 AM ET Duration: ~13 hours Issue Summary: Beginning late Friday evening (3/6/26), inbound toll-free calls from one of our upstream carriers began arriving from previously unknown source IP addresses. These IPs were not included in our existing signaling allowlist, which resulted in affected calls failing to complete. Impact: A subset of inbound toll-free calls routed through this carrier failed to reach our network during the incident window. The issue primarily affected calls originating through the carrier paths using the newly introduced IP addresses. Customer reports began coming in mid-morning Saturday (3/7/26), which is when the pattern was identified and correlated. Root Cause: The upstream toll-free carrier began routing signaling traffic from additional IP addresses within their network without prior notification. Because these addresses were not included in our signaling allowlist, the traffic was rejected. Resolution: Once identified, our team escalated the issue with the carrier and updated our firewall policy to allow the carrier’s full network range associated with their ASN. This ensures future routing changes within their network will not result in similar blocking. Service was confirmed restored at 11:27 AM ET on March 7, 2026. Preventive Actions: • Expanded allowlisting to include the carrier’s ASN network space rather than individual IPs. • Opened communication with the carrier to ensure notification of future routing changes. • Internal monitoring updated to more quickly flag unexpected source IP changes for upstream carriers.

2. postmortem Mar 10, 2026, 07:14 PM UTC

   **Toll-Free Call Failures Due to Unrecognized Carrier Source IPs** Start Time: March 6, 2026 - 10:16 PM ET End Time: March 7, 2026 - 11:27 AM ET Duration: ~13 hours ‌ **Issue Summary:** Beginning late Friday evening \(3/6/26\), inbound toll-free calls from one of our upstream carriers began arriving from previously unknown source IP addresses. These IPs were not included in our existing signaling allowlist, which resulted in affected calls failing to complete. ‌ **Impact:** A subset of inbound toll-free calls routed through this carrier failed to reach our network during the incident window. The issue primarily affected calls originating through the carrier paths using the newly introduced IP addresses. Customer reports began coming in mid-morning Saturday \(3/7/26\), which is when the pattern was identified and correlated. ‌ **Root Cause:** The upstream toll-free carrier began routing signaling traffic from additional IP addresses within their network without prior notification. Because these addresses were not included in our signaling allowlist, the traffic was rejected. ‌ **Resolution:** Once identified, our team escalated the issue with the carrier and updated our firewall policy to allow the carrier’s full network range associated with their ASN. This ensures future routing changes within their network will not result in similar blocking. **Service was confirmed restored at 11:27 AM ET on March 7, 2026.** ‌ **Preventive Actions:** • Expanded allowlisting to include the carrier’s ASN network space rather than individual IPs. • Opened communication with the carrier to ensure notification of future routing changes. • Internal monitoring updated to more quickly flag unexpected source IP changes for upstream carriers.