IPFone Outage History

This communication is a follow-up to the recent service disruption experienced on Wednesday evening \(3/25/2026\), involving Detroit \(DET\) voice and data services. The event was tracked via Ticket # SR45858. **Event Summary** On the evening of 3/25/2026, our Network Operations team identified a disruption affecting voice and data services within the Detroit network. Our team worked with on-site resources to restore routing stability. Service began recovering at approximately 22:15 ET and has remained stable since restoration. **Event Window** **Start of Impact:** 19:49 ET **Service Restoration Begins:** 22:15 ET **Service Restoration Complete:** 23:13 ET **Incident Impact** During the event window, customers may have experienced intermittent voice and/or data service interruptions related to a core network infrastructure issue. **Root Cause** The disruption was attributed to a failure within the core switching infrastructure in the Detroit network. Redundant network paths were available to support service continuity during the equipment failure; however, traffic did not transition to the alternate path as designed. Service was restored following the re-establishment of routing adjacencies. **Next Steps and Preventative Measures** To prevent recurrence, our team is taking the following actions: * Performing additional validation checks across the environment to confirm that the expected redundancy behavior is observed correctly, enhancing resilience against similar events. We sincerely apologize for the disruption and appreciate your continued trust as we strengthen our systems and procedures to better serve you. If you have any further questions, please contact our Support Team at 800.508.1254 or email [[email protected]](mailto:[email protected]). Sincerely, The TelNet Customer Operations Team

**Toll-Free Call Failures Due to Unrecognized Carrier Source IPs** Start Time: March 6, 2026 - 10:16 PM ET End Time: March 7, 2026 - 11:27 AM ET Duration: ~13 hours ‌ **Issue Summary:** Beginning late Friday evening \(3/6/26\), inbound toll-free calls from one of our upstream carriers began arriving from previously unknown source IP addresses. These IPs were not included in our existing signaling allowlist, which resulted in affected calls failing to complete. ‌ **Impact:** A subset of inbound toll-free calls routed through this carrier failed to reach our network during the incident window. The issue primarily affected calls originating through the carrier paths using the newly introduced IP addresses. Customer reports began coming in mid-morning Saturday \(3/7/26\), which is when the pattern was identified and correlated. ‌ **Root Cause:** The upstream toll-free carrier began routing signaling traffic from additional IP addresses within their network without prior notification. Because these addresses were not included in our signaling allowlist, the traffic was rejected. ‌ **Resolution:** Once identified, our team escalated the issue with the carrier and updated our firewall policy to allow the carrier’s full network range associated with their ASN. This ensures future routing changes within their network will not result in similar blocking. **Service was confirmed restored at 11:27 AM ET on March 7, 2026.** ‌ **Preventive Actions:** • Expanded allowlisting to include the carrier’s ASN network space rather than individual IPs. • Opened communication with the carrier to ensure notification of future routing changes. • Internal monitoring updated to more quickly flag unexpected source IP changes for upstream carriers.