TelNet Worldwide Outage History

Started Mar 26, 2026, 12:33 AM UTC· resolved Mar 26, 2026, 04:56 PM UTC

1. investigating Mar 26, 2026, 12:33 AM UTC

   We are aware of and investigating an issue with voice services over the Detroit Network.

2. investigating Mar 26, 2026, 01:41 AM UTC

   We are continuing our investigation and have remote hands on site for assistance. We will provide another update within 60 minutes, or as one becomes available.

3. identified Mar 26, 2026, 02:18 AM UTC

   We have identified the issue and working towards restoration. We will update again when we have more information.

4. monitoring Mar 26, 2026, 03:40 AM UTC

   Our network team (along with on-site support resources) has resolved the routing issue affecting Detroit Voice services. Traffic began successfully routing at approximately 10:15 PM, and all services are currently operating normally. We are continuing to monitor stability and review the underlying cause as part of our follow-up analysis. We’ll share additional updates and a Post Mortem / RFI as more information becomes available.

5. monitoring Mar 26, 2026, 03:46 AM UTC

   We are continuing to monitor for any further issues.

6. resolved Mar 26, 2026, 04:56 PM UTC

   All services impacted by the earlier Detroit network disruption have been fully restored and remain stable. Our Network Operations team identified the issue within core infrastructure and implemented recovery actions to restore normal routing behavior. Service restoration began earlier in the evening and was confirmed fully complete at approximately 23:13 ET. We will continue to monitor the environment to ensure ongoing stability. A formal post-incident summary (RFO) will be shared once our internal review is complete. Thank you for your patience while our teams worked to resolve this issue. If you continue to experience any service concerns, please contact our Support Team at [email protected], [email protected], or [email protected].

7. postmortem Mar 26, 2026, 06:05 PM UTC

   This communication is a follow-up to the recent service disruption experienced on Wednesday evening \(3/25/2026\), involving Detroit \(DET\) voice and data services. The event was tracked via Ticket # SR45858. **Event Summary** On the evening of 3/25/2026, our Network Operations team identified a disruption affecting voice and data services within the Detroit network. Our team worked with on-site resources to restore routing stability. Service began recovering at approximately 22:15 ET and has remained stable since restoration. **Event Window** **Start of Impact:** 19:49 ET **Service Restoration Begins:** 22:15 ET **Service Restoration Complete:** 23:13 ET **Incident Impact** During the event window, customers may have experienced intermittent voice and/or data service interruptions related to a core network infrastructure issue. **Root Cause** The disruption was attributed to a failure within the core switching infrastructure in the Detroit network. Redundant network paths were available to support service continuity during the equipment failure; however, traffic did not transition to the alternate path as designed. Service was restored following the re-establishment of routing adjacencies. **Next Steps and Preventative Measures** To prevent recurrence, our team is taking the following actions: * Performing additional validation checks across the environment to confirm that the expected redundancy behavior is observed correctly, enhancing resilience against similar events. We sincerely apologize for the disruption and appreciate your continued trust as we strengthen our systems and procedures to better serve you. If you have any further questions, please contact our Support Team at 800.508.1254 or email [[email protected]](mailto:[email protected]). Sincerely, The TelNet Customer Operations Team

Read the full incident report →

Started Mar 07, 2026, 03:30 AM UTC· resolved Mar 07, 2026, 04:30 PM UTC

1. resolved Mar 10, 2026, 07:14 PM UTC

   Toll-Free Call Failures Due to Unrecognized Carrier Source IPs Start Time: March 6, 2026 - 10:16 PM ET End Time: March 7, 2026 - 11:27 AM ET Duration: ~13 hours Issue Summary: Beginning late Friday evening (3/6/26), inbound toll-free calls from one of our upstream carriers began arriving from previously unknown source IP addresses. These IPs were not included in our existing signaling allowlist, which resulted in affected calls failing to complete. Impact: A subset of inbound toll-free calls routed through this carrier failed to reach our network during the incident window. The issue primarily affected calls originating through the carrier paths using the newly introduced IP addresses. Customer reports began coming in mid-morning Saturday (3/7/26), which is when the pattern was identified and correlated. Root Cause: The upstream toll-free carrier began routing signaling traffic from additional IP addresses within their network without prior notification. Because these addresses were not included in our signaling allowlist, the traffic was rejected. Resolution: Once identified, our team escalated the issue with the carrier and updated our firewall policy to allow the carrier’s full network range associated with their ASN. This ensures future routing changes within their network will not result in similar blocking. Service was confirmed restored at 11:27 AM ET on March 7, 2026. Preventive Actions: • Expanded allowlisting to include the carrier’s ASN network space rather than individual IPs. • Opened communication with the carrier to ensure notification of future routing changes. • Internal monitoring updated to more quickly flag unexpected source IP changes for upstream carriers.

2. postmortem Mar 10, 2026, 07:14 PM UTC

   **Toll-Free Call Failures Due to Unrecognized Carrier Source IPs** Start Time: March 6, 2026 - 10:16 PM ET End Time: March 7, 2026 - 11:27 AM ET Duration: ~13 hours ‌ **Issue Summary:** Beginning late Friday evening \(3/6/26\), inbound toll-free calls from one of our upstream carriers began arriving from previously unknown source IP addresses. These IPs were not included in our existing signaling allowlist, which resulted in affected calls failing to complete. ‌ **Impact:** A subset of inbound toll-free calls routed through this carrier failed to reach our network during the incident window. The issue primarily affected calls originating through the carrier paths using the newly introduced IP addresses. Customer reports began coming in mid-morning Saturday \(3/7/26\), which is when the pattern was identified and correlated. ‌ **Root Cause:** The upstream toll-free carrier began routing signaling traffic from additional IP addresses within their network without prior notification. Because these addresses were not included in our signaling allowlist, the traffic was rejected. ‌ **Resolution:** Once identified, our team escalated the issue with the carrier and updated our firewall policy to allow the carrier’s full network range associated with their ASN. This ensures future routing changes within their network will not result in similar blocking. **Service was confirmed restored at 11:27 AM ET on March 7, 2026.** ‌ **Preventive Actions:** • Expanded allowlisting to include the carrier’s ASN network space rather than individual IPs. • Opened communication with the carrier to ensure notification of future routing changes. • Internal monitoring updated to more quickly flag unexpected source IP changes for upstream carriers.

Read the full incident report →

Started Feb 05, 2026, 06:56 PM UTC· resolved Feb 10, 2026, 10:09 PM UTC

1. investigating Feb 05, 2026, 06:56 PM UTC

   We’re aware of an increase in reports of audio quality issues, including choppy or intermittent audio from the Detroit (Legacy TelNet Worldwide) Network. Our voice networking team is actively investigating and working to isolate the cause. If you’re experiencing issues or would like to provide an update on an existing ticket, please call or text us at 800.508.1254, or email [email protected]. Our team is here to help. We’ll continue to share updates here as they become available. Thank you for your patience.

2. investigating Feb 05, 2026, 07:14 PM UTC

   We are continuing to investigate this issue.

3. investigating Feb 05, 2026, 07:53 PM UTC

   We are continuing to investigate the underlying cause of the issue. If you’re experiencing issues or would like to provide an update on an existing ticket, please call or text us at 800.508.1254, or email [email protected]. Our team is here to help. We’ll continue to share updates here as they become available. Thank you for your patience.

4. identified Feb 05, 2026, 08:26 PM UTC

   We will be performing emergency maintenance tonight, 2/5/26 at 9pm Eastern. Please be aware that with this maintenance, established calls may be disconnected during the switch. If you’re experiencing issues or would like to provide an update on an existing ticket, please call or text us at 800.508.1254, or email [email protected]. Our team is here to help. We’ll continue to share updates here as they become available. Thank you for your patience.

5. investigating Feb 06, 2026, 01:30 PM UTC

   We are continuing to investigate the issue. If you’re experiencing issues or would like to provide an update on an existing ticket, please call or text us at 800.508.1254, or email [email protected]. Our team is here to help. We’ll continue to share updates here as they become available. Thank you for your patience.

6. monitoring Feb 06, 2026, 05:08 PM UTC

   With the completion of the maintenance last night, we have continued to investigate and monitor the trouble. We are no longer seeing the trouble of duplicate packets. We will continue to monitor to ensure network stability. If you are continuing to experience trouble today or would like to provide an update on an existing ticket, please call or text us at 800.508.1254, or email [email protected]. Our team is here to help. We’ll continue to share updates here as they become available. Thank you for your patience.

7. resolved Feb 10, 2026, 10:09 PM UTC

   This incident has been resolved. An RFO will be made available after review with our Network Teams.

Read the full incident report →

Started Jan 05, 2026, 08:20 PM UTC· resolved Jan 07, 2026, 08:42 PM UTC

1. investigating Jan 05, 2026, 08:20 PM UTC

   We are currently investigating an issue impacting access to our Bill Center. (billing.ipfone.com, customerportal.telnetww.com) At this time, customers may be unable to view invoices, make payments, or access billing-related information. Our teams are actively working to identify the root cause and restore service as quickly as possible. We will provide updates as more information becomes available. We apologize for the inconvenience and appreciate your patience. Next Update: Within 30-60 minutes, or sooner if available

2. identified Jan 05, 2026, 08:42 PM UTC

   We have identified the issue impacting access to Bill Center and are currently working with our vendor toward resolution. At this time, access appears to be restored for users. We will continue to monitor and provide updates as more information becomes available. Next Update: As we receive confirmation from our vendor (30-60 minutes)

3. monitoring Jan 05, 2026, 08:55 PM UTC

   Our Vendor has advised that a fix has been implemented, and the system is currently stable. Our teams will continue to monitor closely and will provide a Root Cause when complete.

4. resolved Jan 07, 2026, 08:42 PM UTC

   The trouble impacting Bill Center and Customer Portal access was caused by a third-party vendor issue. A fix has been implemented and services are stable. We are awaiting additional details from the vendor and will provide an updated RFO when available.

5. postmortem Jan 07, 2026, 08:42 PM UTC

   On _**Monday, January 5th, 2026**_, some customers experienced difficulty accessing our **Bill Center** and **Customer Portal** due to an issue with one of our third-party service providers. Once the issue was identified, the vendor implemented a fix and system access was restored. At this time, services are stable. There was **no impact to customer voice or networking services**, and no data loss has been identified. We are currently awaiting a detailed root cause analysis from the vendor. An updated RFO will be shared upon request once that information is received. We appreciate your patience and apologize for any inconvenience this may have caused.

Read the full incident report →