True IP Solutions incident
Intermittent Inbound Call Degradation
True IP Solutions experienced a major incident on September 28, 2021 affecting Upstream Carriers, lasting 2d 23h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Sep 27, 2021, 05:31 PM UTC
One of our upstream providers has identified a DDOS attack on their services which is causing intermittent problems with inbound calls, DTMF, and SMS delivery. The problem is being actively worked and updates will follow as we receive them.
- identified Sep 27, 2021, 08:02 PM UTC
Upstream affected carriers are continuing to investigate this issue.
- monitoring Sep 27, 2021, 11:50 PM UTC
The upstream carrier has reported all issues resolved at this time. We are not currently seeing inbound call issues, DTMF, or problems with SMS messages. We will continue to monitor the situation closely and post updates if there are any new developments. If you continue to experience problems with any voice service, please contact our support team.
- monitoring Sep 28, 2021, 02:18 PM UTC
The upstream carrier is currently investigating intermittent and residual service disruptions due to yesterday's DDoS attack. We are continuing to monitor and will provide updates as they arise.
- monitoring Sep 29, 2021, 10:30 AM UTC
Our upstream carrier has reported stability within their network at this time. We will continue to monitor the situation closely and post updates if there are any new developments. If you continue to experience problems with any voice service, please contact our support team.
- monitoring Sep 29, 2021, 01:59 PM UTC
Our upstream carrier has reported renewed issues consistent with the DDoS attacks against their network in recent days. We will continue monitoring this incident and searching for ways to mitigate the issue. We will provide updates as they are available and apologize once again for the problems.
- monitoring Sep 30, 2021, 12:18 AM UTC
At this time we are continuing to experience intermittent service disruptions on inbound calls due to the ongoing DDOS attacks against our upstream carrier. For more information regarding this outage please see our post at https://trueipsolutions.com/ddos-attacks-causing-telephone-outages/ and for status updates directly from the upstream carrier suffering these attacks please visit https://status.bandwidth.com/. We will continue to keep you updated as more information is made available.
- resolved Oct 01, 2021, 01:52 PM UTC
We have not observed any inbound calling difficulties related to the DDoS attacks on Bandwidth.com for approximately 36 hours. At this time we are considering the issue resolved, however we will continue to monitor closely and if we see any renewed problems we will update via our status page at https://status.trueipsolutions.com. We are exploring options with our various upstream carriers to prevent this sort of outage from having such a large impact in the future.