Call Tower incident

Call failure from the UK and Frankfurt data centers

Major Resolved View vendor source →

Call Tower experienced a major incident on August 30, 2024 affecting Microsoft Teams - Direct Routing and Global SIP Trunk and 1 more component, lasting 10h 25m. The incident has been resolved; the full update timeline is below.

Started
Aug 30, 2024, 08:23 AM UTC
Resolved
Aug 30, 2024, 06:49 PM UTC
Duration
10h 25m
Detected by Pingoru
Aug 30, 2024, 08:23 AM UTC

Affected components

Microsoft Teams - Direct RoutingGlobal SIP TrunkCisco Webex Calling

Update timeline

  1. investigating Aug 30, 2024, 08:23 AM UTC

    CallTower has opened a master case to investigate this issue as it has been reported by more than one CallTower customer. This issue may or may not directly affect your services and is currently under investigation. If you are affected, please report the issue to [email protected]. CallTower intends to post updates on status.calltower.com approximately every 30 minutes, unless identified otherwise. This issue is being investigated as a top priority.

  2. investigating Aug 30, 2024, 08:55 AM UTC

    Calltower engineers are continuing to actively investigate the call failures in our UK and Frankfurt datacenters. CallTower has opened a master case to investigate this issue as it has been reported by more than one CallTower customer. This issue may or may not directly affect your services and is currently under investigation. If you are affected, please report the issue to [email protected]. CallTower intends to post updates on status.calltower.com approximately every 30 minutes, unless identified otherwise. This issue is being investigated as a top priority.

  3. investigating Aug 30, 2024, 09:32 AM UTC

    The root cause of the failures in our UK and Frankfurt datacenters has not been identified at this time, but Calltower engineers are diligently working to isolate and additional updates will be provided as soon as more information is made available. CallTower has opened a master case to investigate this issue as it has been reported by more than one CallTower customer. This issue may or may not directly affect your services and is currently under investigation. If you are affected, please report the issue to [email protected]. CallTower intends to post updates on status.calltower.com approximately every 30 minutes, unless identified otherwise. This issue is being investigated as a top priority.

  4. investigating Aug 30, 2024, 10:07 AM UTC

    CallTower’s engineering team is actively addressing the root cause of the issue while also exploring potential workarounds. CallTower has opened a master case to investigate this issue as it has been reported by more than one CallTower customer. This issue may or may not directly affect your services and is currently under investigation. If you are affected, please report the issue to [email protected]. CallTower intends to post updates on status.calltower.com approximately every 30 minutes, unless identified otherwise. This issue is being investigated as a top priority.

  5. investigating Aug 30, 2024, 10:39 AM UTC

    CallTower engineers are continuing to pursue a workaround while we isolate the underlying cause of the call failures. We will provide more information as it becomes available. CallTower has opened a master case to investigate this issue as it has been reported by more than one CallTower customer. This issue may or may not directly affect your services and is currently under investigation. If you are affected, please report the issue to [email protected]. CallTower intends to post updates on status.calltower.com approximately every 30 minutes, unless identified otherwise. This issue is being investigated as a top priority.

  6. investigating Aug 30, 2024, 11:26 AM UTC

    Efforts to resolve call failures in the UK and Frankfurt data centers are ongoing. Calltower engineers are looking at any possible workarounds while we work towards a resolution. More information will be provided as soon as it is made available. CallTower has opened a master case to investigate this issue as it has been reported by more than one CallTower customer. This issue may or may not directly affect your services and is currently under investigation. If you are affected, please report the issue to [email protected]. CallTower intends to post updates on status.calltower.com approximately every 30 minutes, unless identified otherwise. This issue is being investigated as a top priority.

  7. monitoring Aug 30, 2024, 11:46 AM UTC

    We are nbow seeing traffic stabilize and getting reports from customers that services are back up in UK and Frankfurt. We are continuing to monitor at this time please retest and if you have any persisting issues, call support at 800-347-5444 and let us know. CallTower has opened a master case for this issue as it is affecting more than one CallTower customer. This issue may or may not directly affect your services. If you are affected, please report the issue to [email protected]. CallTower intends to post updates on status.calltower.com once more information is available, unless identified otherwise.

  8. resolved Aug 30, 2024, 06:49 PM UTC

    We see Traffic has stabilized and have confirmed that inbound and outbound calls are now completing. CallTower will be closing this master case and all associated cases. Within three business days, an Official Root Cause Analysis (RCA) or a Preliminary RCA will be available and provided to customers upon request.