Microsoft Inbound Call Failures

Update timeline

1. investigating Mar 03, 2025, 04:58 PM UTC

   CallTower support is investigating multiple reports of Microsoft inbound calls not completing. CallTower has opened a master case to investigate this issue as it has been reported by more than one CallTower customer. This issue may or may not directly affect your services and is currently under investigation. If you are affected, please report the issue to [email protected]. CallTower intends to post updates on status.calltower.com approximately every 30 minutes, unless identified otherwise. This issue is being investigated as a top priority.

2. identified Mar 03, 2025, 05:24 PM UTC

   CallTower continues to work with Microsoft on the issue affecting the completion of inbound calls. Microsoft has acknowledged an issue affecting their services. Microsoft’s latest update is included below. Issue ID: TM1022107 Affected services: Microsoft Teams Status: Investigating Issue type: Incident Start time: Mar 3, 2025 at 9:46 AM MST User impact Users may not be able to receive calls placed through Microsoft Teams provisioned auto attendants and call queues. Scope of impact Users expecting to make or receive calls routed through Microsoft Teams-provisioned auto attendants and call queues may be affected. Current status Mar 3, 2025 at 10:11 AM MST Our monitoring has identified an increase in call failures for Microsoft Teams auto attendants and call queues. We're analyzing service telemetry and call metadata to better understand the nature of impact and determine our next steps. Next update by: Monday, March 3, 2025 at 11:30 AM MST *CallTower has opened a master case to investigate this issue as it has been reported by more than one CallTower customer. This issue may or may not directly affect your services and is currently under investigation. If you are affected, please report the issue to [email protected]. This issue is being investigated as a top priority.

3. identified Mar 03, 2025, 06:30 PM UTC

   CallTower continues to work with Microsoft on the issue affecting the completion of inbound calls for auto attendants and call queues. Microsoft has provided an update on the issue. Microsoft’s update is included below. Issue ID: TM1022107 Affected services: Microsoft Teams Status: Service degradation Issue type: Incident Start time: Mar 3, 2025 at 9:46 AM MST User impact Users may not be able to receive calls placed through Microsoft Teams provisioned auto attendants and call queues. Scope of impact Users expecting to make or receive calls routed through Microsoft Teams-provisioned auto attendants and call queues may be affected. Current status Mar 3, 2025 at 11:01 AM MST We've detected a potential authentication token issue that may be contributing to impact, and we're further examining this pathway to determine our next troubleshooting steps. Next update by: Monday, March 3, 2025 at 12:30 PM MST *CallTower has opened a master case to investigate this issue as it has been reported by more than one CallTower customer. This issue may or may not directly affect your services and is currently under investigation. If you are affected, please report the issue to [email protected]. This issue is being investigated as a top priority.

4. identified Mar 03, 2025, 07:43 PM UTC

   Microsoft continues to work to resolve the issue affecting inbound calls to auto attendants and call queues for Microsoft Teams. Microsoft has provided an update which is included below. Issue ID: TM1022107 Affected services: Microsoft Teams Status: Service degradation Issue type: Incident Start time: Mar 3, 2025 at 9:46 AM MST User impact Users may not be able to receive calls placed through Microsoft Teams provisioned auto attendants and call queues. Scope of impact Users expecting to make or receive calls routed through Microsoft Teams-provisioned auto attendants and call queues may be affected. Preliminary root case A modification made to an auto attendant application ID record is rejecting Microsoft Teams call queues. Current status Mar 3, 2025 at 12:33 PM MST We've discovered that a modification to an auto attendant application ID record is rejecting Microsoft Teams call queues. We've deployed a fix to update the affected application ID record, which will mitigate impact. Next update by: Monday, March 3, 2025 at 1:30 PM MST *CallTower has opened a master case to investigate this issue as it has been reported by more than one CallTower customer. This issue may or may not directly affect your services and is currently under investigation. If you are affected, please report the issue to [email protected]. This issue is being investigated as a top priority.

5. monitoring Mar 03, 2025, 08:46 PM UTC

   CallTower has continued to work with Microsoft on the issue affecting inbound calling to auto attendants and call queues. Microsoft has applied a fix at this time. CallTower’s testing shows that calls are now completing as expected. Microsoft's update is provided below. Issue ID: TM1022107 Affected services: Microsoft Teams Status: Service degradation Issue type: Incident Start time: Mar 3, 2025 at 9:46 AM MST User impact Users may not be able to receive calls placed through Microsoft Teams provisioned auto attendants and call queues. More info Callers routed through call queues may hear a busy signal or the call may ring for 45+ seconds and then end. Scope of impact Users expecting to make or receive calls routed through Microsoft Teams-provisioned auto attendants and call queues may be affected. Preliminary root case A configuration change to an auto-attendant app ID record that matches inbound calls with users in call queues is preventing users from receiving calls and causing the impact. Current status Mar 3, 2025 at 1:20 PM MST We’ve deployed the configuration to the auto-attendant app and are seeing service functionality improvements as expected. We’re continuing to monitor to confirm the fix is mitigating impact. Next update by: Monday, March 3, 2025 at 2:30 PM MST *CallTower has opened a master case for this issue as it is affecting more than one CallTower customer. This issue may or may not directly affect your services. If you are affected, please report the issue to [email protected].

6. resolved Mar 03, 2025, 10:29 PM UTC

   Microsoft has confirmed that the fix has been deployed and the issue has been resolved. Microsoft’s final update is provided below. Issue ID: TM1022107 Affected services: Microsoft Teams Status: Service restored Issue type: Incident Start time: Mar 3, 2025, 9:46 AM MST End time: Mar 3, 2025, 1:19 PM MST User impact Users may not have been able to receive calls placed through Microsoft Teams provisioned auto attendants and call queues More info Callers routed through call queues may have heard a busy signal or the call may have rung for 45+ seconds and then ended. Scope of impact Users who expected to make or receive calls routed through Microsoft Teams provisioned auto attendants and call queues may have been affected. Preliminary root case A configuration change to an auto-attendant app ID record that matches inbound calls with users in call queues was preventing users from receiving calls and caused the impact. Next steps - We’re reviewing our testing and validation processes to understand how the impact of the configuration change wasn’t identified prior to implementation so that we can improve our testing and validation procedures and avoid similar issues in the future. We'll provide a preliminary Post-Incident Report within two business days and a final Post-Incident Report within five business days. Current status Mar 3, 2025, 1:50 PM MST Our fix was successfully deployed. After an extended period of monitoring, we’ve confirmed through telemetry that service is restored and impact has been fully remediated. *CallTower will be closing this master case and all associated cases.