Telesmart incident
Unplanned Microsoft Teams Outage – 21 July 2022
Telesmart experienced a major incident on July 21, 2022 affecting Cloud Calling for Microsoft Teams and Tendfor Contact Centre and 1 more component, lasting 21h 45m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jul 21, 2022, 02:02 AM UTC
Telesmart would like to inform you that we have been made aware that Microsoft are currently experiencing an issue with multiple services.
- investigating Jul 21, 2022, 02:06 AM UTC
We are continuing to investigate this issue.
- investigating Jul 21, 2022, 02:13 AM UTC
More information from Microsoft is below: User Impact: Users may be unable to access Microsoft Teams. More info: Reported impact scenarios include, but may not be limited to, access to the service, messaging, calling, and meetings. The desktop app may sign in successfully, though all functionalities may be down and users may see a “We ran into a problem. Reconnecting…” message. Users leveraging the web client may see a message stating “Operation failed with unexpected error.”. Current status: We've received reports that users are experiencing issues with the Microsoft Teams service. We're analyzing service telemetry to help identify the source of the issue and determine our next steps to restoring the service.
- investigating Jul 21, 2022, 02:43 AM UTC
There have been no further updates from Microsoft, we are continuing to monitor the communication channels from Microsoft.
- investigating Jul 21, 2022, 03:12 AM UTC
Microsoft have determined that a recent deployment contained a broken connection to an internal storage service, which has resulted in the degrading of MS Teams services. Microsoft are working to direct traffic to a healthy service to mitigate the impact.
- identified Jul 21, 2022, 03:57 AM UTC
Microsoft are looking at rerouting Microsoft Teams traffic to an alternate region in an effort to provide some relief and potentially restore some functionality to the service. Microsoft also reported they nave identified a downstream impact to multiple Microsoft 365 services with Teams integration, such as Microsoft Word, Office Online and SharePoint Online. Microsoft are providing updates for those services via MO402741 in the admin center or through http://status.office.com
- identified Jul 21, 2022, 04:38 AM UTC
Current status from Microsoft: We’re observing continued improvement for a majority of the functions within the Microsoft 365 apps. We’re focused on the areas that have not recovered and are taking action to restore service at the highest priority.
- monitoring Jul 21, 2022, 08:17 PM UTC
Microsoft has restored the services that were affected, we are monitoring the situation.
- resolved Jul 21, 2022, 11:47 PM UTC
Monitoring has shown all services are running normally, we are closing this incident.