Ember IT incident

Microsoft 365 Outage

Major Resolved View vendor source →

Ember IT experienced a major incident on March 15, 2021 affecting Microsoft 365 Services, lasting 3h. The incident has been resolved; the full update timeline is below.

Started
Mar 15, 2021, 07:55 PM UTC
Resolved
Mar 15, 2021, 10:55 PM UTC
Duration
3h
Detected by Pingoru
Mar 15, 2021, 07:55 PM UTC

Affected components

Microsoft 365 Services

Update timeline

  1. investigating Mar 15, 2021, 07:55 PM UTC

    Microsoft is experiencing issues with Office 365, Microsoft Teams, and some Azure services. This may prevent you from being able to log in. We will provide additional information as it's available. Thank you. Matt @ EMBER

  2. identified Mar 15, 2021, 08:37 PM UTC

    Update from Microsoft: SUMMARY OF IMPACT: Starting at approximately 19:15 UTC on 15 Mar 2021, a subset of customers may experience issues authenticating into Microsoft services, including Microsoft Teams, Office and/or Dynamics, Xbox Live, and the Azure Portal. CURRENT STATUS: Engineering teams have been engaged and are currently investigating. The next update will be provided in 60 minutes or as events warrant.

  3. resolved Mar 15, 2021, 10:55 PM UTC

    Closing this case. Microsoft is currently deploying fix. Update from Microsoft. Critical Authentication errors across multiple Microsoft services - Validating Mitigation SUMMARY OF IMPACT: Starting at approximately 19:15 UTC on 15 Mar 2021, a subset of customers may experience issues authenticating into Microsoft services, including Microsoft Teams, Office and/or Dynamics, Xbox Live, and the Azure Portal. Other Azure services/offerings that rely on authentication infrastructure downstream may see further impact. We recommend that customers with custom applications that use Azure AD refresh their metadata cache. CURRENT STATUS: Engineering teams have rolled out a fix to all affected regions for Azure AD. Internal telemetry and customer reports suggests that the error rate for dependent services is rapidly decreasing. Microsoft services are in the process of recovery. Some services and offerings may have varying times of recovery following the underlying fix. The next update will be provided in 60 minutes or as events warrant. This message was last updated at 22:52 UTC on 15 March 2021