Network Connectivity Issue to Azure and Microsoft services

Update timeline

1. investigating May 02, 2019, 09:23 PM UTC

   Customers may experience intermittent connectivity issues with Azure and Microsoft services. Microsoft Engineers are investigating DNS resolution issues affecting network connectivity. Connectivity issues may affect the availability of Compute, Storage, and Database services, and some customers may be unable to file support requests. More information will be provided as it becomes available.

2. investigating May 02, 2019, 10:04 PM UTC

   Update: Starting at 19:43 UTC on 02 May 2019, customers may experience intermittent connectivity issues with Azure and other Microsoft services (including M365, Dynamics, DevOps, etc). Engineers have identified the underlying root cause as DNS resolution issues affecting network connectivity with downstream impact to Compute, Storage, AAD, and Database services. Mitigation steps are being applied, and customers should start to see signs of recovery.

3. investigating May 02, 2019, 10:05 PM UTC

   We are continuing to investigate this issue.

4. investigating May 02, 2019, 11:02 PM UTC

   Starting at 19:43 UTC on 02 May 2019, customers may experience intermittent connectivity issues with Azure and other Microsoft services (including M365, Dynamics, DevOps, etc). Engineers have identified the underlying root cause as an incorrect name server delegation issue affecting DNS resolution, network connectivity, and downstream impact to Compute, Storage, App Service, AAD, and SQL Database resources. Mitigation has been applied, and most services have recovered, with the exception of a small subset of services who may still experience some impact. This message was last updated at 22:55 UTC on 02 May 2019

5. investigating May 02, 2019, 11:15 PM UTC

   Starting at 19:43 UTC on 02 May 2019, customers may experience intermittent connectivity issues with Azure and other Microsoft services (including M365, Dynamics, DevOps, etc). Engineers have identified the underlying root cause as an incorrect name server delegation issue affecting DNS resolution, network connectivity, and downstream impact to Compute, Storage, App Service, AAD, and SQL Database resources. Mitigation has been applied, and the majority of Azure and other Microsoft services have recovered. We are in the process of final validation to ensure full recovery. This message was last updated at 23:08 UTC on 02 May 2019

6. investigating May 03, 2019, 12:02 AM UTC

   Network Connectivity - DNS Resolution Summary of impact: Between 19:43 and 22:35 UTC on 02 May 2019, customers may have experienced intermittent connectivity issues with Azure and other Microsoft services (including M365, Dynamics, DevOps, etc). Most services were recovered by 21:30 UTC with the remaining recovered by 22:35 UTC. Preliminary root cause: Engineers identified the underlying root cause as a nameserver delegation change affecting DNS resolution and resulting in downstream impact to Compute, Storage, App Service, AAD, and SQL Database services. During the migration of a legacy DNS system to Azure DNS, some domains for Microsoft services were incorrectly updated. No customer DNS records were impacted during this incident, and the availability of Azure DNS remained at 100% throughout the incident. The problem impacted only records for Microsoft services. Mitigation: To mitigate, engineers corrected the nameserver delegation issue. Applications and services that accessed the incorrectly configured domains may have cached the incorrect information, leading to a longer restoration time until their cached information expired.

7. resolved May 03, 2019, 12:02 AM UTC

   This incident has been resolved.