IXCloud incident
Microsoft Network Infrastructure - Issues accessing a subset of Microsoft services
IXCloud experienced a notice incident on July 30, 2024 affecting United States East and United States and 1 more component, lasting 9h 44m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jul 30, 2024, 02:00 PM UTC
Starting at approximately 12:00 UTC on 30 July 2024, a subset of customers may experience issues connecting to Microsoft services globally. Current status: MS has multiple engineering teams engaged to diagnose and resolve the issue as soon as possible. MS identified multiple workstreams and is working to mitigate impacted workstreams by performing failover operations. More details will be provided as they become available.
- investigating Jul 30, 2024, 03:02 PM UTC
MS has implemented networking configuration changes and have performed failovers to alternate networking paths to provide relief. Monitoring telemetry shows improvement in service availability from approximately 14:10 UTC onwards, and we are continuing to monitor to ensure full recovery.
- investigating Jul 30, 2024, 05:09 PM UTC
An unexpected usage spike resulted in Azure Front Door (AFD) components performing below acceptable thresholds, leading to intermittent errors, timeout, and latency spikes. MS has implemented network configuration changes and has performed failovers to provide alternate network paths for relief. Their monitoring telemetry shows improvement in service availability from approximately 14:10 UTC onwards, but they are investigating reports of specific services and regions that are still experiencing intermittent errors. They will provide an update on our continued mitigation efforts by 18:00 UTC, or sooner if we have progress to share.
- resolved Jul 30, 2024, 11:45 PM UTC
This incident has been resolved.