Front experienced a minor incident on November 25, 2024 affecting O365, lasting 18h 11m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Nov 25, 2024, 08:41 AM UTC
Our partner Microsoft is experiencing a service degradation affecting Exchange Online mailboxes. You can follow Microsoft Issue ID MO941162 for more details. This may cause outbound/inbound delays for customers using Office 365, as Front cannot connect to Microsoft servers.
- identified Nov 25, 2024, 10:49 AM UTC
Microsoft is still investigating the issue; here is their latest update. > We’re continuing to investigate recent changes, service telemetry and the failure paths to determine the root cause of impact. Source: https://portal.office.com/Adminportal/Home#/servicehealth/:/alerts/MO941162 (You need to be an O365 admin to be able to access it)
- identified Nov 25, 2024, 03:47 PM UTC
Microsoft is deploying a fix > At 15:27 CET > We’re monitoring the progress of the fix, which has been deployed to approximately 60% of the affected environments. We’re continuing our manual restarts on the remaining impacted machines. We are closely monitoring our systems for signs of recovery; however, we have not observed any significant progress at this time.
- monitoring Nov 25, 2024, 08:26 PM UTC
Microsoft deployed a fix. Our systems are recovering. Latest update from Microsoft: > At 20:52 CET > We're continuing to redirect traffic and apply targeted remediation to impacted servers. > We've observed initially positive results in some areas, and we're monitoring service health closely as our mitigation workstreams continue in parallel.
- monitoring Nov 26, 2024, 12:07 AM UTC
Our systems have fully recovered. We are now retrying failed tasks.
- resolved Nov 26, 2024, 02:52 AM UTC
This incident has been resolved.