CodeTwo incident

[Europe] Microsoft Exchange Online Protection issues

Minor Resolved View vendor source →

CodeTwo experienced a minor incident on May 10, 2021 affecting Mail flow and Mail flow and 1 more component, lasting 1h 8m. The incident has been resolved; the full update timeline is below.

Started
May 10, 2021, 08:42 AM UTC
Resolved
May 10, 2021, 09:51 AM UTC
Duration
1h 8m
Detected by Pingoru
May 10, 2021, 08:42 AM UTC

Affected components

Mail flowMail flowMail flowMail flow

Update timeline

  1. identified May 10, 2021, 08:42 AM UTC

    We are currently monitoring an issue caused by a degraded performance of Microsoft Exchange Online Protection (EOP) which may lead to delays in email delivery for customers in Europe. This is a Microsoft 365 issue - not a CodeTwo issue. Do not take any action - emails with signatures should be sent to final recipients once this issue is fixed by Microsoft. The next update will be provided in 30 minutes or as events warrant.

  2. identified May 10, 2021, 09:13 AM UTC

    We are continuing to monitor this issue. The impact seems to be low and only a subset of customers is affected. The next update will be provided in 30 minutes or as events warrant.

  3. resolved May 10, 2021, 09:51 AM UTC

    This incident has been resolved.

  4. postmortem May 11, 2021, 01:51 PM UTC

    Microsoft provided RCA regarding the issue under EX255429 in Microsoft 365 admin center. Here’s the summary: _**Title:** Some users may have been unable to send and receive external email messages within the Exchange Online service User_ _**Impact:** Users may have been unable to send and receive external email messages within the Exchange Online service._ _**Final status:** We've confirmed that cache has finished updating and the affected infrastructure has returned to service. After further monitoring, we've confirmed that this has resolved the issue._ _**Scope of impact:** Impact was specific to users who were served through the affected infrastructure located in Europe._ _**Start time:** Monday, May 10, 2021, 8:40 AM \(6:40 AM UTC\)_ _**End time:** Monday, May 10, 2021, 11:40 AM \(9:40 AM UTC\)_ _**Root cause:** A section of infrastructure, responsible for connecting email message attributes to end users, was not performing as expected due to the associated cache not fully updating._ _**Next steps:** - We're reviewing our caching infrastructure to find ways to prevent this problem from happening again. This is the final update for the event._