CodeTwo Outage History

After switching over to tertiary infrastructure, Azure is now processing our requests correctly and email delivery speed has returned to normal. Emails that were delayed on the previous two environments should be delivered soon with signatures applied. We are continuing to investigate why the primary Azure resources are unhealthy; however, all new emails are currently being processed through our regional failover services and are being delivered in a timely manner with signatures.

Microsoft has just published information about the outage on X: “We’re investigating a potential issue impacting multiple Microsoft 365 services, including Outlook, Microsoft Defender, and Microsoft Purview. Further information can be found in the admin center under MO1221364.” https://x.com/msft365status/status/2014422298506285161

Another update from Microsoft on X: "We've identified a portion of service infrastructure in North America that is not processing traffic as expected. We're working to restore the infrastructure to a healthy state to achieve recovery. More information can be found at https://status.cloud.microsoft or under MO1221364 if accessible."

Some of our apps - such as the Admin Panel and the Manage Signatures App - may experience degraded performance, including slow loading times or failure to load entirely. This issue is due to widespread disruptions affecting Microsoft 365 and Azure services. You can track updates via https://x.com/MSFT365Status/status/1976221477859983782 or https://status.cloud.microsoft, although the latter may also be intermittently unavailable as it's hosted by Microsoft. Microsoft has identified its Front Door services as the likely cause, which is also impacting standard Microsoft 365 users. We are actively monitoring the situation on our end. Despite the app loading issues, email signatures continue to be appended correctly, and mail delivery remains unaffected.

We’re seeing that the Azure Front Door Services issues are now also affecting users in the UK and UAE regions. As a result, Microsoft 365 users in these areas may experience intermittent login issues when accessing their M365 resources. Some CodeTwo apps - also hosted on Azure - may be impacted as well, due to the widespread nature of the disruption. Important Reminder: All email signatures are being appended as expected, and we are not observing any issues with mail delivery at this time. You can follow updates on this Microsoft incident via: https://x.com/MSFT365Status/status/1976221477859983782 https://status.cloud.microsoft (Note: this page may also be intermittently unavailable)

The problem with delays in email delivery appears to be limited to a small subset of customers in Australia who may have experienced degraded performance of Microsoft 365 today. Our services are operating normally and the signatures are imprinted. We're monitoring the situation.

Microsoft is currently experiencing Azure availability issues across several nodes in Australia. We have switched our infrastructure to healthy instances within the same region. However, some emails that were being processed during the failover are still stuck in the affected node. As a result, messages sent in the last 25 minutes may be delivered with a delay to final recipients. All current messages are being delivered without delay, and all emails include signatures as expected. At present, all services are operating normally, and no action is required on your end. We expect the issue to be resolved within approximately 30 minutes.

Microsoft is currently experiencing problems with Exchange Online Protection (EOP). EOP servers appear to have degraded performance and are slow to process emails entering and leaving Microsoft 365 tenants. As a result, some messages processed by CodeTwo may also be delayed while they wait for EOP to scan them before being delivered to final recipients. We have reported the issue to Microsoft and are waiting for a resolution. In the meantime, we have scaled up our services to increase the chances of routing emails through EOP servers that are not affected by this problem. What should you do if you're impacted? Inform your users that email delivery might be delayed (usually by just a few minutes or less) and wait for Microsoft to resolve the issue. Please note that this is a Microsoft-side problem, and it also affects users who are not using CodeTwo or any other smart hosts in the affected region. All signatures are added normally.

Microsoft has confirmed that the issue is on their end. The issue ID is EX1079935. While email delays in Australia are decreasing, we are now seeing the UK region slowly starting to be affected. We anticipated this and have already warmed up failover services to help mitigate the impact as the issue emerges in this region. Our services are continuously retrying to send messages that were previously stuck in queues before reaching the EOP servers so that we can send them through healthy EOP instances. All messages will be delivered with signatures, although some may be delayed.

There’s a problem with our certificate provider (Sectigo), which is currently unable to create new certificates for some customers trying to start a trial subscription. As a result, new users attempting to sign up for our server-side or combo-mode service won’t be able to complete the setup process until the issue is resolved. All existing subscriptions for existing Customers are unaffected. Signatures are being added as normal. We're monitoring the situation. Our Tech Support team is online - feel free to reach out to them if you have any questions.

The issue has been identified and a fix is being implemented.

Microsoft has just reported on their status page (https://azure.status.microsoft/status) that they are experiencing network infrastructure issues in ALL Azure regions. If you are experiencing problems signing in to CodeTwo services at the moment, this is due to the above mentioned problem. As of now, our email signature imprinting services are not affected. We are actively monitoring this case and will keep you updated. Here's the official status from Microsoft: Network Infrastructure - Issues accessing a subset of Microsoft services We are investigating reports of issues connecting to Microsoft services globally. Customers may experience timeouts connecting to Azure services. We have multiple engineering teams engaged to diagnose and resolve the issue. More details will be provided as soon as possible.