CodeTwo experienced a minor incident on July 22, 2024 affecting Outlook add-in and Outlook add-in and 1 more component, lasting 4h 28m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jul 22, 2024, 04:33 PM UTC
Microsoft is currently experiencing a problem with Microsoft 365 add-ins. Some add-ins are not working or working intermittently for a subset of users. All add-ins may also completely disappear from the Microsoft store occasionally. This is a Microsoft issue, and it's not limited to CodeTwo since we observe that other add-ins (even Microsoft native add-ins) are also affected. As a result of this problem, CodeTwo signatures might not be added in Outlook (desktop, OWA and mobile). Server-side (cloud) signatures are added normally. Only signatures added by the Outlook add-in might be affected. We're currently investigating this with Microsoft.
- identified Jul 22, 2024, 05:07 PM UTC
Microsoft engineering team has reported: We have discovered an issue with the backend service that provides add-ins, and we have engaged the backend team to address this ASAP. We are making progress and will let you know as we get more information from them. Just as a reminder - Outlook add-ins are currently affected due to a Microsoft issue, but server-side (cloud) signatures are added normally. Only signatures added by the Outlook add-in might not work.
- monitoring Jul 22, 2024, 07:14 PM UTC
The Outlook product group is currently deploying a fix to production. It should be available worldwide within 45 minutes.
- monitoring Jul 22, 2024, 08:08 PM UTC
The fix has been deployed. Some customers already notified us the problem is solved (so did our Q&A team and SOC). We are currently helping Microsoft monitor for any further issues.
- monitoring Jul 22, 2024, 08:49 PM UTC
We can confirm most of users are no longer affected by the issue. Some users might need to restart Outlook or refresh OWA to fully fix the problem.
- resolved Jul 22, 2024, 09:02 PM UTC
This incident has been resolved.
- postmortem Jul 24, 2024, 09:19 AM UTC
Microsoft has officially confirmed the problem was on their end, and published communication about this incident to the Service Health section of the Microsoft 365 Admin Center for all affected tenants \(Communication ID: EX829938\). You can find it in the “Issue history” section. We also pasted it below: # _Users may be unable able to use already acquired add-ins within the Outlook desktop app and Outlook on the web_ _**User impact**_ _Users may have been unable to use already acquired add-ins within the Outlook desktop app and Outlook on the web._ _**More info**_ _The issue could have been seen in either the new or classic Outlook desktop app, as well as within the Outlook mobile app._ _**Scope of impact**_ _This issue may have affected any user of already acquired add-ins within the new or classic Outlook desktop app, Outlook on the web, and the Outlook mobile app._ _**Root cause**_ _A recent code change resulted in a bug affecting the service that gets add-in manifests into the host, which led to this impact._ _**Next steps**_ _-We’re \(Microsoft\) reviewing our code change testing and validation procedures to assess areas for improvement to avoid similar issues from being reintroduced in the future._ _- We’re \(Microsoft\) refining our understanding of the code bug by reviewing what the code change intended to do and analyzing how it led to this behavior, so that we can better understand how it caused the impact._