Degraded Service in both Client and Server-side signatures for UK and EU regions.

Update timeline

1. investigating Mar 27, 2025, 11:48 AM UTC

   Exclaimer Engineers are currently investigating a possible service alert that may be causing an impact with signature application at this time. Once we have more information from our Engineers this will be provided

2. investigating Mar 27, 2025, 12:04 PM UTC

   We are continuing to investigate this issue.

3. monitoring Mar 27, 2025, 12:18 PM UTC

   We are now seeing signatures being applied through our Add-in and routed through Exclaimer services as expected now. Engineering are continuing to monitor and investigate the cause.

4. resolved Mar 27, 2025, 02:21 PM UTC

   After extended monitoring, Exclaimer Engineers have confirmed that signatures are now being applied to messages when sent or whilst composing. A full postmortem will be provided shortly. If you are continuing to see issues, please raise a ticket with our Support Team who will be able to assist.

5. postmortem Mar 28, 2025, 09:56 AM UTC

   Issue: Degraded Service in both Client and Server-side signatures for UK and EU regions. Incident Length: 1 hour and 2 minutes Incident Date: 27/03/2025, UTC 11:23 – 12:25 27/03/2025 Incident Status: Resolved **Summary** Customers within the EU and UK experienced failures when trying to retrieve their signatures from the Exclaimer Add-in and would encounter delays in message delivery. **Root Cause** Initial implementation work of a new feature within Exclaimer branding system resulted in an unhandled exception whilst the system looked to generate signature templates. The coinciding exception that occurred resulted in the message being queued for retry in applying the signature. This caused a buildup of requests for signatures to be applied to messages. **Mitigation** Increased resources were provided to impacted regions to reduce the buildup of requests, and the changes were rolled back to resolve the issue. **Incident Timeline** 11:17 – Implementation work for a new branding feature was completed and enabled within Exclaimer Cloud. 11:17 - Initial warnings were generated at this time but investigation indicated none of these were customer impacting. 11:22 – Additional alerting advised to an increase in wait time for Exclaimer Add-in applied signatures. 11:30 – SMTP requests to Exclaimer services begin to respond to connection attempts with a deferral due to a lack of signature server processing availability, mail flow alerts are raised due to increased processing time of messages. 11:37 – Exclaimer engineers begin to investigate the issue. 11:47 – A recent deployment to the signature template analytic services is isolated as a potential cause of the issue. 11:50 – The deployment is rolled back within the UK region. Engineers identify that the error rate is unchanged.` `11:51 – Implementation work was disabled but left present within Exclaimer systems. 11:59 – Resourcing within UK and EU regions are increased to process the buildup of signature requests. 12:12 – Exclaimer Engineers confirm that the service is now recovering. Services are continued to be monitored to ensure that the issue does not reoccur. 12:17 – Mail flow duration returned to normal levels. 12:25 – Exclaimer Add-in requests no longer reported any related errors. 12:49 – Exclaimer Engineers identify the cause of the issue and rollback the feature.