[Resolved]: EU: Performance impact across EU deployment

Update timeline

1. resolved Oct 16, 2025, 11:20 AM UTC

   Incident details: Investigating: We have multiple internal alerts and field reports of performance and login issues with uniFLOW Online. Start Time: 10:30 UTC 16-10-2025 Incident Scope: EU (Europe Deployment) Description: Under investigation within the operations team. Next Update: The next update will be in 30 minutes or as information becomes avaliable

2. investigating Oct 16, 2025, 11:20 AM UTC

   Status changed to Investigating

3. investigating Oct 16, 2025, 11:30 AM UTC

   Hello Everyone, We are experiencing a performance lag holding up the release of print jobs. Currently the print queues are improving but some delays will still be felt in the field. The cause is still under investigation, but we are applying manual mitigations to help improve the recovery. Next update will be in 30 minutes or as information becomes available. Kind Regards Online Operations Team

4. monitoring Oct 16, 2025, 11:57 AM UTC

   **Update**: We are seeing print queues returned to normal and no delays visible. Confirmation has also been received from the field that the uniFLOW Online has returned to normal. While the mitigations have resolved the issue the root cause is still under investigation. We have moved this incident into monitoring as we follow the recovery process and continue the investigation. Next Update: The next update will be within the next 12 hours or as information warrants an update.

5. monitoring Oct 16, 2025, 11:57 AM UTC

   Status changed to Monitoring

6. monitoring Oct 16, 2025, 02:04 PM UTC

   Update: We can see a return of slowness in processing some requests in uniFLOW Online. We are still actively looking into the cause while attempting to restore services to normal. Next Update: The next update will be in 1 hour.

7. monitoring Oct 16, 2025, 03:00 PM UTC

   Update: We are currently experiencing intermittent performance issues affecting some request processing. Our team is actively investigating the cause and working to restore normal service levels. Next Update: We will provide the next update within 1 hour.

8. monitoring Oct 16, 2025, 04:00 PM UTC

   Update: We’re seeing some delays in processing email-based mobile print jobs. All other services are operating normally, and we’re working to restore full performance as quickly as possible. Next Update: We’ll share another update within 1 hour.

9. monitoring Oct 16, 2025, 05:06 PM UTC

   Update: The earlier delays in processing email-based mobile print jobs have been resolved. We are now restoring any email print jobs that were queued during the day. All other services remain fully operational. Next Update: The next update will be within the next 12 hours or as information warrants an update.

10. resolved Oct 16, 2025, 06:44 PM UTC

    Hello Everyone, Update: Incident Resolved. Date/Time: October 16th, 6:00 pm UTC We will review the findings and collected information from this incident to further improve our online services. There will be a post-mortem published once we have concluded our investigation but no longer then 20 business days. We are sorry for the inconvenience this has caused. Kind Regards Online Operations Team

11. resolved Oct 16, 2025, 06:44 PM UTC

    Status changed to Resolved

12. resolved Nov 12, 2025, 09:20 AM UTC

    **User Impact** Users would have intermittently noticed a slowdown in processing certain tasks in uniFLOW Online. This would have been noticeable when attempting to release print jobs as well as the processing of email print jobs and the delivering of emails amongst other tasks. **Scope of Impact:** EU Deployment **Incident Start Date and Time** October 15, 2025, 10:07 UTC **Incident End Date and Time** October 15, 2025, 18:05 UTC **Root Cause:** A storage account that uniFLOW Online makes use of to process scans was impacted by a service degradation on the Microsoft Azure side that led to increased latency and temporary throttling of requests from Microsoft Azure. This led to a delay in the processing of scan jobs in uniFLOW Online. Due to this delay other tasks that were queued to be processed were subsequently impacted. The delayed processing of scan jobs also resulted in uniFLOW Online generating emails to users informing them of the delay in processing their scan jobs. This, unfortunately, placed additional load on uniFLOW Online as well as the uniFLOW Online mail servers used to send and receive emails. **How did we respond:** 10:01 UTC a drop in successful requests against the impacted storage account began causing delays in scan job processing tasks. 10:08 UTC these delays started impacting other tasks. 10:12 UTC an alert was sent to the Online Operations Team triggering an investigation. 10:14 UTC additional resources were automatically brought online to assist with the tasks to be processed. 10:31 UTC the storage account recovers from the service degradation. Scan processing delays are resolved, and the systems start recovering. 11:09 UTC load against the email infrastructure starts. 11:19 UTC the system has recovered and all tasks except email processing is back to normal processing times. 12:17 UTC changes are made against the email infrastructure in an attempt to assist recovery. 12:48 UTC continued load against the email infrastructure results in additional intermittent slowdowns of task processing. 14:11 UTC noticeable improvement to system recovery. 15:05 UTC the system has recovered and all tasks except email processing is back to normal processing times. 16:40 UTC further mitigation controls put in place against the email infrastructure to assist recovery. 16:43 UTC email processing is recovered, and the system is monitored closely as queued emails are processed again. 18:05 UTC full system recovery and all email queues have returned to normal operating levels. **How are we making incidents like this less likely or less impactful?** We apologize for the impact on affected customers. We are continuously taking steps to improve the uniFLOW Online Platform and our processes to help ensure such incidents do not occur in the future. In this case, this includes (but is not limited to): We have escalated the initial storage degradation to Microsoft engineers who have confirmed a significant increase in resource utilization on their side resulting in the degradation. They have, additionally, stated that they are evaluating further improvements to prevent similar issues in the future. We have modified our email infrastructure globally on Saturday, 18 October 2025, to ensure better handling of such scenarios in future. **Was this incident related to previous incidents?** No, this incident is not related to previous incidents.