Tungsten Automation EMEA incident

AP Essentials - Slow processing of emails

Major Resolved View vendor source →

Tungsten Automation EMEA experienced a major incident on July 1, 2025 affecting AP Essentials, lasting 5h 53m. The incident has been resolved; the full update timeline is below.

Started
Jul 01, 2025, 10:12 AM UTC
Resolved
Jul 01, 2025, 04:05 PM UTC
Duration
5h 53m
Detected by Pingoru
Jul 01, 2025, 10:12 AM UTC

Affected components

AP Essentials

Update timeline

  1. investigating Jul 01, 2025, 10:12 AM UTC

    We are currently investigating an issue with slow processing of emails.

  2. monitoring Jul 01, 2025, 12:01 PM UTC

    A fix has been implemented and we are monitoring the results.

  3. resolved Jul 01, 2025, 04:05 PM UTC

    The incident has now been resolved and we will continue to monitor the service.

  4. postmortem Jul 08, 2025, 02:45 PM UTC

    ## Root Cause Analysis **Event** Product / Platform: AP Essentials Date Reported: 07/01/2025 Date Resolved: 07/03/2025 Severity Level: Critical **Overview** Between July 1 and July 3, 2025, the APS Essentials email processing system experienced intermittent performance issues, leading to delays and failures in processing inbound emails. This incident affected some emails submitted by suppliers and customers. **Incident Detail** Our system monitoring detected periods of slow email processing, and further investigation confirmed that some emails failed to be processed at all. These issues were intermittent and did not affect all messages equally. Immediately upon detection, our team took action to reprocess all emails that had not been processed by the system. Most affected emails were successfully recovered and processed as intended. However, some emails may still not have been processed, even after reprocessing attempts. **Resolution** We addressed the performance issues by significantly increasing the servers' capacities and continue to monitor the system to maintain stable operation. All identifiable unprocessed emails have been located and released. If a supplier or customer still notices missing invoices that were not processed, we kindly ask them to resend the affected documents to ensure successful delivery. We apologize for any inconvenience this issue may have caused and thank you for your understanding.