Amplience incident

Scheduling delays

Resolved after 4 hours 26 minutes, affecting CMS, on June 27, 2025.

postmortem Jul 02, 2025, 01:49 PM UTC

## Issue Summary: Between 26 and 27 June, some users experienced delays in content publishing and inconsistencies in the UI. In certain cases, content appeared to be stuck in a “Scheduling” state, even though it had successfully published on the backend. ## Root Cause: The incident was triggered by the localization of a large content item involving over 16,000 entries. This placed unexpected load on part of our infrastructure, leading to memory pressure and delays in processing publish events. As a result, UI updates did not accurately reflect the publishing status. ## Resolution: We increased system resources and redeployed affected services, which restored full functionality. ‌ ## Corrective Actions: To help prevent similar issues in future, we have already: * Added new alerts to proactively monitor system load and memory usage. * Improved internal documentation and escalation processes. We are also: * Reviewing how large-scale content changes are processed across the platform to improve system resilience. ‌ If you have any questions or would like to discuss this further, please contact your Customer Success Manager or email us at [**[email protected]**](mailto:[email protected])**.**

Started
Jun 27, 2025, 03:28 AM UTC
Resolved
Jun 27, 2025, 07:55 AM UTC
Duration
4h 26m
Detected by Pingoru
Jun 27, 2025, 03:28 AM UTC

Affected components

CMS

Update timeline

  1. investigating Jun 27, 2025, 03:28 AM UTC

    We are currently investigating this issue.

  2. identified Jun 27, 2025, 05:42 AM UTC

    The issue has been identified and a fix is being implemented.

  3. resolved Jun 27, 2025, 07:55 AM UTC

    This incident has been resolved.

  4. postmortem Jul 02, 2025, 01:49 PM UTC

    ## Issue Summary: Between 26 and 27 June, some users experienced delays in content publishing and inconsistencies in the UI. In certain cases, content appeared to be stuck in a “Scheduling” state, even though it had successfully published on the backend. ## Root Cause: The incident was triggered by the localization of a large content item involving over 16,000 entries. This placed unexpected load on part of our infrastructure, leading to memory pressure and delays in processing publish events. As a result, UI updates did not accurately reflect the publishing status. ## Resolution: We increased system resources and redeployed affected services, which restored full functionality. ‌ ## Corrective Actions: To help prevent similar issues in future, we have already: * Added new alerts to proactively monitor system load and memory usage. * Improved internal documentation and escalation processes. We are also: * Reviewing how large-scale content changes are processed across the platform to improve system resilience. ‌ If you have any questions or would like to discuss this further, please contact your Customer Success Manager or email us at [**[email protected]**](mailto:[email protected])**.**

Your product runs on more than Amplience

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →