Amplience incident

Staging links return 504 errors

Resolved after 5 minutes, affecting DAM and Dynamic Media, on March 4, 2025.

postmortem Mar 07, 2025, 02:00 PM UTC

Incident Start Time: 15.56 GMT Incident End Time: 16.20 GMT ‌ **Issue summary** On Tuesday, 4th March, users of Virtual Staging Environments \(VSE\) experienced 504 Gateway Timeouts errors when attempting to preview content and assets. The delivery of published content and assets were not impacted. ‌ **Root cause** Earlier in the day, we deployed a low-risk update to the VSE stack, which was successfully validated post-deployment. However, the service was not scaled appropriately to accommodate the traffic demands, ultimately reaching capacity and triggering 504 errors. ‌ The issue was resolved by scaling the service appropriately. ‌ **Corrective Actions** Future releases to the VSE will include proactively monitoring the drift within Cloudformation to ensure service capacity meets traffic requirements. ‌ ‌If you would like to discuss this further, please contact your customer success manager \([[email protected]](mailto:[email protected])\)

Started
Mar 04, 2025, 04:16 PM UTC
Resolved
Mar 04, 2025, 04:22 PM UTC
Duration
5m
Detected by Pingoru
Mar 04, 2025, 04:16 PM UTC

Affected components

DAMDynamic Media

Update timeline

  1. investigating Mar 04, 2025, 04:16 PM UTC

    We are currently investigating an issue where staging (Virtual Staging Environements) links return 504 error codes. Existing published images are not impacted.

  2. monitoring Mar 04, 2025, 04:20 PM UTC

    A fix has been implemented, and we are monitoring the results.

  3. resolved Mar 04, 2025, 04:22 PM UTC

    This incident is now resolved.

  4. postmortem Mar 07, 2025, 02:00 PM UTC

    Incident Start Time: 15.56 GMT Incident End Time: 16.20 GMT ‌ **Issue summary** On Tuesday, 4th March, users of Virtual Staging Environments \(VSE\) experienced 504 Gateway Timeouts errors when attempting to preview content and assets. The delivery of published content and assets were not impacted. ‌ **Root cause** Earlier in the day, we deployed a low-risk update to the VSE stack, which was successfully validated post-deployment. However, the service was not scaled appropriately to accommodate the traffic demands, ultimately reaching capacity and triggering 504 errors. ‌ The issue was resolved by scaling the service appropriately. ‌ **Corrective Actions** Future releases to the VSE will include proactively monitoring the drift within Cloudformation to ensure service capacity meets traffic requirements. ‌ ‌If you would like to discuss this further, please contact your customer success manager \([[email protected]](mailto:[email protected])\)

Your product runs on more than Amplience

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →