Amplience had 34 outages in the last 2 years totaling 26h 31m of downtime — averaging 1.4 incidents per month.
There were 34 Amplience outages since July 4, 2024 totaling 26h 31m of downtime. Each is summarised below — incident details, duration, and resolution information.
We have identified the root cause of a known issue and are currently implementing a fix. Further information to follow.
The fix has been deployed and the impact is currently being monitored.
We can confirm the fix has been released and validated
We are investigating an issue in Dynamic Content, search indexes are unavailable
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
Users may experience an issue when switching repositories, where the selected repository updates but the displayed content does not reflect the newly selected repository.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
**Incident Start Time:** Friday, 8th May, 09.57 BST **Incident End Time:** Monday, 12th May, 14.18 BST **Issue Summary:** An incident occurred within Dynamic Content that required an immediate rollback to a previous version. Users were unable to navigate through different Repositories. **Root Cause:** The issue was introduced in the latest Content Management App release, which included a new multi-select feature for the repository library. The feature caused unintended behaviour that impacted repository navigation functionality. **Corrective Actions:** * Review release verification procedures to improve early detection of UI interaction issues.
We experienced a minor outage with app.amplience.com logging into Dynamic Content. The spike to performance has since subsided and we a now monitoring the situation. We can confirm that Dynamic Content is now accessible.
The issue is resolved. app.amplience.net logins are working as expected
**Incident Start Date** 19:40 BST **Incident End Date** 19:54 BST **Issue Summary** On Wednesday evening, users were unable to log into [app.amplience.net](http://app.amplience.net) for a period of 14 minutes. **Root Cause** Our third-party authentication provider, Auth0, experienced a DNS outage caused by a misconfiguration on their side. This incident also impacted some of our alerting capabilities, which we are currently reviewing with our partners. **Resolution** Auth0 resolved their DNS issue, restoring normal authentication services. **Corrective Actions** * Review and strengthen alerting to ensure visibility is maintained during third-party DNS-related incidents * Work with Auth0 to understand the full scope of the misconfiguration and their preventative measures
Feb 19, 16:07 GMT Resolved - We have monitored this issue and confirm we are fully operational Feb 19, 15:19 GMT Monitoring - We have identified the issue and Content Hub is now displaying assets, we are continuing to monitor the situation. Feb 19, 15:01 GMT Investigating - We are currently investigating an issue with assets not displaying on Content Hub
Jan 22, 17:02 GMT Resolved - A patch has now been released. Jan 22, 15:27 GMT Update - We are continuing to work on a fix for this issue. Jan 22, 15:23 GMT Identified - The issue has been identified and a fix is being implemented. Jan 22, 15:23 GMT Investigating - Users may experience issue editing nested content within the content form.
Some users may have experienced issues accessing ondemand.amplience.com and support.amplience.com. Email [email protected] remained operational during this time. To minimise any impact, we temporarily rerouted ondemand traffic away from the affected service. Traffic has now been restored to the normal route. Both issues were caused by a disruption affecting a third-party service. All issues are now resolved.
Nov 25, 10:44 GMT Resolved - This incident has been resolved, and purging functionality is now operating as expected Nov 25, 10:09 GMT Monitoring - We are currently observing an issue with one of our upstream partners, Akamai, which is experiencing an outage. As a result this will impact purging functionality within the Dynamic Media app. Our team is actively monitoring the situation and will provide further updates as they become available.
Nov 18, 19:27 GMT Resolved - We are satisfied the Cloudflare outage is resolved after our extended monitoring period. We can confirm we were not impacted and did not see any degraded performance. Our platform traffic is stable and we will continue to monitor our healthy service to ensure we remain operational. Nov 18, 13:25 GMT Monitoring - We continue to monitor our own infrastructure and can confirm that our own delivery services are not impacted. Traffic remains diverted away from Cloudflare. Nov 18, 12:15 GMT Identified - We are currently observing an issue with one of our upstream partners, Cloudflare, which is experiencing an outage. As a preventative measure, our platform is actively rebalancing traffic. Our systems continue to operate as expected, and customers may experience some intermittent performance degradation during this period. We are closely monitoring the situation and will continue to track Cloudflare’s recovery.
Oct 22, 16:09 BST Resolved - We can confirm that the affected feature has been rolled back and functionality has returned to normal. Oct 22, 16:06 BST Identified - We’ve identified an issue with a recent release that’s impacting content creation. Our team is currently rolling back the affected feature, and we’ll confirm once the rollback is complete
Oct 20, 10:44 BST Resolved - The incident has been resolved. Oct 20, 09:57 BST Investigating - Users may be experiencing slower than normal response times when contacting support. We are working with our provider to resolve this as soon as possible.
Oct 20, 10:43 BST Resolved - This incident is now resolved. Oct 20, 10:32 BST Monitoring - Our upstream partners have restored service, and we are monitoring. Oct 20, 10:04 BST Identified - We are currently seeing degraded Content Delivery performance in one region. Traffic is being redirected away from the affected region, and we are working with our upstream partners to restore normal service.
Sep 19, 22:54 BST Resolved - The incident has been resolved Sep 19, 21:51 BST Monitoring - A fix has been implemented and we are monitoring the results. Sep 19, 21:42 BST Identified - We are experiencing an outage with our Media Delivery (assets) service, which impacts published assets. We have identified the fault and now investigating the fix
Sep 11, 12:28 BST Resolved - This incident was resolved at 12.19 BST Sep 11, 12:19 BST Monitoring - Assets now display in content hub and we are monitoring. Sep 11, 12:16 BST Investigating - We are currently investigating the issue
Jun 27, 08:55 BST Resolved - This incident has been resolved. Jun 27, 06:42 BST Identified - The issue has been identified and a fix is being implemented. Jun 27, 04:28 BST Investigating - We are currently investigating this issue.
We are currently investigating this issue.
Freshservice is now working as normal
This incident has been resolved.
We are currently investigating an issue with a third-party provider that is impacting manual uploads within Content Hub. Please note that bulk ingestion via FTP remains operational and can be used as an alternative in the meantime.
Service appears to be restored by the third party, and we will continue to observe
This Incident is resolved
We are currently investigating an issue where staging (Virtual Staging Environements) links return 504 error codes. Existing published images are not impacted.
A fix has been implemented, and we are monitoring the results.
This incident is now resolved.
Incident Start Time: 15.56 GMT Incident End Time: 16.20 GMT **Issue summary** On Tuesday, 4th March, users of Virtual Staging Environments \(VSE\) experienced 504 Gateway Timeouts errors when attempting to preview content and assets. The delivery of published content and assets were not impacted. **Root cause** Earlier in the day, we deployed a low-risk update to the VSE stack, which was successfully validated post-deployment. However, the service was not scaled appropriately to accommodate the traffic demands, ultimately reaching capacity and triggering 504 errors. The issue was resolved by scaling the service appropriately. **Corrective Actions** Future releases to the VSE will include proactively monitoring the drift within Cloudformation to ensure service capacity meets traffic requirements. If you would like to discuss this further, please contact your customer success manager \([[email protected]](mailto:[email protected])\)
We are currently investigating the cause of the bulk upload issues experienced over the last 24 hours, we will provide further updates as soon as possible.
Our operations team have confirmed the backlog of stuck processing jobs have now cleared.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
Incident Start Time: 19/01/2025 22.00 GMT Incident End Time: 21/01/2025 12.30 GMT **Issue summary** On Sunday night, some bulk ingestion jobs were initiated but did not complete, resulting in them becoming stuck. **Root cause** This was due to unexpected behaviour from routine repair services on our back-end database. **Corrective actions** Additional alarms have been implemented to monitor and capture this new behaviour. Upgrades to this part of our platform are already underway to improve stability and performance. If you would like to discuss this further, please contact your customer success manager \([[email protected]](mailto:[email protected])\)
We are aware of an issue using enums in the content form screen, we are looking into this and will provide further updates soon.
The issue has been identified and a rollback is being implemented.
We have implemented a rollback plan and are currently confirming this has fixed the issue.
This incident has been resolved.
Incident Start Time: 16.11 GMT Incident End Time: 19.55 GMT **Issue summary** An incident occurred within Dynamic Content that necessitated an immediate rollback to a previous version. The symptom of this incident was observed during content creation, where a schema using enums did not list any values. **Root cause** A recent Dynamic Content release introduced a critical bug. **Corrective actions** To restore service, we immediately rolled back to a previous version. If you would like to discuss this further, please contact your customer success manager \([[email protected]](mailto:[email protected])\)
We are currently investigating an issue with VSEs which may experience degraded performance
We have found and resolved the issue
The incident was resolved at 19:56 and no further issues were seen.
**Incident Report** **Incident Start Date:** 01/12/2024 17:50 **Incident End Date:** 01/12/2024 19:56 **Summary** On the evening of December 1st, our monitoring systems detected degraded performance in our Virtual Staging Environment. During this time, some customers may have encountered 5XX errors when previewing images. It is important to note that this issue was isolated to image previews and **did not** impact the dynamic image service. **Root Cause** The disruption was triggered by an unexpected surge in traffic to our Virtual Staging Environment. **Resolution** To resolve the issue, we promptly scaled up the service to handle the increased load.