Amplience had 37 outages in the last 2 years totaling 25h 48m of downtime — averaging 1.5 incidents per month.
There were 37 Amplience outages since July 4, 2024 totaling 25h 48m of downtime. Each is summarised below — incident details, duration, and resolution information.
We are aware that a subset of legacy processing jobs are experiencing intermittent issues. We have identified the cause and are working to restore normal service. We will provide further updates as progress continues.
We have successfully implemented a fix.
We are currently investigating scheduled editions that failed to publish between 05.07 - 05.52 BST
A third-party provider that we use for a backend service carried out emergency maintenance earlier today, which impacted some of our services. During this period, we identified a small number of Editions that failed to Publish. We have contacted the affected customers directly and provided a workaround to allow them to proceed. We also observed degraded performance impacting DC-API during the same timeframe. Our teams are continuing to monitor the situation closely while we work with the provider to understand the full impact and prevent any recurrence.
**Incident Start Time:** 05.07 BST **Incident End Time:** 05.52 BST **Issue Summary:** An incident occurred with one of our back end databases. This caused degraded performance to DC-API and some Dynamic Content Editions were in a stuck status. **Root Cause:** Some inefficient queries to our back end database caused our primary node to fail over to our secondary node. **Corrective Actions:** We have already introduced corrective mitigations with additional indexes and working with MongoDB we have further improvements planned.
We are investigating an issue where some scheduled editions are failing to publish.
We are investigating an issue in Dynamic Content, search indexes are unavailable
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
**Incident Start Time:** 19.49 BST **Incident End Time:** 22.03 BST **Issue Summary:** An incident occurred within Dynamic Content that required a rollback to a previous version. In the Development Tab of Dynamic Content, search indexes were returning zero records. **Root Cause:** The issue was introduced in a recent Dynamic Content release, which introduced a new Content Security Policy \(CSP\) header. This header unintentionally blocked Algolia search index requests, resulting in searches returning no records. **Corrective Actions:** * Improve validation and regression testing around CSP-related changes to ensure third-party services such as Algolia continue to function as expected. * Whitelist the security headers in a future release.
Users may experience an issue when switching repositories, where the selected repository updates but the displayed content does not reflect the newly selected repository.
A fix has been implemented and we are monitoring the results.
This incident has been resolved.
**Incident Start Time:** Friday, 8th May, 09.57 BST **Incident End Time:** Monday, 12th May, 14.18 BST **Issue Summary:** An incident occurred within Dynamic Content that required an immediate rollback to a previous version. Users were unable to navigate through different Repositories. **Root Cause:** The issue was introduced in the latest Content Management App release, which included a new multi-select feature for the repository library. The feature caused unintended behaviour that impacted repository navigation functionality. **Corrective Actions:** * Review release verification procedures to improve early detection of UI interaction issues.
We experienced a minor outage with app.amplience.com logging into Dynamic Content. The spike to performance has since subsided and we a now monitoring the situation. We can confirm that Dynamic Content is now accessible.
The issue is resolved. app.amplience.net logins are working as expected
**Incident Start Date** 19:40 BST **Incident End Date** 19:54 BST **Issue Summary** On Wednesday evening, users were unable to log into [app.amplience.net](http://app.amplience.net) for a period of 14 minutes. **Root Cause** Our third-party authentication provider, Auth0, experienced a DNS outage caused by a misconfiguration on their side. This incident also impacted some of our alerting capabilities, which we are currently reviewing with our partners. **Resolution** Auth0 resolved their DNS issue, restoring normal authentication services. **Corrective Actions** * Review and strengthen alerting to ensure visibility is maintained during third-party DNS-related incidents * Work with Auth0 to understand the full scope of the misconfiguration and their preventative measures
Feb 19, 16:07 GMT Resolved - We have monitored this issue and confirm we are fully operational Feb 19, 15:19 GMT Monitoring - We have identified the issue and Content Hub is now displaying assets, we are continuing to monitor the situation. Feb 19, 15:01 GMT Investigating - We are currently investigating an issue with assets not displaying on Content Hub
Jan 22, 17:02 GMT Resolved - A patch has now been released. Jan 22, 15:27 GMT Update - We are continuing to work on a fix for this issue. Jan 22, 15:23 GMT Identified - The issue has been identified and a fix is being implemented. Jan 22, 15:23 GMT Investigating - Users may experience issue editing nested content within the content form.
Some users may have experienced issues accessing ondemand.amplience.com and support.amplience.com. Email [email protected] remained operational during this time. To minimise any impact, we temporarily rerouted ondemand traffic away from the affected service. Traffic has now been restored to the normal route. Both issues were caused by a disruption affecting a third-party service. All issues are now resolved.
Nov 25, 10:44 GMT Resolved - This incident has been resolved, and purging functionality is now operating as expected Nov 25, 10:09 GMT Monitoring - We are currently observing an issue with one of our upstream partners, Akamai, which is experiencing an outage. As a result this will impact purging functionality within the Dynamic Media app. Our team is actively monitoring the situation and will provide further updates as they become available.
Nov 18, 19:27 GMT Resolved - We are satisfied the Cloudflare outage is resolved after our extended monitoring period. We can confirm we were not impacted and did not see any degraded performance. Our platform traffic is stable and we will continue to monitor our healthy service to ensure we remain operational. Nov 18, 13:25 GMT Monitoring - We continue to monitor our own infrastructure and can confirm that our own delivery services are not impacted. Traffic remains diverted away from Cloudflare. Nov 18, 12:15 GMT Identified - We are currently observing an issue with one of our upstream partners, Cloudflare, which is experiencing an outage. As a preventative measure, our platform is actively rebalancing traffic. Our systems continue to operate as expected, and customers may experience some intermittent performance degradation during this period. We are closely monitoring the situation and will continue to track Cloudflare’s recovery.
Oct 22, 16:09 BST Resolved - We can confirm that the affected feature has been rolled back and functionality has returned to normal. Oct 22, 16:06 BST Identified - We’ve identified an issue with a recent release that’s impacting content creation. Our team is currently rolling back the affected feature, and we’ll confirm once the rollback is complete
Oct 20, 10:44 BST Resolved - The incident has been resolved. Oct 20, 09:57 BST Investigating - Users may be experiencing slower than normal response times when contacting support. We are working with our provider to resolve this as soon as possible.
Oct 20, 10:43 BST Resolved - This incident is now resolved. Oct 20, 10:32 BST Monitoring - Our upstream partners have restored service, and we are monitoring. Oct 20, 10:04 BST Identified - We are currently seeing degraded Content Delivery performance in one region. Traffic is being redirected away from the affected region, and we are working with our upstream partners to restore normal service.
Sep 19, 22:54 BST Resolved - The incident has been resolved Sep 19, 21:51 BST Monitoring - A fix has been implemented and we are monitoring the results. Sep 19, 21:42 BST Identified - We are experiencing an outage with our Media Delivery (assets) service, which impacts published assets. We have identified the fault and now investigating the fix
Sep 11, 12:28 BST Resolved - This incident was resolved at 12.19 BST Sep 11, 12:19 BST Monitoring - Assets now display in content hub and we are monitoring. Sep 11, 12:16 BST Investigating - We are currently investigating the issue
We are currently investigating this issue.
Freshservice is now working as normal
This incident has been resolved.
We are currently investigating an issue with a third-party provider that is impacting manual uploads within Content Hub. Please note that bulk ingestion via FTP remains operational and can be used as an alternative in the meantime.
Service appears to be restored by the third party, and we will continue to observe
This Incident is resolved
We are currently investigating an issue where staging (Virtual Staging Environements) links return 504 error codes. Existing published images are not impacted.
A fix has been implemented, and we are monitoring the results.
This incident is now resolved.
Incident Start Time: 15.56 GMT Incident End Time: 16.20 GMT **Issue summary** On Tuesday, 4th March, users of Virtual Staging Environments \(VSE\) experienced 504 Gateway Timeouts errors when attempting to preview content and assets. The delivery of published content and assets were not impacted. **Root cause** Earlier in the day, we deployed a low-risk update to the VSE stack, which was successfully validated post-deployment. However, the service was not scaled appropriately to accommodate the traffic demands, ultimately reaching capacity and triggering 504 errors. The issue was resolved by scaling the service appropriately. **Corrective Actions** Future releases to the VSE will include proactively monitoring the drift within Cloudformation to ensure service capacity meets traffic requirements. If you would like to discuss this further, please contact your customer success manager \([[email protected]](mailto:[email protected])\)