Amplience experienced a notice incident on January 21, 2025 affecting Content Hub, lasting 4h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jan 21, 2025, 10:40 AM UTC
We are currently investigating the cause of the bulk upload issues experienced over the last 24 hours, we will provide further updates as soon as possible.
- identified Jan 21, 2025, 12:36 PM UTC
Our operations team have confirmed the backlog of stuck processing jobs have now cleared.
- monitoring Jan 21, 2025, 02:32 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jan 21, 2025, 02:41 PM UTC
This incident has been resolved.
- postmortem Jan 23, 2025, 11:11 AM UTC
Incident Start Time: 19/01/2025 22.00 GMT Incident End Time: 21/01/2025 12.30 GMT **Issue summary** On Sunday night, some bulk ingestion jobs were initiated but did not complete, resulting in them becoming stuck. **Root cause** This was due to unexpected behaviour from routine repair services on our back-end database. **Corrective actions** Additional alarms have been implemented to monitor and capture this new behaviour. Upgrades to this part of our platform are already underway to improve stability and performance. If you would like to discuss this further, please contact your customer success manager \([[email protected]](mailto:[email protected])\)