Amplience experienced a minor incident on October 30, 2024 affecting Dynamic Media, lasting 3h 19m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 30, 2024, 09:33 PM UTC
We are currently looking into an issue with Video Transcoding. We will update shortly once we have investigated further.
- identified Oct 30, 2024, 09:48 PM UTC
We have identified the issue and looking to roll back to a working version. We will update you once we have confirmed the roll back was successful.
- identified Oct 30, 2024, 10:43 PM UTC
We are testing a fix that is in place, we will update shortly once we have confirmed the fix is working as expected.
- identified Oct 30, 2024, 11:11 PM UTC
Video Transcoding is partially resolved. All new transcodes will process as normal. We are working on a fix for any affected videos stuck during the issue.
- resolved Oct 31, 2024, 12:52 AM UTC
The issue has now been resolved.
- postmortem Nov 01, 2024, 01:09 PM UTC
**Incident Start Date:** 30/10/2024, 15:10 GMT **Incident End Date:** 30/10/2024, 23:30 GMT **Summary:** During this time, some users may have encountered a video processing status that appeared "In Progress" indefinitely. **Root Cause:** While videos were processed successfully, the system was unable to confirm this to end users due to an authentication error following a recent release. As a result, the processing status appeared stuck, despite completion in the backend. **Resolution:** To resolve this, we rolled back the affected release and restored full functionality. We were able to identify all videos impacted and reprocess the correct status code \(In progress to complete\). We’re also implementing additional safeguards to prevent similar issues in the future