IDVerse experienced a major incident on December 12, 2024 affecting OCR & DFA Engine API and OCR & DFA Engine API and 1 more component, lasting 2h 38m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Dec 12, 2024, 06:09 AM UTC
As of approximately 3:30PM AEDT 12/12 (4:30am UTC) end users conducting IDV applications may have experienced issues with proper document upload, resulting in errors IDverse technical teams are working to resolve this ASAP and will provide regular updates as information becomes available
- monitoring Dec 12, 2024, 06:25 AM UTC
At of 4:30pm AEDT (5:30am UTC) IDVerse technical teams have resolved the underlying errors. IDV transactions should now progress through for end-users We will continue to monitor the situation and provide further updates as more information becomes available
- resolved Dec 12, 2024, 08:47 AM UTC
This incident has been resolved. Postmortem to follow.
- postmortem Dec 13, 2024, 05:08 AM UTC
### Errors when uploading documents due to OCR Engine API layer deployment ### **Summary** An error in the deployment process occurred when making routine updates to the API layer. This update was deployed at 05:35 UTC . While the engines themselves were up and running, users attempting an IDV during this time would not have been able to complete their IDV transaction given the OCR engine directly impacts the uploading of documents. ### **Remediation** The deployment was rolled back and completed at approx 06:40 UTC. ### **Future improvements identified** Our API layer deployment process is currently under review to identify improvements, including further automation, to mitigate against system disruptions. We are also reviewing infrastructure monitoring, including how the notifications flow to impacted customers.