Onfido experienced a critical incident on February 24, 2025 affecting API and API and 1 more component, lasting —. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- resolved Feb 24, 2025, 02:36 PM UTC
Onfido experienced a critical outage across all regions on Feb 24th, 2025, at 14:15 UTC, lasting 9 minutes (resolved on Feb 24th, 2025, 14:24 UTC). During this time, any request made to our systems failed, resulting in no applicants or reports being created and no uploads being processed. We take pride in running a robust, reliable service and are working hard to prevent this from happening again. Once we conclude our investigation, we will publish a detailed postmortem.
- postmortem Feb 27, 2025, 04:37 PM UTC
### Summary Onfido experienced an outage on Feb 24th, 14:14 UTC, for 9 minutes. Client API requests to upload documents and create checks returned an error response during this period. While this also impacted our SDK traffic, our logs indicate a very small number of user sessions failed to complete, as they were largely recovered by re-tries. ### Root Causes During preparations to rollout a planned database update, part of the change was inadvertently pushed to our production environment before it was ready. This was due to a misconfigured release pipeline. ### Timeline 14:14 UTC: A change to a database is inadvertently pushed to production 14:15 UTC: The team responsible for this upgrade is alerted to an increase in database errors 14:17 UTC: The problem is identified, and a fix is released 14:22 UTC: The fix is fully applied to impacted regions ### Remedies The offending pipeline is being corrected and an investigation will be done to assess if any other pipeline suffers from the same misconfiguration. Additional controls will be established to avoid such misconfigurations in future.