Identifi incident
We are investigating a problem in our Azure environment
Identifi experienced a critical incident on August 16, 2024 affecting Identifi Azure US Status, lasting 2h 50m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Aug 16, 2024, 03:32 PM UTC
The issue has been identified and a fix is being implemented.
- identified Aug 16, 2024, 04:20 PM UTC
We are continuing to investigate a fix for this issue.
- monitoring Aug 16, 2024, 04:42 PM UTC
A fix has been implemented and we will continue to monitor.
- resolved Aug 16, 2024, 06:23 PM UTC
This incident has been resolved.
- postmortem Aug 20, 2024, 12:53 PM UTC
At approximately 11:20 AM US ET \(15:20 UTC\) on August 16, 2024, Identifi observed that a sub-set of customers hosted in our Azure infrastructure were not available. Identifi staff began troubleshooting the issue. By approximately 12:45 PM US ET Identifi staff found and resolved the problem. All impacted customers were brought back online within 10 minutes. Root Cause An internal Encryption Key expired resulting in a failure of the Identifi Product to connect to databases and process cookies. Lessons Learned and Process Improvements Identifi has found the cloud resource that caused key ring failures during key rotation. We have added proper permissions and resolved the issue. Additional monitoring has been put in place to warn of any future issues of this type.