Qlik Cloud experienced a minor incident on February 7, 2024 affecting Qlik Cloud – AWS – US East (N. Virginia), lasting 7h 4m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Feb 07, 2024, 03:44 PM UTC
Our team has identified an issue with one of our backend services and is diligently working on resolving the issue. Customers with tenants in Qlik Cloud US may experience issues logging in. We are actively working to get this issue resolved.
- investigating Feb 07, 2024, 04:50 PM UTC
Our team is continuing to diligently work on resolving the issue. We will keep you posted on any further updates.
- investigating Feb 07, 2024, 06:04 PM UTC
Our team is continuing to diligently work in collaboration with our Cloud providers on resolving the issue. We will keep you posted on any further updates.
- identified Feb 07, 2024, 06:59 PM UTC
Our team has identified the issue and is diligently working on resolving the issue.
- identified Feb 07, 2024, 07:52 PM UTC
The team continues to work on implementing the fix. We appreciate your patience as the fix is taking more time than initially expected. We also plan to have it implemented shortly or will provide an update if we run into further delays.
- monitoring Feb 07, 2024, 08:13 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Feb 07, 2024, 10:48 PM UTC
This incident has been resolved.
- postmortem Feb 13, 2024, 06:52 PM UTC
# **Root Cause Analysis** On February 7, 2024, from 15:44 to 20:13 UTC, Qlik customers of the Qlik Cloud - US region were intermittently unable to log in or access their tenant. This was due to an impact on our backend services which required scaling and migration efforts, working with our cloud technology partners. During the last hour of the incident, as a part of the remediation, a resource contention on the backend caused an issue for all users. Restoration efforts proved ultimately effective but took longer than expected. The removal of resource contention coupled with the migration and scaling efforts restored service to all users. **Root Cause Summary and Remediation** Qlik's backend services became impacted due to a surge in internal traffic caused by an untested usage pattern. The ensuing traffic was atypical and triggered a deadlock in our backend database which delayed performing scale up operations until it was resolved. Qlik recognizes that this incident had an impact on our valued customers. To further reiterate our commitment to improving our customers' experience with Qlik, we have identified key areas to prevent a recurrence of this issue inclusive of heightened monitoring, additional governance and controls, and increased scalability testing.