Qonversion experienced a critical incident on September 7, 2025 affecting API and Realtime Dashboards and 1 more component, lasting 17h 38m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Sep 07, 2025, 11:20 AM UTC
We're experiencing an elevated level of API errors and are currently looking into the issue.
- identified Sep 07, 2025, 12:29 PM UTC
The issue has been identified and a fix is being implemented.
- identified Sep 07, 2025, 01:37 PM UTC
We switched our backend to the reserved backend solution, we're answering codes 200 for most of our API endpoints (public and SDK). These requests will be processed as it should be later. We confirmed the main issue, and we're on the way to fixing it.
- identified Sep 07, 2025, 02:53 PM UTC
We're still on fixing the main issue.
- identified Sep 07, 2025, 03:42 PM UTC
We located the main issue, it is connected with one of our cloud providers. We're preparing the workaround.
- identified Sep 07, 2025, 04:59 PM UTC
We have fixed the main issue and are currently testing it. We are working on switching back from the reserve backend solution to the main one.
- identified Sep 08, 2025, 12:01 AM UTC
The main issue was resolved at approximately 16:30 UTC on September 7th. Earlier we entered incident mode and switched to the backup backend. We are now merging historical events stage. At approximately 19:30 UTC, we resumed the delivery of delayed purchase events. Normal processing was fully restored by 21:00 UTC. The endpoints `users`, `purchases`, and `restores` are functioning well for now. The remaining endpoints are still operating through the backup backend and will be merged later.
- identified Sep 08, 2025, 07:04 AM UTC
We continue switching from backup backend to the main and merging historical events. In addition to previous, the following API endpoints works fine now: - `identity` (SDK identify()), - `init` (SDK inits), - GET `entitlements` (SDK method checkEntitlements), POST `entitlements`.
- resolved Sep 08, 2025, 12:39 PM UTC
The incident has been fully resolved. Over the past 12 hours, we have been monitoring our infrastructure and transitioning back from our backup backend solution (Aegis). Our team is currently restoring data for some previous purchases. All services are operating normally.