Coveo Cloud experienced a major incident on March 19, 2025 affecting Analytics - Analytics Read API and Commerce - Merchandising Hub, lasting 8h 39m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Mar 19, 2025, 03:08 PM UTC
We're investigating an issue affecting the Analytics Read service in the US region. Analytics reports and dashboards in this region are currently unavailable. Our team is investigating and will post regular updates until solved. If you need help or to get in touch with us, please visit our Help Portal
- investigating Mar 19, 2025, 03:11 PM UTC
We are continuing to investigate this issue.
- monitoring Mar 19, 2025, 03:26 PM UTC
A fix has been implemented and the impact has been mitigated. We are continuing to closely monitor the service.
- identified Mar 19, 2025, 03:34 PM UTC
We've now confirmed that the issues are caused by a problem which has been acknowledged by one of our infrastructure providers. Although a capacity increase appeared to momentarily resolve the issues, we are now seeing intermittent errors again. For now this means the outage is partial and we are continuing to look at potential mitigation actions.
- identified Mar 19, 2025, 05:47 PM UTC
The incident with our infrastructure provider is still ongoing. In an attempt to decrease the load on the infrastructure we have temporarily paused analytics data processing workflows in this region. This improves the success rate of queries being made against the service at the cost of a delay in the freshness of the data. We will continue to provide updates until solved.
- identified Mar 19, 2025, 06:25 PM UTC
We are continuing to work on a fix for this issue.
- identified Mar 19, 2025, 08:06 PM UTC
The incident is being mitigated by our cloud infrastructure provider. We are seeing recovery with most queries now being successful. Analytics data processing has been resumed, though over the course of this incident we accumulated a backlog that will need to be processed. Dashboards and reports are available again but the data they display may not be up to date.
- resolved Mar 19, 2025, 11:48 PM UTC
This incident has been resolved and the analytics event backlog has been fully processed. Thank you for your patience.