Clari incident

AI projections service is not working as expected

Minor Resolved View vendor source →

Clari experienced a minor incident on March 21, 2025 affecting Analyze, lasting 9h 38m. The incident has been resolved; the full update timeline is below.

Started
Mar 21, 2025, 10:56 AM UTC
Resolved
Mar 21, 2025, 08:35 PM UTC
Duration
9h 38m
Detected by Pingoru
Mar 21, 2025, 10:56 AM UTC

Affected components

Analyze

Update timeline

  1. investigating Mar 21, 2025, 10:56 AM UTC

    Some customers may be seeing problems with fetching AI predictions in Analytics Pulse view. Our engineering team is actively investigating.

  2. identified Mar 21, 2025, 12:24 PM UTC

    Our engineering teams have identified the issue and are implementing a fix. Many customers are unable to use AI projections in Analytics Pulse and Trend The next status update will be after the fix has been applied.

  3. identified Mar 21, 2025, 02:05 PM UTC

    Our engineering teams have identified the issue and are implementing a fix. Trend is not available for many customers and unable to use AI projections in Trend and Pulse. The next status update will be after the fix has been applied.

  4. monitoring Mar 21, 2025, 05:57 PM UTC

    Our engineering teams have resolved the issue and we are monitoring the fix.

  5. resolved Mar 21, 2025, 08:35 PM UTC

    We have confirmed the issue has been resolved for the impacted users. A summary of the incident will be posted within the next three US business days.

  6. postmortem Mar 26, 2025, 10:14 PM UTC

    At 07:44 UTC on March 21, 2025, a change to a key database caused queries to be executed inefficiently when retrieving certain data. This caused APIs serving the Clari AI Projection to time out which resulted in the Trend Analytics module being unavailable and the AI Projection in Pulse Analytics not loading. To resolve this issue we created a set of new indexes on this database to improve performance, with normal operations restored by 18:12 UTC. At no time during this incident was user data at risk.