Coveo HIPAA incident

Generative Answering Issues

Resolved after 2 hours 37 minutes, affecting Coveo ML - Generative Answering, on June 21, 2024.

resolved Jun 21, 2024, 06:43 PM UTC

All production model rebuilds have now been completed. The incident has been resolved.

Started
Jun 21, 2024, 04:05 PM UTC
Resolved
Jun 21, 2024, 06:43 PM UTC
Duration
2h 37m
Detected by Pingoru
Jun 21, 2024, 04:05 PM UTC

Affected components

Coveo ML - Generative Answering

Update timeline

  1. investigating Jun 21, 2024, 04:05 PM UTC

    We are investigating an issue impacting the Generative Answering API that is currently affecting a subset of models that were recently updated to the latest GPT-3.5 version, causing some minor variance in the answers returned to some queries. Our team is actively working on this issue and will post regular updates until solved. If you need help or to get in touch with us, please visit our Help Portal

  2. identified Jun 21, 2024, 04:39 PM UTC

    The issue has been identified and a fix is being rolled out.

  3. identified Jun 21, 2024, 05:23 PM UTC

    The fix has been deployed and validated. We have now begun rebuilding the impacted models but expect this process to take several hours. Live production models are being prioritized for immediate rebuilds while non-production ones will be left to be rebuilt as per their respective schedules in the coming week.

  4. resolved Jun 21, 2024, 06:43 PM UTC

    All production model rebuilds have now been completed. The incident has been resolved.

Your product runs on more than Coveo HIPAA

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →