Decisions incident

Service degradation for creating Meeting Recaps

Minor Resolved View vendor source →

Decisions experienced a minor incident on November 6, 2024 affecting Decisions AI, lasting 19d 18h. The incident has been resolved; the full update timeline is below.

Started
Nov 06, 2024, 06:54 PM UTC
Resolved
Nov 26, 2024, 01:39 PM UTC
Duration
19d 18h
Detected by Pingoru
Nov 06, 2024, 06:54 PM UTC

Affected components

Decisions AI

Update timeline

  1. identified Nov 06, 2024, 06:54 PM UTC

    We are investigating a service degradation for creating Meeting Recap with AI. The functionality still works, but can tak up to multiple minutes to finalize. Based on insight up to this point, it seems to be related to Azure OpenAI server processing performance.

  2. identified Nov 07, 2024, 12:58 PM UTC

    We are continuing to see reduced performance and cases of failures with Meeting Recap. We are working with Microsoft to improve the service for the Azure OpenAI environment.

  3. monitoring Nov 12, 2024, 08:44 AM UTC

    The Meeting Recap process is still experiencing service degradation. The number of failures have been drastically reduced, but some users will experience slower than normal processing of Meeting Recap. The service degradation is caused by performance issues in the MS Azure OpenAI infrastructure. We are actively working with MS product teams to improve the performance.

  4. resolved Nov 26, 2024, 01:39 PM UTC

    Microsoft have confirmed that the underlying issue with some requests failing in Azure OpenAI is addressed, and the indirect issue of capacity being improved