Templafy incident

Document agents not responding in West Europe (Production 0), West Europe (Production 1) and West Europe (Production 5)

Major Resolved View vendor source →

Templafy experienced a major incident on September 29, 2026 affecting AI Assistant, lasting 1h 56m. The incident has been resolved; the full update timeline is below.

Started
Sep 29, 2026, 05:01 PM UTC
Resolved
Sep 29, 2026, 06:58 PM UTC
Duration
1h 56m
Detected by Pingoru
Sep 29, 2026, 05:01 PM UTC

Affected components

AI Assistant

Update timeline

  1. investigating Sep 29, 2026, 05:01 PM UTC

    Intermittent request failures and increased latency across Azure OpenAI, Azure AI Foundry, and Cognitive Services. We are following the updates from Microsoft.

  2. monitoring Sep 29, 2026, 06:56 PM UTC

    Microsoft pushed a fix which seems to resolve the issue. We will monitor document agents in Templafy at this point of time.

  3. resolved Sep 29, 2026, 06:58 PM UTC

    The incident has been resolved, and further information will be provided in a postmortem shortly. We apologize for the impact to affected customers.

  4. postmortem Oct 01, 2026, 08:16 AM UTC

    # Investigation On September 29, 2026, at approximately 12:03 PM CET, an upstream Microsoft Azure platform issue began affecting services supporting Document Agents. Customers experienced Document Agents not responding as expected. Requests to the supporting Microsoft services experienced intermittent failures, increased latency, and HTTP 5XX errors. The investigation identified an upstream Microsoft Azure platform issue affecting Azure OpenAI Service, Foundry Agent Service, Foundry Models, and Cognitive Services. The disruption primarily affected services hosted in Sweden Central, with regional impact also observed in Spain Central. As the investigation progressed, the engineering team monitored updates from Microsoft and validated Document Agent functionality internally to determine when the dependent services had recovered. # Mitigation The engineering team continued monitoring the affected Azure services while Microsoft worked to restore the impacted platform components. At approximately 8:54 PM CET on September 29, 2026, Microsoft implemented remediation for the underlying platform issue. Following this change, our engineering team began monitoring service recovery and performed internal tests of Document Agents to confirm that requests were completing successfully. The team maintained an observation period to ensure that service health remained stable before closing the incident. # Resolution Following Microsoft's remediation, the affected Azure services recovered and Document Agent functionality returned to normal. The engineering team confirmed through monitoring and internal testing that Document Agents were responding successfully and that the previously observed request failures had subsided. The incident was resolved at 9:03 PM CET on September 29, 2026, after service stability had been confirmed. # Impact and Scope The incident affected Document Agents for customers served by the following production clusters: * West Europe \(Production 0\) * West Europe \(Production 1\) * West Europe \(Production 5\) Document Agents depend on Azure OpenAI Service to function. Affected customers could experience Document Agents not responding, intermittent request failures, or increased latency. The customer impact was caused by the Microsoft Azure platform disruption affecting the supporting services in Sweden Central. No data loss was identified as part of this incident. # Post-Incident Actions This incident was caused by an upstream Microsoft Azure platform issue. Document Agents depend on Microsoft services, and recovery during this incident depended on Microsoft's remediation. We are reviewing options to improve resilience against regional failures affecting these critical cloud dependencies. In particular, we will evaluate additional regional failover capabilities for the services supporting Document Agents so that workloads can be redirected when an underlying Azure region becomes unavailable or degraded. These measures may help reduce customer impact from future disruptions, while Document Agents remain dependent on the availability of the supporting Microsoft services.