Multiple Regions - Integration Service -

Update timeline

1. monitoring Feb 27, 2026, 04:33 PM UTC

   Starting at 2/26/2026 1:30 PM UTC to 2/27/2026 3:40 PM UTC We experienced an incident affecting Integration Service connectors used by the AI Trust Layer “Bring Your Own” (BYO) feature. Customers using BYO LLM/LLM Configurations to power Agents, GenAI Activities, or other agentic products may have encountered workflow failures during the impacted period. The issue occurred because we were unable to retrieve required connection details at runtime, resulting in failed requests to customer-configured external models. The issue has been identified and resolved. Service is fully restored, though workflows that failed during the incident window may need to be re-run. We continue to monitor the system to ensure stability.

2. resolved Feb 27, 2026, 05:12 PM UTC

   Starting at 2/26/2026 1:30 PM UTC to 2/27/2026 3:40 PM UTC We experienced an incident affecting Integration Service connectors used by the AI Trust Layer “Bring Your Own” (BYO) feature. Customers using BYO LLM/LLM Configurations to power Agents, GenAI Activities, or other agentic products may have encountered workflow failures during the impacted period. The issue occurred because we were unable to retrieve required connection details at runtime, resulting in failed requests to customer-configured external models. The issue has been identified and resolved. Service is fully restored, though workflows that failed during the incident window may need to be re-run. We continue to monitor the system to ensure stability.

3. postmortem Mar 18, 2026, 08:00 AM UTC

   ## _Customer Impact_ Between February 26, 2026 at 1:30 pm UTC and February 27, 2026 at 3:40 pm UTC, few customers experienced workflow failures when using Bring Your Own \(BYO\) large language model \(LLM\) configurations. Workflows that connected to external AI providers, such as Vertex AI and other supported LLM services, returned HTTP 401 \(Unauthorized\) or HTTP 500 \(Internal Server Error\) responses and failed to run successfully. Any workflows that failed during this period needed to be manually re-run after the issue was resolved. ## _Root Cause_ A recent update to Integration Service introduced a serialization and deserialization issue in how requests to external LLM providers were processed. Because of this issue, some requests to external AI providers failed, which caused workflows using BYO LLM configurations to stop running successfully. ## _Detection_ The issue was discovered through monitoring that detected an increase in failed requests related to BYO LLM connections. Because the issue affected only certain workflows and external model connections, the overall service remained available, which delayed detection. ## _Response_ Once the issue was identified, engineers investigated the failures affecting BYO LLM connections. **February 27, 2026** * **Shortly after detection** — Engineers confirmed the issue and began reverting the recent change. * A fix was deployed to restore the previous behavior for configuration handling. * Service improvements were observed as error rates began to decrease. * Engineers validated the fix by confirming that workflows connecting to external LLM providers were running successfully. * **3:40 PM UTC** — Service recovery was confirmed and all affected functionality returned to normal. Throughout the incident, customer updates were shared on the status page and monitoring continued to ensure stability. ## _Follow-Up_ We are implementing improvements to reduce the likelihood of similar incidents. _Short-term improvements_ * Add additional validation before deployments that affect external AI integrations, including serialization and deserialization handling. * Improve monitoring so issues with external model connections are detected faster. _Long-term improvements_ * Expand testing for supported external AI providers such as Vertex AI, OpenAI, and Azure OpenAI. * Introduce gradual rollouts for changes affecting critical integrations to detect issues earlier.