UiPath Outage History

Started Aug 04, 2026, 07:14 PM UTC· resolved Aug 05, 2026, 03:39 AM UTC

1. investigating Aug 04, 2026, 07:14 PM UTC

   We are investigating reports of degraded performance impacting Agent/Maestro workflow runs that make LLM calls for Agents in the US region. Impact: Users may experience delayed or failed Agent/Maestro workflow runs when an LLM step is executed. Our teams are working to identify the cause and will share more details as the investigation progresses.

2. identified Aug 04, 2026, 07:19 PM UTC

   We have identified the issue causing degraded performance for Agent/Maestro workflow runs that make LLM calls for Agents in the US region. Impact: Users may continue to experience delayed or failed Agent/Maestro workflow runs when an LLM step is executed. Our teams are working on a fix and will share more details as remediation progresses.

3. monitoring Aug 04, 2026, 08:31 PM UTC

   We have implemented a fix for the issue causing degraded performance for Agent/Maestro workflow runs that make LLM calls for Agents in the US region. Impact: Users may have experienced delayed or failed Agent/Maestro workflow runs when an LLM step was executed. Our teams are monitoring the service to confirm continued stability and will share more details as needed.

4. monitoring Aug 04, 2026, 09:39 PM UTC

   We have implemented a fix for the issue causing degraded performance for Agent/Maestro workflow runs that make LLM calls for Agents in the US region. Impact: Users may have experienced delayed or failed Agent/Maestro workflow runs when an LLM step was executed. Our teams are continuing to monitor the fix for stability and will share more details as needed.

5. monitoring Aug 04, 2026, 10:53 PM UTC

   We have implemented a fix for the issue causing degraded performance for Agent/Maestro workflow runs that make LLM calls for Agents in the US region. Impact: Users may have experienced delayed or failed Agent/Maestro workflow runs when an LLM step was executed. Our teams are continuing to monitor the fix and confirm there is no further impact.

6. investigating Aug 05, 2026, 01:01 AM UTC

   We are continuing to investigate reports of degraded performance impacting Agent/Maestro workflow runs that make LLM calls for Agents in the US region. Impact: Users may experience delayed or failed Agent/Maestro workflow runs when an LLM step is executed. After applying mitigation steps and monitoring the service, we are still observing latency in some workflows. Our teams are investigating whether an upstream provider may be contributing to the continued latency and will share more details as the investigation progresses.

7. identified Aug 05, 2026, 02:10 AM UTC

   We have identified that the degraded performance impacting Agent/Maestro workflow runs that make LLM calls for Agents in the US region may be related to a third-party upstream dependency. Impact: Users may experience delayed or failed Agent/Maestro workflow runs when an LLM step is executed. Our teams are continuing to work with the upstream provider and will share more details as remediation progresses.

8. resolved Aug 05, 2026, 03:39 AM UTC

   The issue has been resolved and Agent/Maestro LLM workflow performance has returned to expected levels Impact: No ongoing user impact.

9. postmortem Aug 20, 2026, 05:46 PM UTC

   ## Customer impact Between August 4, 2026, at 3:25 AM UTC and August 5, 2026, at 3:39 AM UTC, a subset of customers in the US region experienced degraded performance affecting Agent and Maestro workflow runs. This issue was isolated to workflow steps executing Large Language Model (LLM) calls in tandem with PII masking. The event lasted approximately 24 hours. Affected users experienced delayed or failed executions, characterized by extended runtimes, workflows exceeding the five-minute threshold, and timeout errors. ## Root cause An upstream, third-party service powering our PII detection capabilities experienced a localized configuration anomaly within its US region. This restriction prevented the service from scaling dynamically to accommodate transaction volumes. The upstream provider verified the issue and applied a manual correction. The infrastructure constraint had caused requests involving PII detection and related LLM workflow steps to experience elevated latency or time out. ## Detection Initial support tickets regarding timeouts were received on August 4, 2026, at 3:25 AM UTC. Because failures were intermittent and a portion of requests continued to process successfully, an incident was not declared at that time. Following subsequent reports from multiple customers, a formal incident was declared at 6:27 PM UTC. Automated latency alerts did not trigger as intended, preventing proactive internal detection prior to customer escalation. ## Response At 7:14 PM UTC on August 4, 2026, we published a public status update acknowledging the degraded Agent and Maestro workflow performance in the US region. By 7:19 PM UTC, teams identified a probable cause and initiated remediation efforts. An initial mitigation patch was deployed at 8:31 PM UTC, followed by close performance monitoring; however, customer feedback at 8:41 PM UTC and 8:50 PM UTC indicated persistent latency. Continued investigation at 11:04 PM UTC confirmed ongoing elevated response times and timeout errors originating from the third-party service, prompting deeper escalation and joint investigation with the upstream cloud provider. At 2:06 AM UTC on August 5, 2026, the provider confirmed their scaling configuration issue was resolved. Telemetry indicated a downward trend in workflow durations by 2:22 AM UTC, with performance steadily normalizing over the subsequent three hours. The incident was officially closed at 3:39 AM UTC after Agent and Maestro metrics fully returned to baseline operational levels. ## Follow up - We are auditing our alerting thresholds to ensure subtle latency increases are captured and escalated earlier, including verification that throttling-related notifications route directly to on-call response teams. - We are evaluating high-capacity and automated scaling alternatives for the third-party service integration to mitigate future upstream infrastructure constraints. - We are refining our request management and rate-limiting policies to prevent disproportionate localized utilization from affecting concurrent workflows.

Read the full incident report →

Started Aug 03, 2026, 04:00 PM UTC· resolved Aug 03, 2026, 09:49 PM UTC

1. resolved Aug 07, 2026, 08:27 PM UTC

   Between August 3, 2026 at 16:00 UTC and August 5, 2026 at 21:49 UTC, a small number of Enterprise customer organizations in the EU region experienced missing data in AI Trust Layer prompt logs. The issue was introduced by a configuration change applied on August 3, 2026. The configuration was reverted on August 5, 2026, restoring connectivity between the affected environments and our service. We have continued to monitor the affected environments following the revert and have confirmed that the issue is fully mitigated. No customer action is required.

2. postmortem Aug 18, 2026, 11:37 PM UTC

   ## Customer impact Between approximately 4:00 pm UTC on August 3, 2026 and 9:24 pm UTC on August 5, 2026, a subset of customers in the EU region were unable to view large language model prompt audit logs in AI Trust Layer. The background save operation for prompt log records failed during this period, and those records were not stored. The impact lasted approximately 2 days, 5 hours, and 24 minutes. Running operations were not affected, but the missing prompt log data is unrecoverable. We sincerely apologize for the disruption and the loss of audit trail data during this window. ## Root cause The root cause was a routing configuration mismatch introduced during a service configuration migration. Two internal configuration sources — one controlling service routing and one serving as the service address registry — contained mismatched destination addresses for the affected EU deployment. This caused audit log upload requests to reach an endpoint that returned errors. Because these errors were not retried, the background save operation silently dropped prompt log records instead of storing them. ## Detection An automated alert then fired at 4:14 pm UTC on August 3, 2026 and the initial investigation began. The issue was escalated after a customer reported missing audit logs on August 5, 2026. The gap between detection and resolutions reflects a gap that will be reflected on in the follow up actions. ## Response At 7:14 pm UTC on August 5, 2026, an incident response channel and video call were opened to investigate the customer impact and routing behavior. By 8:30 pm UTC, the team identified that the service routing configuration did not match the service address registry. The plan was to correct the affected addresses and redeploy the configuration. At 8:34 pm UTC, the impact was confirmed as missing large language model prompt logs for AI Trust Layer in one EU region, with running operations unaffected. The corrected configuration was prepared and deployed. At 9:24 pm UTC, traffic was observed returning to the affected deployment after the update. The incident was marked mitigated at 9:49 pm UTC. ## Follow up 1. We are reviewing and updating our automated end-to-end monitoring to ensure that failures in prompt log capture are detected promptly after service updates in all regions. 2. We are conducting a detailed impact analysis to understand the full scope of affected customers and features during the impact window.

Read the full incident report →

Started Aug 03, 2026, 10:30 AM UTC· resolved Aug 03, 2026, 10:30 AM UTC

1. resolved Aug 03, 2026, 02:11 PM UTC

   Between 03-08-2026 07:47 UTC and 03-08-2026 11:24 UTC, organizations on the Delayed US and Delayed EU update rings were unable to deploy UiPath Solutions. The issue was introduced by a configuration change applied at 07:47 UTC. That change was reverted at 11:24 UTC, which restored Solutions deployment functionality for all affected organizations. We have monitored the affected environments since the revert and confirm the issue is fully mitigated. No customer action is required.

2. postmortem Sep 02, 2026, 02:56 PM UTC

   ## Customer impact Between 07:47 UTC and 11:24 UTC on August 3, 2026, organizations on the Delayed US and Delayed EU update rings were unable to use UiPath Solutions deployment functionality: the Solutions management pages were unavailable and deployment operations could not be performed. Running automations and all other Orchestrator functionality were not affected, and no data was lost. The total customer-impacting duration was approximately 3 hours and 37 minutes. No customer action was required after the fix was applied. ## Root cause As part of routine configuration hygiene, a set of feature-flag definitions that were no longer referenced by the latest Orchestrator release was removed from our centralized configuration system at 07:47 UTC on August 3, 2026. The removal was validated against the latest codebase only. Environments on the Delayed update rings intentionally run an earlier Orchestrator release, and that release still evaluated one of the removed flags to control access to Solutions management functionality. Once its definition was removed, the flag resolved to its default (disabled) value in those environments, which made the Solutions deployment experience unavailable. ## Detection The issue was brought to our attention through a customer report received at 10:15 UTC on August 3, 2026, approximately 2 hours and 28 minutes after the configuration change was applied; no automated alert detected the degradation. The formal incident was opened at 13:43 UTC, after the customer-impacting period had already ended. ## Response Engineering traced the failure to the configuration change and reverted it at 11:24 UTC, which immediately restored UiPath Solutions deployment functionality for all affected organizations. The affected environments were monitored after the revert, and the incident was marked resolved at 14:11 UTC once monitoring confirmed they remained healthy. ## Follow up 1. **Feature-flag evaluation telemetry** — We are adopting runtime usage telemetry for feature flags, so that cleanup decisions are driven by actual evaluation data from every environment and release version, rather than by static code inspection. 2. **Configuration deletion safeguard** — We are adding a safeguard to the centralized configuration platform that detects when a flag definition is still being consumed by any deployed service version and prevents its deletion until usage reaches zero.

Read the full incident report →

Started Jul 31, 2026, 07:43 AM UTC· resolved Jul 31, 2026, 08:13 AM UTC

1. identified Jul 31, 2026, 07:43 AM UTC

   We are investigating degraded performance impacting UiPath Apps authored from Studio Web service in Multiple Regions. Our teams are working to identify the cause and we will share more details as the investigation progresses.

2. monitoring Jul 31, 2026, 08:00 AM UTC

   We have a mitigation available and are beginning to roll it out to affected regions. We will provide more updates as the mitigation takes effect.

3. resolved Jul 31, 2026, 08:13 AM UTC

   The mitigation has been deployed across all affected regions and Apps performance has returned to normal. The issue is resolved.

4. postmortem Aug 28, 2026, 02:15 AM UTC

   ## Customer impact Between July 30, 2026 at 9:07 am UTC and July 31, 2026 at 8:13 am UTC, a subset of customers experienced degraded behavior in UiPath Apps authored from Studio Web. **Expressions in newly created apps failed to evaluate**, which could prevent those apps from working as expected. The impact affected a subset of**Studio Web customers in the Europe Region,**other regions, and services were not impacted. Existing deployed apps continued to work normally. ## Root cause The root cause was an internal software component update deployed the previous day. This update caused expression evaluation to fail for newly deployed UiPath Apps authored from Studio Web, producing an internal software error during expression processing. ## Detection The issue was detected through an automated alert for the Studio Web Apps service on July 31, 2026 at 7:29 am UTC. Customer impact was declared within two minutes of detection, initially assessed at multiple regions and after downgraded to only Community Region. ## Response At**7:32 am UTC**, the initial impact description was shared: expression evaluation was failing in Studio Web Apps, while older deployed apps continued to work. At **7:43 am UTC**, a public status update was posted noting that degraded performance was under investigation for UiPath Apps authored from Studio Web in multiple regions, later impact being downgraded to only Community Region. By **7:59 am UTC**, a corrective update to revert the software component change was prepared. At **8:00 am UTC**, the fix began rolling out to affected regions, and the incident was marked mitigated after 30 minutes and 36 seconds. At **8:13 am UTC**, the fix had been deployed across all affected regions, Apps performance had returned to normal, and the incident was marked resolved. ## Follow up - **Strengthen pre-production testing** so regressions are caught before reaching production. - **Enhance monitoring** to faster detect packaging/publishing issues that are resulted due to compiler.

Read the full incident report →

Started Jul 30, 2026, 10:44 PM UTC· resolved Jul 31, 2026, 01:28 AM UTC

1. investigating Jul 30, 2026, 10:44 PM UTC

   We are investigating reports of degraded performance impacting Extraction and Classifications for Document Understanding in US. Our teams are working to identify the cause and will share more details as the investigation progresses.

2. monitoring Jul 30, 2026, 11:48 PM UTC

   We have implemented a fix for the degraded performance impacting Extraction and Classification for Document Understanding in US. We are monitoring the results and will provide an update shortly.

3. monitoring Jul 31, 2026, 01:08 AM UTC

   We are continuing to monitor the fix. Performance and error rates have returned to normal levels. We will provide an update once we confirm full recovery.

4. resolved Jul 31, 2026, 01:28 AM UTC

   This incident has been resolved. Extraction and Classification for Document Understanding in US are operating normally.

5. postmortem Aug 06, 2026, 02:04 PM UTC

   ## Customer impact Between July 30, 2026 at 10:00 pm UTC and July 31, 2026 at 1:28 am UTC, a subset of customers using Helix models for Document Understanding in the US experienced degraded performance, including failed requests returning HTTP 503 errors and client-side timeouts. The impact lasted approximately 3 hours and 28 minutes. Some requests continued to process successfully, but affected customers may have seen retries, delayed processing, or failed document extraction and classification jobs. ## Root cause The root cause was duplicate re-processing of the same document pages during a period of high retries. Successful processing results were cached; when the queue was full, failed submissions were not cached. Repeated retries re-entered processing instead of being deduplicated. During the incident window, duplicate processing accounted for the majority of compute work, which drove sustained resource saturation and caused additional 503 responses - which increased load and led to additional 503 responses. ## Detection Automated alerting detected the first queue overflow errors began around 10:00 pm UTC. ## Response At 10:44 pm UTC, we posted an investigation notice for degraded Extraction and Classification performance in the US. By 11:26 pm UTC, we freed capacity by scaling down an unused processing path and freeing additional capacity for the affected Document Understanding workload. ## Follow up 1. We plan to implement short-lived failure caching for repeated document submissions, so that duplicate retries after a queue overflow are rejected early instead of re-entering expensive processing. 2. We are improving monitoring and alerting to detect this class of issue sooner.

Read the full incident report →

Started Jul 30, 2026, 04:42 AM UTC· resolved Jul 30, 2026, 12:30 PM UTC

1. investigating Jul 30, 2026, 04:42 AM UTC

   We are investigating degraded performance impacting Orchestrator Service in Japan and Australia Region. Our teams are working to identify the cause and we will share more details as the investigation progresses.

2. investigating Jul 30, 2026, 04:51 AM UTC

   We are investigating degraded performance impacting Insights Service in Japan and Australia Region. Our teams are working to identify the cause and we will share more details as the investigation progresses.

3. identified Jul 30, 2026, 05:14 AM UTC

   We've identified the cause of an issue affecting a subset of recently created queue items. No data has been lost, and we've confirmed all affected items can be fully recovered. We're validating a fix and will begin recovery shortly.

4. identified Jul 30, 2026, 06:22 AM UTC

   We've prepared a fix and are validating it before applying it to affected queue items. We'll post again once recovery is underway

5. identified Jul 30, 2026, 07:26 AM UTC

   Recovery is complete in Japan and we're validating the changes. We'll apply the same fix to the remaining regions and update as each completes.

6. identified Jul 30, 2026, 08:12 AM UTC

   The fix has been applied to all impacted regions and the affected items have been recovered. As a precaution, we're rolling the same fix out to remaining regions. No further customer impact is expected.

7. identified Jul 30, 2026, 09:21 AM UTC

   We're still rolling the same fix out to remaining regions as a precaution. No further customer impact is expected.

8. identified Jul 30, 2026, 10:33 AM UTC

   The precautionary rollout to remaining regions is still in progress.

9. monitoring Jul 30, 2026, 11:12 AM UTC

   The fix has been applied across all regions and we're monitoring to confirm full stability.

10. resolved Jul 30, 2026, 12:30 PM UTC

    The fix has been applied across all regions and all affected items have been recovered. Monitoring is complete and all systems are operating normally. The issue is resolved.

11. postmortem Jul 31, 2026, 01:45 PM UTC

    ## Customer impact Between July 30, 2026 at 4:32 am UTC and 8:12 am UTC \(approximately 3 hours and 40 minutes\), a subset of customers in the Asia-Pacific region - Japan, Australia, and Singapore - experienced degraded Insights behavior for recently created queue items. These items could not be processed by the Insights data pipeline because their folder association was missing. No data was lost, and all affected items were recovered. ## Root cause The root cause was a planned database maintenance intended to clean up folder references that pointed to deleted folders or classic folders. This work was meant to proactively fix recurring tenant-migration problems and reduce the remaining references to classic folders, a prerequisite for removing them. Unintentionally, the change left some active queue definitions without a folder association, and newly created queue items copied that missing association. As a result, the Insights data pipeline could not process those queue items until the associations were restored. ## Detection The issue was detected by automated alerting on July 30, 2026 at 4:32 am UTC, when an alert reported an extended pipeline failure in the Insights backend. Detection occurred at the same time the incident was opened. The public status page was updated at 4:42 am UTC. ## Response By 5:14 am UTC, analysis confirmed that the issue affected a subset of recently created queue items, that no data had been lost, and that all affected items could be recovered. At 6:17 am UTC, a recovery script was prepared to restore valid folder associations on queue definitions. At 6:22 am UTC, validation of the fix began before applying it to affected queue items. Recovery was applied region by region. Japan was fixed by 7:16 am UTC and confirmed recovered at 7:42 am UTC. Australia was completed at 7:49 am UTC, and all three impacted regions were reported resolved at 7:55 am UTC. By 8:12 am UTC, the fix had been applied to all impacted regions and affected items had been recovered. As a precaution, the same recovery was then applied to remaining regions where the cleanup had run. This precautionary work was completed by 11:00 am UTC, monitoring began at 11:12 am UTC, and the incident was resolved at 12:30 pm UTC after monitoring confirmed normal operation. ## Follow up Our goal is to ensure that routine database maintenance can no longer leave an active queue definition without a valid main folder association, and that any similar issue is detected and recovered before it affects customers. To achieve this, the following preventive actions are being tackled: 1. Add pre-execution validation and staged, region-by-region rollout to folder-cleanup maintenance, so active queue definitions always retain a valid main folder association. 2. Maintain a validated recovery procedure to quickly restore folder associations if a similar issue occurs.

Read the full incident report →

Started Jul 29, 2026, 11:54 AM UTC· resolved Jul 29, 2026, 02:04 PM UTC

1. investigating Jul 29, 2026, 11:54 AM UTC

   We are investigating reports of degraded performance impacting Extraction and Classifications for Document Understanding in Europe. Our teams are working to identify the cause and will share more details as the investigation progresses.

2. investigating Jul 29, 2026, 12:05 PM UTC

   We are continuing to investigate this issue.

3. monitoring Jul 29, 2026, 12:57 PM UTC

   We have applied the fix, we are currently monitoring the situation closely.

4. resolved Jul 29, 2026, 02:04 PM UTC

   The fix has been successfully applied, and our monitoring confirms that Extraction and Classifications for Document Understanding in Europe are operating normally. This incident has been resolved.

5. postmortem Jul 31, 2026, 10:39 AM UTC

   ## Customer impact Between July 22 and July 29, 2026, customers using Document Understanding in the Europe region experienced intermittent delays and, in a limited number of cases, failed document operations. The issue affected document extraction and splitting/classification operations using Helix 2.0-based models— other model types were not affected. The impact occurred in three windows, each detected by our automated monitoring: * July 22, approximately 5:10 pm to 7:39 pm UTC * July 24, approximately 7:45 am to 3:01 pm UTC * July 29, approximately 10:55 am to 2:05 pm UTC During these windows, operations that normally complete in under a minute could take several minutes, and a small number of operations failed after all retry attempts. Between the first two windows, a low rate of intermittent delays persisted. The issue was isolated to the Europe region and did not affect other geographies. ## Root cause The root cause was a concurrency issue in the inference service that runs these models. Two stages of processing—preparing incoming requests and generating results—shared an internal component that only one operation can use at a time. Under high-load, operations queued for this shared component, slowing result generation even though compute capacity was available. Slow operations kept their processing slots occupied, so new requests queued behind them and some were rejected, and a small number of operations ultimately failed after exhausting all retry attempts. Large, image-heavy documents increased use of the shared component and made the slowdown worse. Retried requests for the same document are automatically deduplicated, so retries did not multiply the processing work; however, rejected requests did not tell clients when to retry, which added some pressure during the constrained periods. The July 29 recurrence happened because a protective configuration change applied after the earlier occurrences was unintentionally reverted during a routine deployment. It was re-applied the same day and has now been made permanent. ## Detection Automated alerts detected the initial issue at 5:14 pm UTC on July 22, approximately four minutes after failures began to rise. Monitoring showed elevated failed requests and longer processing times. After the first window was resolved, our engineering team continued to track a low rate of intermittent delays through end-to-end tests and telemetry reviews while the investigation continued. A new automated alert detected the July 24 recurrence, and the July 29 recurrence was detected by automated alerts within minutes of onset. ## Response On July 22, automated capacity scaling restored throughput and the failure rate returned to near-baseline levels. The status page was updated at 5:45 pm UTC, and after continued monitoring showed no further customer-visible failures, the incident was closed at 7:39 pm UTC. After closure, our engineering team continued investigating the underlying cause—that investigation was still in progress when the issue recurred on July 24. On July 24, following the recurrence, our engineering team restarted the affected inference workers and applied a protective configuration change that reduces concurrent processing per worker and increases minimum capacity. The status page was updated at 9:22 am UTC, and the incident was resolved at 3:01 pm UTC after extended monitoring. On July 29, engineers identified that the protective configuration had been unintentionally reverted, re-applied it, and restarted the affected workers. The status page was updated at 11:54 am UTC. Service was confirmed stable—including by affected customers—and the incident was resolved at 2:05 pm UTC. ## Follow-up We are taking the following steps to prevent recurrence and improve resilience: 1. **Retry handling:** Rejected requests now include explicit retry-timing guidance so that clients back off appropriately during constrained periods. Requests for the same document remain automatically deduplicated, so retries do not multiply processing work. 2. **Contention fix:** We have completed a code change that removes the internal contention that caused slow processing under high loads. It is being rolled out and validated under production traffic. 3. **Configuration safeguards:** The protective configuration has been made permanent in our deployment configuration, with change controls in place so that it cannot be unintentionally reverted. 4. **Observability and alerting:** We are improving monitoring and alerting to detect this class of issue sooner, and tightening incident-closure criteria so that incidents are not closed while residual degradation remains.

Read the full incident report →

Started Jul 24, 2026, 07:00 PM UTC· resolved Jul 24, 2026, 07:00 PM UTC

1. resolved Aug 05, 2026, 11:59 AM UTC

   From 24 July at 19:00 UTC, customers in the US and EU region experienced issues receiving Digest and Summary emails from the UiPath Notification Service. The issue has now been resolved, and email delivery has been restored. We are continuing to monitor the service to ensure stable delivery.

2. postmortem Aug 13, 2026, 11:13 AM UTC

   ## Customer impact Between July 24, 2026 19:00 UTC and August 6, 2026 23:00 UTC, customers in the US region did not receive Digest and Summary emails from the UiPath Notification Service. Customers in the Europe region were similarly affected between July 30, 2026 12:00 UTC and August 5, 2026 11:59 UTC. These emails deliver scheduled digests of product notifications on each customer's configured cadence. No other regions or Notification Service functionality were affected. No notifications were lost: all notifications remained available in the in-app notification center throughout the incident, and the emails accumulated during the impact window were delivered after the fix. No customer action was or is required. ## Root cause The background jobs that assemble Digest and Summary emails were processed through a single, serialized execution lane in each region, with limited parallelism inside each run. When one customer's unusually large volume of notifications entered the pipeline, processing it occupied that lane for extended periods and repeatedly timed out — blocking scheduled email runs for all customers in the region. The differing onset dates (July 24 in the US, July 30 in Europe) reflect when each region's load crossed this threshold. ## Detection The issue was reported through a customer support ticket. No automated alert covered digest email delivery volume, which is why the failure went undetected for this period; closing this monitoring gap is the primary preventive follow-up from this incident. ## Response After the issue was acknowledged on August 5, 2026, we confirmed that Digest and Summary emails were not being delivered in the US region and, upon checking all regions, Europe was also affected. All other regions were verified healthy. A fix was deployed on August 5, 2026 to the US region by 07:45 UTC and to the Europe region by 09:45 UTC, and email delivery was verified in both, though delays remained while the large accumulated backlog drained. The fix removed the serialization so scheduled email runs execute independently, and follow-up improvements increased processing parallelism and throughput to accelerate backlog delivery. The accumulated emails from the impact window were then delivered: most schedules caught up within hours, and daily digest delivery completed over the following day. The incident was resolved on August 5 at 11:59 UTC, and we continued monitoring delivery and confirmed resolution with affected customers. ## Follow up 1. **Detection (deployed)** — We are adding automated alerting on digest email delivery volume in every region, so that any interruption in delivery is detected within minutes rather than through customer reports. 2. **Recurrence prevention (deployed)** — The serialization defect was corrected, processing parallelism was made tunable per region, and the affected schedule now runs with a tightened execution timeout and automatic retry so a single long or failing run cannot stall subsequent runs. 3. **Architectural remediation (in progress)** — We are re-architecting the batch email pipeline to separate email scheduling from email delivery to avoid noisy-neighbor issues, and to add per-schedule health visibility.

Read the full incident report →

Started Jul 24, 2026, 09:22 AM UTC· resolved Jul 24, 2026, 03:01 PM UTC

1. investigating Jul 24, 2026, 09:22 AM UTC

   We are investigating degraded performance impacting Document Understanding Service in Europe Region. Our teams are working to identify the cause and we will share more details as the investigation progresses.

2. investigating Jul 24, 2026, 09:41 AM UTC

   We are investigating degraded performance affecting Document Understanding Helix 2.0. The extractor and splitter/classifier are experiencing increased delay and occasional failed requests in the Europe region

3. monitoring Jul 24, 2026, 10:42 AM UTC

   Our engineering team has applied a mitigation to address the issue. Since implementing the mitigation, we have observed a significant reduction in errors, and the affected service is returning to a stable state. We are continuing to closely monitor the service to ensure stability and will provide further updates.

4. resolved Jul 24, 2026, 03:01 PM UTC

   The mitigation has been effective, and all affected services are now operating normally. Service health is stable, and error rates have returned to expected levels. We apologize for the disruption and appreciate your patience throughout this incident.

5. postmortem Jul 31, 2026, 10:38 AM UTC

   ## Customer impact Between July 22 and July 29, 2026, customers using Document Understanding in the Europe region experienced intermittent delays and, in a limited number of cases, failed document operations. The issue affected document extraction and splitting/classification operations using Helix 2.0-based models— other model types were not affected. The impact occurred in three windows, each detected by our automated monitoring: * July 22, approximately 5:10 pm to 7:39 pm UTC * July 24, approximately 7:45 am to 3:01 pm UTC * July 29, approximately 10:55 am to 2:05 pm UTC During these windows, operations that normally complete in under a minute could take several minutes, and a small number of operations failed after all retry attempts. Between the first two windows, a low rate of intermittent delays persisted. The issue was isolated to the Europe region and did not affect other geographies. ## Root cause The root cause was a concurrency issue in the inference service that runs these models. Two stages of processing—preparing incoming requests and generating results—shared an internal component that only one operation can use at a time. Under high-load, operations queued for this shared component, slowing result generation even though compute capacity was available. Slow operations kept their processing slots occupied, so new requests queued behind them and some were rejected, and a small number of operations ultimately failed after exhausting all retry attempts. Large, image-heavy documents increased use of the shared component and made the slowdown worse. Retried requests for the same document are automatically deduplicated, so retries did not multiply the processing work; however, rejected requests did not tell clients when to retry, which added some pressure during the constrained periods. The July 29 recurrence happened because a protective configuration change applied after the earlier occurrences was unintentionally reverted during a routine deployment. It was re-applied the same day and has now been made permanent. ## Detection Automated alerts detected the initial issue at 5:14 pm UTC on July 22, approximately four minutes after failures began to rise. Monitoring showed elevated failed requests and longer processing times. After the first window was resolved, our engineering team continued to track a low rate of intermittent delays through end-to-end tests and telemetry reviews while the investigation continued. A new automated alert detected the July 24 recurrence, and the July 29 recurrence was detected by automated alerts within minutes of onset. ## Response On July 22, automated capacity scaling restored throughput and the failure rate returned to near-baseline levels. The status page was updated at 5:45 pm UTC, and after continued monitoring showed no further customer-visible failures, the incident was closed at 7:39 pm UTC. After closure, our engineering team continued investigating the underlying cause—that investigation was still in progress when the issue recurred on July 24. On July 24, following the recurrence, our engineering team restarted the affected inference workers and applied a protective configuration change that reduces concurrent processing per worker and increases minimum capacity. The status page was updated at 9:22 am UTC, and the incident was resolved at 3:01 pm UTC after extended monitoring. On July 29, engineers identified that the protective configuration had been unintentionally reverted, re-applied it, and restarted the affected workers. The status page was updated at 11:54 am UTC. Service was confirmed stable—including by affected customers—and the incident was resolved at 2:05 pm UTC. ## Follow-up We are taking the following steps to prevent recurrence and improve resilience: 1. **Retry handling:** Rejected requests now include explicit retry-timing guidance so that clients back off appropriately during constrained periods. Requests for the same document remain automatically deduplicated, so retries do not multiply processing work. 2. **Contention fix:** We have completed a code change that removes the internal contention that caused slow processing under high loads. It is being rolled out and validated under production traffic. 3. **Configuration safeguards:** The protective configuration has been made permanent in our deployment configuration, with change controls in place so that it cannot be unintentionally reverted. 4. **Observability and alerting:** We are improving monitoring and alerting to detect this class of issue sooner, and tightening incident-closure criteria so that incidents are not closed while residual degradation remains.

Read the full incident report →

Started Jul 21, 2026, 02:18 PM UTC· resolved Jul 22, 2026, 04:17 AM UTC

1. investigating Jul 21, 2026, 02:18 PM UTC

   We are investigating reports of missing robot logs in the US region. Our teams are working to identify the cause and will share more details as the investigation progresses.

2. investigating Jul 21, 2026, 02:37 PM UTC

   We are investigating reports of missing robot logs in the US region. Our teams are working to identify the cause and will share more details as the investigation progresses.

3. monitoring Jul 21, 2026, 03:00 PM UTC

   Mitigation has been applied. We are monitoring closely to ensure stability.

4. investigating Jul 21, 2026, 04:07 PM UTC

   We are still investigating the reports of delayed ingestion of robot logs. Our teams are working to identify the cause and will share more details as the investigation progresses.

5. identified Jul 21, 2026, 05:05 PM UTC

   We have identified and applied new mitigation and recovery is estimated to take several more hours.

6. identified Jul 21, 2026, 06:12 PM UTC

   We have identified and applied new mitigation and recovery is estimated to take several more hours.

7. identified Jul 21, 2026, 06:24 PM UTC

   We have identified and applied new mitigation and recovery is estimated to take several more hours.

8. identified Jul 21, 2026, 06:35 PM UTC

   We have identified impact in the US Delayed environment as well, and are continuing mitigation activities.

9. identified Jul 21, 2026, 07:38 PM UTC

   We are continuing mitigation activities, the delayed US region is showing improvements.

10. identified Jul 21, 2026, 08:54 PM UTC

    We are continuing mitigation activities, the delayed US region is showing improvements.

11. identified Jul 21, 2026, 09:55 PM UTC

    We are continuing mitigation activities, the delayed US region is showing improvements.

12. identified Jul 21, 2026, 11:14 PM UTC

    We are continuing mitigation activities, the delayed US region is showing improvements.

13. identified Jul 22, 2026, 12:25 AM UTC

    We are continuing mitigation activities, the delayed US region is showing improvements.

14. identified Jul 22, 2026, 01:38 AM UTC

    We are continuing mitigation activities, the delayed US region is showing improvements.

15. identified Jul 22, 2026, 02:41 AM UTC

    We are continuing mitigation activities, the delayed US region is showing improvements.

16. identified Jul 22, 2026, 03:58 AM UTC

    We are continuing mitigation activities, the delayed US region is showing improvements.

17. identified Jul 22, 2026, 04:15 AM UTC

    We are continuing mitigation activities and the impacted regions are showing improvements.

18. resolved Jul 22, 2026, 04:17 AM UTC

    Our mitigation efforts are completed and impacted services are fully stable now. We will post a detailed summary of incident soon.

19. postmortem Aug 05, 2026, 07:42 AM UTC

    ## Customer impact Between July 21, 2026 at 1:58 pm UTC and July 22, 2026 at approximately 1:00 am UTC, Orchestrator robot job logs in the US and Delayed US region were delayed. Customers could run jobs, but recent robot logs were not immediately available in the Orchestrator interface, affecting real-time monitoring and troubleshooting. No data was lost. Delayed logs continued to be processed and became visible once the system caught up. The customer-facing impact lasted approximately 11 hours. ## Root cause A server in the search infrastructure that Orchestrator uses to store robot logs became unavailable and could not restart automatically because its access credentials had expired. Once the server was manually restored, the search system began redistributing data across the remaining servers. This redistribution consumed significant system capacity and slowed the processing of new robot logs, creating a backlog before logs became visible in Orchestrator. Usually, adding a server to the cluster does not cause processing constraints, but in this instance, the system proceeded to redistribute about 30% of the data between servers, instead of populating only the newly added server. ## Detection An automated service alert fired at 1:58 pm UTC on July 21, 2026, and an incident was opened for investigation. Customer reports quickly confirmed the impact, with delayed or missing robot logs observed across multiple accounts in the US region. ## Response At 2:37 pm UTC on July 21, we posted an update confirming we were investigating reports of missing robot logs in the US region. By 3:07 pm UTC, the unavailable server had been restored, and by 3:20 pm UTC, event processing capacity was increased as an additional mitigation. Despite these measures, log delays persisted. At 6:08 pm UTC, our engineering team adjusted search system settings to reduce the load from writing newly processed logs. At 6:29 pm UTC, the team moved the most active log data off an overburdened server, and at 6:53 pm UTC, internal data redistribution was temporarily paused because it was competing with log processing recovery. By 7:11 pm UTC, all available mitigations had been applied and the remaining recovery depended on the system working through the backlog. Logs caught up around 1:00 am UTC on July 22, 2026, and the incident was marked resolved at 4:17 am UTC after continued monitoring confirmed stability. ## Follow up 1. Optimize the data rebalancing limits so that they do not impact ingestion of new data. 2. Increase spare capacity in the cluster, so that even massive data reshuffling does not cause processing constraints. 3. Replace the expiring access credential mechanism and apply the corrected configuration across all search infrastructure to prevent recurrence. 4. Improve automated alerting so that an unavailable search server that is not recovered within a short period triggers a higher-severity alert for faster response.

Read the full incident report →

Started Jul 20, 2026, 07:16 PM UTC· resolved Jul 20, 2026, 07:32 PM UTC

1. identified Jul 20, 2026, 07:16 PM UTC

   We have identified the cause of the degraded performance impacting [impacted functionality / user symptom] for [Service] in [Region] and are working on mitigation. Impact: Users may experience [describe ongoing impact, if any]. Additional updates will be provided as we move toward resolution.

2. identified Jul 20, 2026, 07:18 PM UTC

   The issue causing degraded performance impacting Jobs for Orchestrator in US has been identified, and corrective actions are underway. Impact: Users may continue to see delays when executing jobs. We will provide further updates as mitigation progresses.

3. resolved Jul 20, 2026, 07:32 PM UTC

   The issue has been resolved and Orchestratorperformance has returned to expected levels after degraded performance impacted jobs in US. Impact: No ongoing user impact.

4. postmortem Jul 23, 2026, 07:59 AM UTC

   ## Customer impact Between **July 20, 2026 at 4:28 pm UTC and 7:32 pm UTC**, a subset of customers experienced degraded Orchestrator Jobs performance in the **U.S. region**, with delays in job processing during that window. ## Root cause The incident was triggered by a routine product deployment to Orchestrator that included a schema change to an existing column on a core Orchestrator database table. Specifically, a change to the column's size. The large size of the Jobs table made this schema change operation significantly more resource-intensive and longer-running than anticipated, causing it to still be in progress once normal U.S. traffic picked up, an overlap that had not been anticipated. The combination of the ongoing schema change and concurrent production traffic generated increased database load, which caused latency and delays in job processing. As the change rolled out, the affected databases were impacted progressively rather than all at once. The script had been reviewed and tested prior to rollout. However, the resource impact observed here did not reproduce during pre-deployment testing, as it depended on production-scale traffic conditions that were not present in the test environment. ## Detection Our automated monitoring identified the issue at 4:28 pm UTC on July 20, 2026, and the alert was acknowledged immediately. As the extent of the customer impact became clear, the incident was formally declared customer-impacting at 7:12 pm UTC. ## Response As the schema change progressed, the resulting database load increased progressively across the affected databases rather than all at once. The team mitigated the issue in two stages: a first round of database capacity increases around 5:00–5:30 pm UTC addressed the databases affected at that point, and as impact continued to build on the remaining databases, a second round of capacity increases around 7:00–7:30 pm UTC was applied, fully resolving the elevated load. At 7:19 pm UTC, a public status update confirmed the cause had been identified: degraded performance affecting Orchestrator Jobs in the U.S. region, with users potentially seeing delays when executing jobs. Mitigation actions were already under way at that point. The database schema change script was stopped, and database capacity was increased as a mitigation measure. During the response call, the engineering team monitored database utilization, which was observed dropping and then stabilizing. By the end of the call, all databases were underutilized and the issue was confirmed mitigated. At 7:32 pm UTC, the incident was marked resolved, and the public status page confirmed that Orchestrator performance had returned to expected levels ## Follow-up Several actions will be taken following this incident: * We are building a safer, more automated process for running database schema changes that reliably schedules resource-intensive changes outside of peak traffic hours, rather than relying on manual timing. * We are extending our monitoring systems to more accurately include customer-visible slowdowns in parts of the platform that rely on Orchestrator \(Maestro, Agents\).

Read the full incident report →

Started Jul 20, 2026, 03:18 PM UTC· resolved Jul 20, 2026, 03:18 PM UTC

1. resolved Jul 20, 2026, 03:18 PM UTC

   Customers on the UiPath Community plan hosted in the North Europe region may have experienced UiPath Orchestrator service degradation between 14:22 and 14:39 UTC on 20 July 2026. During this period the region's Orchestrator database reached 100% worker utilization, causing a portion of requests to fail or respond slowly until utilization recovered. Client side retries generally succeeded, no data loss occurred, and the issue is now fully resolved.

2. postmortem Aug 10, 2026, 08:32 AM UTC

   ## Customer impact Between July 20, 2026 at 2:22 pm UTC and July 20, 2026 at 2:39 pm UTC, a subset of some users of UiPath Orchestrator in Europe region experienced service degradation lasting approximately 17 minutes. During this window, some Orchestrator requests failed or responded slowly. Automatic retries from client applications generally succeeded, and no data was lost. ## Root cause During a scheduled Orchestrator deployment, a required database preparation step had not been effectively applied in Europe region. This preparation step was designed to make a subsequent database change safe and non-disruptive. Because it was missing, the deployment triggered a blocking database change on a heavily used table during peak traffic. This blocking operation prevented other database queries from completing, which exhausted available database processing capacity and caused Orchestrator requests to fail or slow down until the operation completed and capacity recovered. ## Detection Automated alerting detected the elevated failure rate at 2:30 pm UTC on July 20, 2026, approximately 8 minutes after impact began. ## Response Our engineering team began investigating the elevated failure rate as the automated alerts came in. A formal incident response channel was opened at 2:53 pm UTC and, within one minute, the team confirmed that the service had already recovered and that customers had been affected. At 3:10 pm UTC, we drafted a customer-facing impact summary confirming the degradation window of 2:22 pm to 2:39 pm UTC, intermittent request failures, successful retries in most cases, and no data loss. At 3:18 pm UTC, we updated the public status page to reflect the incident as resolved. We closed the incident at 3:35 pm UTC. ## Follow up 1. Reapply the required database preparation step using the safe, non-blocking method before resuming the deployment in the affected environment. 2. Add validation to our deployment process so that prerequisite steps are independently confirmed as completed in the target environment before a deployment proceeds — rather than relying solely on task-tracking status.

Read the full incident report →

Started Jul 18, 2026, 08:03 AM UTC· resolved Jul 18, 2026, 08:30 AM UTC

1. monitoring Jul 18, 2026, 08:03 AM UTC

   We have identified the cause of the degraded performance impacting Evalutions for Agents in India region and we have applied a mitigation and are monitoring the situation. Impact: Users may experience timeouts when running evaluations. Additional updates will be provided We have identified the cause of the degraded performance affecting evaluations for agents in the India region. We have implemented mitigation measures and are monitoring the situation. Impact: Users may experience timeouts when running evaluations. We will provide additional updates as we work toward a resolution.as we move toward resolution.

2. resolved Jul 18, 2026, 08:30 AM UTC

   We have identified and resolved the cause of the degraded performance affecting Agent Evaluations in the India region. Impact: Users may have experienced timeouts when running evaluations. This incident is now resolved. No further updates will be provided.

3. postmortem Jul 29, 2026, 11:43 AM UTC

   ## Customer impact On July 18, 2026, customers in the India region may have experienced degraded performance, timeouts, or failures when using Agent Evaluations and Health Score. Some published Conversational Agents may also have been unable to initialize conversations. Serverless Robot jobs may have started or executed more slowly than normal. ## Impact timeline All times below are on July 18, 2026, in UTC. | Time | Affected services | Customer impact | | --- | --- | --- | | Approximately 07:00–07:09 | Agent Builder | Customers may have experienced degraded performance, timeouts, or failures in Agent Evaluations and Health Score. Some published Conversational Agents may also have been unable to initialize conversations. | | 07:09–08:30 | Agent Builder and Serverless Robots | The Agent Builder impact continued. Serverless Robot jobs may also have started or executed more slowly than normal. | | 08:30–09:06 | Serverless Robots | Agent Builder service had recovered. Serverless Robot jobs may still have started or executed more slowly than normal until service was fully restored. | ## Root cause An unusually large traffic spike in the India region exceeded the available service capacity for Agent Builder and Serverless Robots. ## Detection The issue was detected through automated service alerts. Our engineering teams began investigating and published customer notifications for the affected Agent Builder and Serverless Robot services. ## Response We restored service by increasing regional capacity, controlling the abnormal workload, and monitoring the affected features until performance stabilized. ## Follow up We are improving capacity safeguards, workload controls, isolation, and the performance of affected request paths to reduce the likelihood and customer impact of similar traffic spikes.

Read the full incident report →

Started Jul 18, 2026, 07:18 AM UTC· resolved Jul 18, 2026, 09:06 AM UTC

1. investigating Jul 18, 2026, 07:18 AM UTC

   We are Investigating the cause of the degraded performance impacting Job Execution for Serverless Robots in India. Impact: Users may experience slow job execution. Additional updates will be provided as we move toward resolution.

2. identified Jul 18, 2026, 08:05 AM UTC

   We have identified the issue and our teams has started working on applying the fix.

3. monitoring Jul 18, 2026, 08:57 AM UTC

   A fix has been applied and we are now monitoring the results. Job execution performance for Serverless Robots in India has returned to normal levels. We will continue to monitor the situation

4. resolved Jul 18, 2026, 09:06 AM UTC

   This incident has been resolved. Job execution performance for Serverless Robots in India has returned to normal levels and remained stable through the monitoring period.

5. postmortem Jul 29, 2026, 11:43 AM UTC

   ## Customer impact On July 18, 2026, customers in the India region may have experienced degraded performance, timeouts, or failures when using Agent Evaluations and Health Score. Some published Conversational Agents may also have been unable to initialize conversations. Serverless Robot jobs may have started or executed more slowly than normal. ## Impact timeline All times below are on July 18, 2026, in UTC. | Time | Affected services | Customer impact | | --- | --- | --- | | Approximately 07:00–07:09 | Agent Builder | Customers may have experienced degraded performance, timeouts, or failures in Agent Evaluations and Health Score. Some published Conversational Agents may also have been unable to initialize conversations. | | 07:09–08:30 | Agent Builder and Serverless Robots | The Agent Builder impact continued. Serverless Robot jobs may also have started or executed more slowly than normal. | | 08:30–09:06 | Serverless Robots | Agent Builder service had recovered. Serverless Robot jobs may still have started or executed more slowly than normal until service was fully restored. | ## Root cause An unusually large traffic spike in the India region exceeded the available service capacity for Agent Builder and Serverless Robots. ## Detection The issue was detected through automated service alerts. Our engineering teams began investigating and published customer notifications for the affected Agent Builder and Serverless Robot services. ## Response We restored service by increasing regional capacity, controlling the abnormal workload, and monitoring the affected features until performance stabilized. ## Follow up We are improving capacity safeguards, workload controls, isolation, and the performance of affected request paths to reduce the likelihood and customer impact of similar traffic spikes.

Read the full incident report →

Started Jul 17, 2026, 09:24 PM UTC· resolved Jul 17, 2026, 11:27 PM UTC

1. identified Jul 17, 2026, 09:24 PM UTC

   The issue causing degraded performance impacting Job execution for Serverless in Australia has been identified, and corrective actions are underway. Impact: Users may continue to see delays when starting Jobs. We will provide further updates as mitigation progresses.

2. monitoring Jul 17, 2026, 11:00 PM UTC

   The issue has been mitigated. We are continuing to monitor job execution.

3. resolved Jul 17, 2026, 11:27 PM UTC

   The issue has been resolved, and job execution is functioning normally.

4. postmortem Jul 27, 2026, 07:14 PM UTC

   ## Customer impact Between July 17, 2026 at 9:18 pm UTC and July 17, 2026 at 11:27 pm UTC, a subset of customers in Australia experienced delays when starting serverless jobs. Jobs were expected to fall back to an alternate execution path, but that fallback added startup latency while the platform detected the failed start and retried execution. The incident lasted 2 hours and 8 minutes, with mitigation reached at July 17, 2026 at 11:00 pm UTC and full recovery confirmed at July 17, 2026 at 11:27 pm UTC. ## Root cause Our cloud service provider introduced new CPU hardware \(a new processor model\) into the virtual machine family used to run serverless job execution in Australia. The virtualization runtime we use to isolate job workloads was incompatible with the new hardware, so job execution environments did not start successfully on newly provisioned capacity. Because those environments could not start on the affected resources, serverless jobs were retried through a fallback execution path. That fallback helped jobs continue running, but it introduced delays while the platform recognized the startup failure and redirected execution. The same issue had been observed in the United States region the previous day, which enabled the team to identify the cause quickly and deploy a fix that had already been developed. ## Detection The issue was detected through automated alerting for the serverless job execution service on July 17, 2026 at 9:18 pm UTC. Our engineering team began investigating immediately. The issue was confirmed as customer-impacting on July 17, 2026 at 9:19 pm UTC. ## Response On July 17, 2026 at 9:24 pm UTC, we published a customer status update confirming that the cause of degraded serverless job execution performance had been identified and that corrective actions were underway. As an immediate step, the team removed the affected compute resources to attempt to obtain compatible replacements. In parallel, our engineering team deployed a fix to address the compute resource incompatibility. By July 17, 2026 at 10:04 pm UTC, the fix was being deployed and was expected to complete shortly. After the fix was applied and verified, we moved the incident to monitoring on July 17, 2026 at 11:00 pm UTC and continued validating job execution. On July 17, 2026 at 11:27 pm UTC, we confirmed that job execution was functioning normally and marked the incident resolved. ## Follow-up 1. Deploy a validation component that verifies newly provisioned compute capacity can run job workloads before customer jobs are scheduled onto it. \(Completed\) 2. Update the job isolation runtime so workloads start correctly on the new processor generation, removing the incompatibility. \(In progress\) 3. Engage the cloud service provider regarding the introduction of new CPU hardware into the requested virtual machine family, to improve advance visibility of hardware changes that can affect workload compatibility.

Read the full incident report →

Started Jul 16, 2026, 05:03 PM UTC· resolved Jul 16, 2026, 06:45 PM UTC

1. investigating Jul 16, 2026, 05:03 PM UTC

   We are currently investigating an issue affecting Coded Apps. Initial findings indicate the issue may be related to our cloud provider's services. We will provide updates as more information becomes available.

2. monitoring Jul 16, 2026, 06:00 PM UTC

   The issue with the cloud service provider has recovered. We are continuing to monitor the service for any further issues.

3. resolved Jul 16, 2026, 06:45 PM UTC

   The issue is now resolved and no new errors since last 30 minutes. Coded Apps across all geo's are loading as expected.

4. postmortem Jul 23, 2026, 06:34 AM UTC

   ## Customer impact Between approximately **July 16, 2026 at 4:39 pm UTC and July 16, 2026 at 6:00 pm UTC**, some customers experienced issues loading their UiPath Coded Apps hosted on `*.uipath.host` domain. ## Root cause The issue was caused by a service disruption at one of our cloud infrastructure providers, which caused a component UiPath Coded Apps depends on to become temporarily unavailable. Affected users could not load their apps during the disruption. Service recovered automatically once the provider's service was restored. ## Detection The issue was detected through automated service availability alerts for UiPath Coded Apps. ## Response After detecting the alert at 4:39 pm UTC on July 16, 2026, our team identified the root cause as an external outage at our cloud infrastructure provider. We updated the public status page as investigation progressed \(investigating at 5:03 pm UTC\). Since the root cause was external to UiPath and required recovery on the provider side, our response focused on monitoring the provider's recovery and communicating clearly to affected customers. At 6:00 pm UTC on July 16, 2026, the provider's service recovered and Coded Apps were observed to be loading normally again. The public status page was updated to monitoring at that time. At 6:45 pm UTC, the incident was marked resolved after no new errors had been observed for the prior 30 minutes. The final status update confirmed that Coded Apps across all geographies were loading as expected. ## Follow-up To reduce the chance of a similar issue and to improve our resilience to provider-side disruptions, we are evaluating architectural changes to Coded Apps so that a service disruption at any single infrastructure provider does not cause the same class of failure. This includes reviewing alternative storage approaches, fallback strategies, and consistency trade-offs.

Read the full incident report →

Started Jul 16, 2026, 10:36 AM UTC· resolved Jul 16, 2026, 11:07 AM UTC

1. identified Jul 16, 2026, 10:36 AM UTC

   We are investigating an issue causing ingestion failures in Process Mining across multiple regions. The root cause has been identified, and our engineering teams are actively working on mitigation and service restoration. Customers may experience failures in data ingestion into Process Mining during this time.

2. identified Jul 16, 2026, 11:02 AM UTC

   We have applied a mitigation in the Europe, United States, Japan and Australia regions, restoring Process Mining ingestion capabilities in these locations. Also we are actively working to restore functionality in the remaining affected regions

3. resolved Jul 16, 2026, 11:07 AM UTC

   The mitigation has been successfully applied across all affected regions and Process Mining ingestion functionality has been fully restored.The issue has been resolved.

4. postmortem Jul 31, 2026, 08:28 AM UTC

   ## **Customer Impact** Between **July 16, 2026 at 10:20 UTC** and **July 16, 2026 at 11:07 UTC**, a subset of customers experienced failures when ingesting new data into **Process Mining**. Existing Process Mining data remained available and was not affected; the impact was limited to new data ingestion requests. The issue affected Process Mining data ingestion across multiple regions and was treated as a global customer-impacting event. ## **Root Cause** The incident was caused by a storage account secret rotation that interrupted access between the Process Mining ingestion services and their underlying storage accounts. Although the storage account secrets were rotated successfully, the updated credentials were not immediately applied to the dependent services. As a result, new data ingestion requests failed until the affected services were restarted and began using the updated credentials. ## **Detection** The issue was detected through automated monitoring at **10:20 UTC** on **July 16, 2026**, when an alert was triggered for the Process Mining backend service. The incident was declared customer-impacting one minute later at **10:21 UTC**. Because automated monitoring detected the issue immediately, there was effectively no gap between the start of customer impact and detection. ## **Response** Following detection, engineering confirmed that new Process Mining data ingestions were failing across multiple regions and initiated incident response. Customers were informed through the public status page while the investigation was underway. After identifying the storage secret rotation as the root cause, engineering restarted the affected Process Mining services so they could begin using the updated credentials. Service was restored region by region as the affected services recovered, and once ingestion functionality was confirmed across all affected regions, the incident was resolved. ## **Follow-Up** To help prevent similar incidents, we are implementing the following improvement: * **Automatic service restart on credential changes:** Implement a mechanism to automatically restart Process Mining service instances when critical configuration or secret changes occur, ensuring updated credentials are applied without requiring manual intervention.

Read the full incident report →

Started Jul 16, 2026, 06:00 AM UTC· resolved Jul 16, 2026, 06:22 AM UTC

1. monitoring Jul 16, 2026, 06:00 AM UTC

   Between 2026-07-16 03:21 UTC and 03:41 UTC, and again between 2026-07-16 05:01 UTC and 05:36 UTC, customers in the US region may have experienced delayed execution of Serverless Jobs. We have applied a fix to address the issue, and the service has recovered. We are actively monitoring the service to ensure continued stability.

2. resolved Jul 16, 2026, 06:22 AM UTC

   The issue affecting Serverless Job execution latency in the US region has been resolved. Between 2026-07-16 03:21 UTC – 03:41 UTC and 2026-07-16 05:01 UTC – 05:36 UTC, some customers experienced delayed execution of Serverless Jobs in the US region. A fix has been implemented, and delayed job processing has recovered. We have verified that the service is operating normally and will continue to monitor it to ensure ongoing stability. This issue was caused by an underlying cloud infrastructure health event within our cloud service provider’s platform. We will work closely with the provider to improve service resiliency and help minimize the likelihood of similar incidents in the future.

3. postmortem Aug 03, 2026, 11:24 AM UTC

   ## Customer impact On July 16, 2026, some customers experienced delayed execution of Serverless Jobs in the US region during two separate intervals: 3:21 am UTC to 3:41 am UTC, and 5:01 am UTC to 5:36 am UTC. ## Root cause Our cloud service provider introduced new CPU hardware (a new processor model) into the virtual machine family used to run Serverless Job execution in the US region. The virtualization runtime we use to isolate job workloads was incompatible with the new hardware, so workloads scheduled onto newly provisioned capacity failed to start. Affected jobs were executed through an alternate execution path, and the time needed to detect the failure and fall back resulted in delayed job processing for a subset of customers. ## Detection The issue was detected through an automated alert for the Serverless service on July 16, 2026 at 5:46 am UTC. This was 2 hours, 25 minutes after the first impact interval began, and approximately 10 minutes after the second impact interval ended. The incident was acknowledged at 5:46 am UTC and marked as customer-impacting at 5:47 am UTC. ## Response At 5:46 am UTC on July 16, 2026, an incident alert was created and acknowledged for the Serverless service. At 5:47 am UTC, the incident was declared customer-impacting and responders began coordinating the response. At 5:54 am UTC, the confirmed customer impact windows were documented and the issue was declared mitigated. At 6:00 am UTC, a public status update was posted confirming that a fix had been applied, delayed job processing had recovered, and the service was being actively monitored. At 6:22 am UTC, the public status page was updated to resolved after verification that Serverless Job execution in the US region was operating normally. ## Follow up 1. Deploy a validation component that verifies newly provisioned compute capacity can run job workloads before customer jobs are scheduled onto it. (Completed) 2. Update the job isolation runtime so workloads start correctly on the new processor generation, removing the incompatibility. (In progress) 3. Engage the cloud service provider regarding the introduction of new CPU hardware into the requested virtual machine family, to improve advance visibility of hardware changes that can affect workload compatibility.

Read the full incident report →

Started Jul 09, 2026, 03:42 PM UTC· resolved Jul 09, 2026, 03:42 PM UTC

1. resolved Jul 09, 2026, 04:40 PM UTC

   Between 15:42 UTC and 15:58 UTC, Automation Cloud Data Fabric customers in the Japan region experienced a brief interruption in service. The issue occurred due to database maintenance. Our team responded promptly, mitigated the issue, and service has been fully restored.

2. postmortem Jul 20, 2026, 07:48 AM UTC

   ## Customer impact Between **July 9, 2026 at around 15:42 UTC and July 9, 2026 at around 15:58 UTC**, Japan Datafabric customers experienced a brief interruption in service. The issue occurred due to database maintenance. ## Root cause The root cause was an outdated configuration reference that did not transition to the database endpoint during maintenance, while the remaining configuration references were updated successfully. After the issue was identified, the configuration was corrected, and service was fully restored. ## Detection The incident was identified during post-maintenance regression testing and was addressed immediately, minimizing customer impact. ## Response On July 9, 2026, between 15:42 UTC and 15:58 UTC, Japan Datafabric customers experienced service errors during a database maintenance activity. During this 16-minute period, some customer requests failed. The issue was identified during post-maintenance regression testing. The root cause was traced to an database endpoint configuration during the maintenance activity. After the configuration was updated to reference the correct database endpoint, service was fully restored and the issue was mitigated. ## Follow-up Extend maintenance automation to also detect and update secrets holding DB auth creds missed during maintenance.

Read the full incident report →

Started Jul 09, 2026, 11:34 AM UTC· resolved Jul 09, 2026, 02:35 PM UTC

1. identified Jul 09, 2026, 11:34 AM UTC

   Generating a taxonomy with Autopilot on new IXP Unstructured and Complex Documents projects is currently unavailable across all regions. We have identified the root cause and a fix is being rolled out. We will update this page once the fix is fully deployed and the issue is resolved

2. identified Jul 09, 2026, 12:51 PM UTC

   We are working on deploying the fix

3. identified Jul 09, 2026, 02:08 PM UTC

   We are deploying fix in progress

4. resolved Jul 09, 2026, 02:35 PM UTC

   Issue has been resolved.

5. postmortem Jul 13, 2026, 05:11 PM UTC

   ## Customer impact Between July 7, 2026 \(when the defect reached production\) and July 9, 2026 at 2:35 pm UTC \(when the fix finished rolling out\), some attempts to create a new Unstructured and Complex Documents project did not complete normally. The project itself was always created and remained accessible from the project list, but the page showed an error such as "No such source" or "No such project" and returned to the project list instead of opening the new project—interrupting the Autopilot taxonomy generation that runs during project creation. The failure was intermittent, and affected all commercial cloud regions where the feature is available. ## Root cause A recent change introduced a short-lived, in-memory cache of user authorization data in the service, added to improve resilience of authorization checks under load. The cache did not refresh when a user's permissions changed. When a user created a new project, the permissions granted on that new project were not visible to requests served from a stale cache entry \(up to 60 seconds old\), so immediate follow-up requests failed as if the project did not exist. This caused the error, the redirect back to the project list, and the interruption of Autopilot taxonomy generation. ## Detection The issue was identified through internal testing and user reports on the morning of July 9, 2026, and an incident was declared at 11:02 am UTC. The root cause was identified within minutes of the reports. The public status page was updated at 11:34 am UTC, once the affected feature and regional scope were confirmed. We acknowledge the defect was present in production for approximately two days before detection; improving automated detection of this workflow is a follow-up action. ## Response The engineering team traced the failure to the authorization cache change and rolled back that change across all affected regions. The first region was fixed and verified at 12:52 pm UTC, and all regions were remediated by 1:19 pm UTC. Status page updates were posted at 11:34 am, 12:51 pm, and 2:08 pm UTC while the rollout progressed, and the incident was marked resolved at 2:35 pm UTC after verification. ## Follow-up To prevent a recurrence, we are: 1. Replacing the rolled-back change with a corrected version that refreshes cached authorization data when permissions change, validated end-to-end before release. 2. Adding automated end-to-end checks covering project creation with Autopilot taxonomy generation, so a regression in this workflow is detected within minutes.

Read the full incident report →

Started Jul 08, 2026, 10:23 PM UTC· resolved Jul 09, 2026, 05:12 AM UTC

1. investigating Jul 08, 2026, 10:23 PM UTC

   Automation Solutions Service is experiencing connectivity issues while deploying packages containing integration services entities (connectors). We are currently investigating the issue.

2. identified Jul 08, 2026, 11:36 PM UTC

   We have identified the issue. Currently working on deploying the fix.

3. identified Jul 09, 2026, 12:52 AM UTC

   We are currently deploying the fix in lower environment to validate.

4. identified Jul 09, 2026, 04:47 AM UTC

   We are currently rolling out the fix to the production environment.

5. resolved Jul 09, 2026, 05:12 AM UTC

   Degraded performance impacting deploying packages has been fully resolved. Impact: All affected functionality is operating normally.

6. postmortem Jul 23, 2026, 06:45 AM UTC

   ## Customer impact Between approximately **July 8, 2026 at 5:00 pm – 6:00 pm UTC and July 9, 2026 at 5:08 am UTC \(about 12 hours\)**, some customers in the **U.S. and Europe** regions experienced issues when **deploying Automation Solutions packages that included Integration Service connectors or connections**. Affected deployments stayed in progress and could time out after one hour instead of completing normally. Deployments that did not include Integration Service connectors or connections completed successfully. ## Root cause The issue was caused by a recent change in Integration Service \(Connection Service\) that was intended to improve connection handling. That change interrupted the completion signal that Automation Solutions expects after Integration Service finishes processing connectors or connections in a package. Because Automation Solutions waits for that signal before marking a deployment complete, affected deployments remained in progress until they timed out. We confirmed this by reverting to the change, after which deployments completed successfully again. ## Detection The issue was detected through automated alerts and by reproducing stuck Automation Solutions deployments. The first alerts were observed around 5:00 pm – 6:00 pm UTC on July 8, 2026. The incident was formally declared at 9:26 pm UTC, and the public status page was updated to investigate at 10:23 pm UTC. ## Response After the incident was declared at **9:26 pm UTC on July 8, 2026**, our team confirmed that only Automation Solutions deployments containing Integration Service connectors or connections were affected, and identified a recent Integration Service change as the cause. We updated the public status page as the investigation progressed \(**investigating** at 10:23 pm UTC, **identified** at 11:36 pm UTC\). We first attempted a targeted fix, then switched to reverting to the change that introduced the problem. On **July 9, 2026**, the revert completed at **5:08 am UTC**. We verified that solution deployments worked again and marked the status page **resolved** at **5:12 am UTC**. ## Follow-up To reduce the chance of a similar issue, we are: 1. Gating any reintroduction of the reverted Integration Service change on automated validation of Automation Solutions deployments that include Integration Service connectors or connections, so this path is covered before the change ships again 2. Expanding automated tests to cover the Automation Solutions deployment path through Integration Service, so regressions of this kind are caught before production 3. Continuing to monitor the original connection-handling issue this change was intended to fix. Any future fix for that issue will be required to pass the automated regression tests described above before deployment

Read the full incident report →

Started Jul 08, 2026, 05:26 PM UTC· resolved Jul 08, 2026, 06:56 PM UTC

1. investigating Jul 08, 2026, 05:26 PM UTC

   We are investigating reports of degraded performance impacting LLM Calls for Agents in the US and EU regions. Impact: Users may experience errors from LLM calls by Agents. Our teams are working to identify the cause and will share more details as the investigation progresses.

2. monitoring Jul 08, 2026, 05:41 PM UTC

   Mitigation has been applied and performance is improving for LLM Calls in Agents across the US and EU regions. Impact: Users may still experience failed LLM calls, but this should continue to improve. We are monitoring closely to ensure stability.

3. resolved Jul 08, 2026, 06:56 PM UTC

   The issue has been resolved and Agents performance has returned to expected levels after degraded performance impacted LLM Calls in US and Europe. Impact: No ongoing user impact.

4. postmortem Jul 14, 2026, 08:56 AM UTC

   ## Customer Impact On July 8, for about five and a half hours \(roughly 12:00 to 17:40 UTC\), a subset of Agents customers in the US and EU intermittently received HTTP 500 errors on LLM completion calls. Only calls routed to one affected instance failed, so retries often succeeded. No data was lost and no other functionality was affected. ## Root Cause The failures were caused by SNAT port exhaustion on a single node in one of our shared production clusters. That node reached its fixed limit of outbound ports, which are shared by all pods running on it, so it could no longer open new outbound connections. As a result, the agents calls to the LLM Gateway failed at the network layer and returned 500 to customers. The exhaustion was driven by the combined outbound connection volume of several services running on that node. We have identified the code that causes this problem, and are implementing a fix. ## Response Detection was delayed because of an independent under-reporting issne in telemetry, so our alerts stayed below paging severity. Once identified, the affected node was drained, which moved the pods onto healthy nodes and stopped the errors immediately. Draining a shared production node is a last-resort action, since it also evicts other teams' workloads from that node, which is why it was applied late rather than early. ## Follow-Up 1. Harden the code against SNAT leaks. 2. Create a new alert that would help us identify such issues quickly on the service sde.

Read the full incident report →

Started Jul 07, 2026, 11:45 AM UTC· resolved Jul 08, 2026, 05:58 PM UTC

1. identified Jul 07, 2026, 11:45 AM UTC

   Integration Service Customers Facing intermittent failures during connection creation and activities execution in US and Europe regions

2. identified Jul 07, 2026, 01:04 PM UTC

   The issue causing degraded performance affecting Integration Service in the US and Europe regions has been identified, and mitigation efforts are currently underway.

3. identified Jul 07, 2026, 02:30 PM UTC

   Fix has been applyed we are currently monitoring the service to ensure stability

4. monitoring Jul 07, 2026, 04:09 PM UTC

   Fix has been applyed we are currently monitoring the service to ensure stability

5. monitoring Jul 07, 2026, 05:36 PM UTC

   Fix has been applyed we are currently monitoring the service to ensure stability

6. identified Jul 07, 2026, 07:04 PM UTC

   we are seeing some intermittent failures. We are still investigating further on this.

7. identified Jul 07, 2026, 07:17 PM UTC

   Issue resolved in Us region. we are seeing some intermittent failures in Europe region. We are investigating further on this.

8. identified Jul 07, 2026, 07:18 PM UTC

   Issue resolved in Us region. we are seeing some intermittent failures in Europe region. We are investigating further on this.

9. identified Jul 07, 2026, 09:13 PM UTC

   We are working to deploy the fix

10. identified Jul 07, 2026, 10:37 PM UTC

    We are working to deploy the fix

11. identified Jul 08, 2026, 12:03 AM UTC

    We are working to deploy the fix

12. identified Jul 08, 2026, 05:45 AM UTC

    We are working to deploy the fix

13. identified Jul 08, 2026, 04:50 PM UTC

    Issue is now resolved in Europe. We are seeing some errors in US and we are continuing to on the fix.

14. monitoring Jul 08, 2026, 05:50 PM UTC

    Issue is mitigated now. we are now monitoring the issue.

15. resolved Jul 08, 2026, 05:58 PM UTC

    Issue is resolved now.

16. postmortem Jul 15, 2026, 07:15 PM UTC

    ## Customer impact Between July 7, 2026 at 10:26 am UTC and July 8, 2026 at 5:58 pm UTC, a subset of customers experienced intermittent failures when creating Integration Service connections and when running activities that depend on those connections. Customers encountered intermittent HTTP 502 errors and connection reset failures during connection creation and execution. The impact affected customers using Integration Service in the United States and Europe regions. The incident lasted approximately 31 hours and 32 minutes \(This was not a complete outage. Requests continued to succeed intermittently, and many failed requests were successful upon retry.\). The U.S. region initially recovered before intermittent failures continued in the Europe region, and residual errors were later observed again in the U.S. region. ## Root cause The root cause was exhaustion of available outbound connection capacity on shared infrastructure. This capacity is shared across multiple UiPath services, and a high aggregate volume of concurrent outbound connections exhausted the available capacity. Once the limit was reached, new outbound connections could not be opened reliably. As a result, the Integration Service components responsible for connection creation and activity execution intermittently failed to establish new outbound connections, which surfaced to customers as intermittent 502 responses and connection reset errors. The condition was not caused by any single service; it resulted from the combined outbound connection volume of multiple services sharing the same infrastructure. ## Detection The issue was detected through automated monitoring and customer reports on July 7, 2026 at 10:26 am UTC. The incident was declared customer-impacting within approximately one minute of the initial alert. Customer support cases were also reported later that morning, confirming that the failures were visible to affected users. ## Response At 11:23 am UTC on July 7, our engineering team identified that a high volume of external connections on shared infrastructure was blocking new outbound connections once capacity was reached. At 11:39 am UTC, additional capacity for the affected Integration Service component in the Europe region was requested. By 12:26 pm UTC, connection reset errors had decreased, and the team continued monitoring. At 12:49 pm UTC, two overloaded infrastructure components were identified for remediation to reduce the immediate impact. At 2:30 pm UTC, the public status page was updated to indicate that a fix had been applied and that the service was being monitored. At 4:09 pm UTC, the incident was marked as mitigated. During continued monitoring, additional infrastructure capacity was added at approximately 5:48 pm UTC. At 7:17 pm UTC, the U.S. region was reported as resolved, while intermittent failures continued in the Europe region. The incident was reopened for continued investigation. On July 8, 2026, a connection pooling change was deployed to the U.S. region at 3:09 pm UTC to improve connection efficiency; however, errors continued afterward. Full recovery was achieved by remediating the affected infrastructure to restore outbound connection capacity. Europe was reported resolved at 4:50 pm UTC. Remaining U.S. errors then subsided. The incident was marked mitigated at 5:50 pm UTC, and full recovery was confirmed when the incident was marked resolved at 5:58 pm UTC. ## Follow-up The following actions are planned or recommended based on the investigation findings: ### Additional Recommendations 1. Improve connection efficiency \(connection reuse\) to reduce the number of outbound connections created under load, with careful validation before rollout which would prevent incidents like these in future 2. Improve detection and mitigation mechanisms so that outbound connection capacity pressure is identified and remediated before it causes customer-facing connection creation or activity execution failures.

Read the full incident report →

Started Jul 06, 2026, 01:53 AM UTC· resolved Jul 06, 2026, 02:40 AM UTC

1. investigating Jul 06, 2026, 01:53 AM UTC

   We are investigating an issue affecting the Apps runtime in the Japan region. Impacted customers may see delayed load times or failures when attempting to open deployed Apps. We will provide updates as more information becomes available.

2. resolved Jul 06, 2026, 02:40 AM UTC

   The issue is has been resolved and we are actively monitoring the application.

3. postmortem Jul 30, 2026, 10:32 AM UTC

   ## Customer impact Between July 5, 2026 at ~22:30 UTC and July 6, 2026 at ~02:00 UTC, a limited number of tenants in the Japan region experienced increased latency and degraded performance when using Business Apps. Some requests may have timed out during this period. ## Root cause During a routine service update, a database index used by a frequently-executed query was adjusted. The change itself was correct; however, at deployment the database generated a new execution plan for that query. The plan was built from a non-representative dataset \(a tenant with a small amount of data\), so it was not well-suited to production-scale data. As a result, for tenants with a large amount of data the query began executing inefficiently, which caused the increased latency and request timeouts that customers experienced. ## Detection The issue was detected through an automated latency alert at 00:44 UTC on July 6, 2026. ## Response The affected Orchestrator database was identified as needing additional capacity. Additional capacity was added to the database, after which the query regenerated an efficient execution plan and performance returned to normal. The incident was mitigated by approximately 02:00 UTC on July 6, 2026, and monitoring continued afterward to confirm recovery. ## Follow up * Activating query changes independently of deployment, so execution plans are generated against representative production data. * Tooling to detect and clear inefficient execution plans automatically, so the database can regenerate a better one. * Monitoring to flag when a single query consumes a disproportionate share of database resources, so we can act before performance is affected.

Read the full incident report →

Started Jul 03, 2026, 03:28 AM UTC· resolved Jul 03, 2026, 10:33 AM UTC

1. investigating Jul 03, 2026, 02:24 AM UTC

   We have identified the issue and currently working on a fix.

2. identified Jul 03, 2026, 03:28 AM UTC

   We have identified the cause of the degraded performance impacting Orchestrator API trigger jobs in US and we are working on mitigation.

3. identified Jul 03, 2026, 03:47 AM UTC

   We are continuing to work on a fix for this issue.

4. identified Jul 03, 2026, 05:09 AM UTC

   The fix is now live in the US and Japan. The team is currently focused on mitigating the issue in the remaining environments

5. identified Jul 03, 2026, 06:11 AM UTC

   The fix is live and working as expected in the US and Japan. We've started deploying it to the remaining affected regions.

6. identified Jul 03, 2026, 07:22 AM UTC

   Deployment and validation for the remaining regions are underway. We will share further updates as soon as they become available.

7. identified Jul 03, 2026, 08:27 AM UTC

   Services have been restored to operational status in the EU, Australia, and Switzerland. Validation for the remaining regions is underway.

8. identified Jul 03, 2026, 08:44 AM UTC

   Services have been restored to operational status in the India, UAE, Canada, EU, Australia, and Switzerland. Validation for the remaining regions (UK, Singapore) is underway.

9. identified Jul 03, 2026, 09:54 AM UTC

   The fix has now been successfully deployed to the UK. As a result, the following regions are now operational: UK, India, UAE, Canada, EU, Australia, Japan, US and Switzerland

10. monitoring Jul 03, 2026, 10:05 AM UTC

    Fix is now live in all regions. Early metrics from monitoring look stable. We'll keep a close eye on everything for the next 30 minutes before declaring full resolution.

11. resolved Jul 03, 2026, 10:33 AM UTC

    This incident has been resolved.

12. postmortem Jul 16, 2026, 09:37 AM UTC

    ## Customer impact Between **July 3, 2026 at 02:19 UTC** and **July 3, 2026 at 10:00 UTC**, a subset of customers experienced incorrect execution behavior for **Orchestrator API trigger jobs**. Jobs initiated through API triggers did not consistently use their configured execution target, including the intended template, account, and machine settings. The issue affected customers in the **U.S. and Japan regions**. Once identified, the fix was deployed to those regions first and then rolled out to the remaining affected regions. Service was fully restored after deployment was completed across all environments. ## Root cause The incident was caused by a recent change to Orchestrator’s job allocation logic that unintentionally altered how execution targets were resolved for API-triggered jobs. As a result, API-triggered jobs did not consistently honor their configured template, account, and machine settings, and could instead be allocated to other eligible machines and accounts within the same folder. ## Detection The issue was first identified through a customer report on **July 3, 2026 at 02:19 UTC**. The incident was acknowledged immediately, and engineering quickly confirmed the underlying issue. A public status page update was published at **02:24 UTC** to inform customers that the issue had been identified and that remediation was in progress. ## Response Following the customer report, engineering initiated an incident response and began mitigating the issue by reverting the affected environments to the previous stable version while simultaneously preparing a permanent fix. After the corrective changes were developed, reviewed, and validated, they were deployed first to the U.S. and Japan regions, where service was confirmed to be operating normally. The deployment was then completed across the remaining affected regions. Following validation that API-triggered jobs were once again honoring their configured execution targets, the incident was marked mitigated and subsequently resolved. ## **Follow-Up** **Completed** * Added automated regression tests to verify that API-triggered jobs consistently preserve and honor their configured execution targets, including template, account, and machine settings.

Read the full incident report →