[Multiple Regions] Orchestrator degraded performance

Update timeline

1. identified Jun 30, 2026, 08:10 PM UTC

   We have identified the cause of the degraded performance impacting the GetJobs Activity for Orchestrator in Europe, Switzerland, Canada, India, Australia and UAE. We are actively working on a mitigation, which will be applied shortly Additional updates will be provided as we move toward resolution.

2. identified Jun 30, 2026, 09:30 PM UTC

   We have a mitigation available, and are beginning to roll it out to affected regions. We will provide more updates as the mitigation takes effect

3. monitoring Jun 30, 2026, 10:44 PM UTC

   The mitigation is taking effect, and we are seeing performance return to normal

4. monitoring Jun 30, 2026, 10:48 PM UTC

   The mitigation is taking effect, and we are seeing performance return to normal

5. monitoring Jun 30, 2026, 10:59 PM UTC

   All regions have recovered, we will monitor to ensure stability

6. resolved Jun 30, 2026, 11:17 PM UTC

   The incident has been resolved.

7. postmortem Jul 15, 2026, 10:51 AM UTC

   ## Customer Impact Beginning on **June 29, 2026 at 4:00 pm UTC and continuing until June 30, 2026 at 10:59 pm UTC**, customers across six regions—Europe, Switzerland, Canada, India, Australia, and the UAE, experienced failures when using the Get Jobs activity in Orchestrator. Affected automations returned the error: "**Could not find a property named 'Environment' on type SimpleReleaseDto.**" The total customer-facing impact duration was approximately 31 hours, with the earliest affected region impacted for more than 24 hours before the issue was identified. Automations that depended on this activity failed when attempting to retrieve job information. ## Root Cause As part of the announced deprecations related to robots and environments, a corresponding server-side change was made that stopped accepting requests in the legacy format. Multiple already-released versions of the _Get Jobs_ activity were still sending requests using that deprecated format specifically including the `Environment` property. As a result, the server rejected those requests, causing the activity to fail across all affected versions. ## Detection The issue was detected through a customer-impacting alert on June 30, 2026 at 7:49 pm UTC. At 8:40 pm UTC, a log review showed that one affected region \(Australia\) had already been impacted for more than 24 hours, while Canada and India were in their first hours of impact. The issue was introduced with a hotfix deployed in Europe region on June 25 and subsequently propagated to Switzerland, Canada, India, Australia, and the UAE between June 29 and June 30. ## Response On June 30, 2026 at 7:58 pm UTC, a server-side fix was identified for deployment. At 8:03 pm UTC, the engineering team determined the scope of degradation for customers using the Get Jobs activity and recommended updating our public status page. At 8:10 pm UTC, the status page was updated to indicate that the cause had been identified and that mitigation work was underway. The mitigation was a server-side compatibility fix that restored acceptance of the deprecated request format used by existing Get Jobs activity versions. Deployment began on June 30, 2026 at 9:10 pm UTC. By 10:22 pm UTC, successful requests were observed in the first recovered regions, and by 10:44 pm UTC the incident was marked mitigated as performance was returning to normal. Deployment completed across all affected regions on June 30, 2026 at 10:54 pm UTC. At 10:59 pm UTC, all regions were confirmed to have recovered and the public status page moved to monitoring. The incident was marked resolved on June 30, 2026 at 11:17 pm UTC. ## Follow-up 1. We added automated regression tests to validate compatibility for older Get Jobs request formats before similar server-side changes are released, preventing this category of regression in future updates. 2. We are adding a deprecation alert that will capture similar cases in the future.