Delays in Serverless job execution - United States

Update timeline

1. monitoring Jul 16, 2026, 06:00 AM UTC

   Between 2026-07-16 03:21 UTC and 03:41 UTC, and again between 2026-07-16 05:01 UTC and 05:36 UTC, customers in the US region may have experienced delayed execution of Serverless Jobs. We have applied a fix to address the issue, and the service has recovered. We are actively monitoring the service to ensure continued stability.

2. resolved Jul 16, 2026, 06:22 AM UTC

   The issue affecting Serverless Job execution latency in the US region has been resolved. Between 2026-07-16 03:21 UTC – 03:41 UTC and 2026-07-16 05:01 UTC – 05:36 UTC, some customers experienced delayed execution of Serverless Jobs in the US region. A fix has been implemented, and delayed job processing has recovered. We have verified that the service is operating normally and will continue to monitor it to ensure ongoing stability. This issue was caused by an underlying cloud infrastructure health event within our cloud service provider’s platform. We will work closely with the provider to improve service resiliency and help minimize the likelihood of similar incidents in the future.

3. postmortem Aug 03, 2026, 11:24 AM UTC

   ## Customer impact On July 16, 2026, some customers experienced delayed execution of Serverless Jobs in the US region during two separate intervals: 3:21 am UTC to 3:41 am UTC, and 5:01 am UTC to 5:36 am UTC. ## Root cause Our cloud service provider introduced new CPU hardware (a new processor model) into the virtual machine family used to run Serverless Job execution in the US region. The virtualization runtime we use to isolate job workloads was incompatible with the new hardware, so workloads scheduled onto newly provisioned capacity failed to start. Affected jobs were executed through an alternate execution path, and the time needed to detect the failure and fall back resulted in delayed job processing for a subset of customers. ## Detection The issue was detected through an automated alert for the Serverless service on July 16, 2026 at 5:46 am UTC. This was 2 hours, 25 minutes after the first impact interval began, and approximately 10 minutes after the second impact interval ended. The incident was acknowledged at 5:46 am UTC and marked as customer-impacting at 5:47 am UTC. ## Response At 5:46 am UTC on July 16, 2026, an incident alert was created and acknowledged for the Serverless service. At 5:47 am UTC, the incident was declared customer-impacting and responders began coordinating the response. At 5:54 am UTC, the confirmed customer impact windows were documented and the issue was declared mitigated. At 6:00 am UTC, a public status update was posted confirming that a fix had been applied, delayed job processing had recovered, and the service was being actively monitored. At 6:22 am UTC, the public status page was updated to resolved after verification that Serverless Job execution in the US region was operating normally. ## Follow up 1. Deploy a validation component that verifies newly provisioned compute capacity can run job workloads before customer jobs are scheduled onto it. (Completed) 2. Update the job isolation runtime so workloads start correctly on the new processor generation, removing the incompatibility. (In progress) 3. Engage the cloud service provider regarding the introduction of new CPU hardware into the requested virtual machine family, to improve advance visibility of hardware changes that can affect workload compatibility.