Sauce Labs incident

2026-August-07 Service Incident

Resolved after 57 minutes, on August 7, 2026.

postmortem Aug 14, 2026, 04:43 PM UTC

### **Dates:** Friday, August 7th 2026, 11:00 UTC - 13:04 UTC. ### **What happened:** Windows and Intel Mac jobs in `us-west1` failed to start due to virtual machine \(VM\) allocation starvation. ### **Why it happened:** A service crash loop left VMs in an allocated but unclaimed state, while a cleanup bug prevented the system from releasing the orphaned capacity to boot new VMs. ### **How we fixed it:** Restored service stability and cleared stale allocations to resume VM provisioning and clear queued jobs. ### **What we are doing to prevent it from happening again:** Fixing allocator cleanup logic, strengthening deployment health checks, and improving capacity accounting for stale allocations.

Started
Aug 07, 2026, 12:08 PM UTC
Resolved
Aug 07, 2026, 01:06 PM UTC
Duration
57m
Detected by Pingoru
Aug 07, 2026, 12:08 PM UTC

Update timeline

  1. investigating Aug 07, 2026, 12:08 PM UTC

    We are currently seeing elevated error rates for virtual desktop tests in the US-West-1 datacenter. We are currently investigating.

  2. resolved Aug 07, 2026, 01:06 PM UTC

    After taking remedial action, Virtual Desktop tests are starting successfully in the US-West-1 Data center. All services are fully operational. We are closely monitoring the situation

  3. postmortem Aug 14, 2026, 04:43 PM UTC

    ### **Dates:** Friday, August 7th 2026, 11:00 UTC - 13:04 UTC. ### **What happened:** Windows and Intel Mac jobs in `us-west1` failed to start due to virtual machine \(VM\) allocation starvation. ### **Why it happened:** A service crash loop left VMs in an allocated but unclaimed state, while a cleanup bug prevented the system from releasing the orphaned capacity to boot new VMs. ### **How we fixed it:** Restored service stability and cleared stale allocations to resume VM provisioning and clear queued jobs. ### **What we are doing to prevent it from happening again:** Fixing allocator cleanup logic, strengthening deployment health checks, and improving capacity accounting for stale allocations.

Your product runs on more than Sauce Labs

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →