Sauce Labs experienced a major incident on January 8, 2025 affecting US-West and US-West and 1 more component, lasting 1h 14m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jan 08, 2025, 11:40 AM UTC
We are currently seeing Live and automated test results are not being retained intermittently in our US-West-1 data centers. We are investigating.
- investigating Jan 08, 2025, 11:56 AM UTC
We are currently seeing Live and Automated test results are not being retained for Virtual Device tests intermittently in our US-West-1 data centers. We are investigating.
- investigating Jan 08, 2025, 12:41 PM UTC
We are continuing to investigate this issue.
- investigating Jan 08, 2025, 12:44 PM UTC
We are currently seeing intermittent issues with test assets not being retained for Virtual device tests in our US-West-1 data center. We are investigating.
- resolved Jan 08, 2025, 12:54 PM UTC
After taking remedial action, test assets are being retained successfully in all tests. This incident is resolved.
- postmortem Apr 01, 2025, 04:14 PM UTC
### **Dates:** Wednesday 8 January 2025, 11:00 - 12:25 UTC ### **What happened:** We were experiencing intermittent issues with Virtual device tests missing test assets in the US-West-1 region ### **Why it happened:** The asset uploader service was experiencing Http errors uploading assets to our backend storage. ### **How we fixed it:** The 3rd-party provider resolved an issue with their infrastructure. ### **What we are doing to prevent it from happening again:** We are improving the caching and retry logic of the asset uploader service to prevent further occurrence.