Sauce Labs Outage History

Started Oct 13, 2026, 08:58 AM UTC· resolved Oct 13, 2026, 10:10 PM UTC

1. investigating Oct 13, 2026, 08:58 AM UTC

   We are experiencing an issue with the Sauce Labs UI in EU-Central-1 data center. We are investigating.

2. investigating Oct 13, 2026, 10:04 AM UTC

   We are still experiencing issues with the Sauce Labs UI in the EU-Central-1 data center. We are continuing to investigate.

3. investigating Oct 13, 2026, 11:38 AM UTC

   We have made progress in narrowing down the cause of the ongoing Sauce Labs UI issues in the EU-Central-1 data center. The investigation is ongoing.

4. investigating Oct 13, 2026, 01:30 PM UTC

   We are working closely with our third-party providers to resolve the ongoing issue in the EU-Central-1 data center. The investigation is ongoing.

5. investigating Oct 13, 2026, 03:39 PM UTC

   We are continuing to work with our partners and relevant teams to investigate and resolve the ongoing issue in the EU-Central-1 data center. We will share further updates as more information becomes available.

6. resolved Oct 13, 2026, 10:10 PM UTC

   The root cause of issue affecting our UI access in the EU-Central-1 is tied to an ongoing incident with one of our third-party infrastructure providers in the data center. While our team is keeping this incident ticket closed on our end, we are actively tracking the provider's ongoing investigation and will monitor performance closely.

Read the full incident report →

Started Oct 13, 2026, 05:10 AM UTC· resolved Oct 13, 2026, 05:45 AM UTC

1. resolved Oct 13, 2026, 10:09 AM UTC

   On 13 October, between 05:10 UTC and 05:45 UTC, we experienced elevated error rates for VDC, RDC, Sauce Connect, and IPSec services in the US-West-1 data center. The issue has been resolved. All services are now fully operational.

Read the full incident report →

Started Oct 08, 2026, 04:48 PM UTC· resolved Oct 08, 2026, 06:50 PM UTC

1. investigating Oct 08, 2026, 04:48 PM UTC

   We are experiencing a degradation in error ingestion for Error Reporting (Backtrace) using crashpad on Android and Linux across all Data Centers. We are currently investigating.

2. investigating Oct 08, 2026, 04:52 PM UTC

   We are continuing to investigate this issue.

3. monitoring Oct 08, 2026, 05:20 PM UTC

   The root cause of error ingestion for Error Reporting (Backtrace) has been identified. Remediation has been applied, and we are actively monitoring the situation.

4. resolved Oct 08, 2026, 06:50 PM UTC

   After taking remedial action, the error ingestion issue in Error Reporting (Backtrace) has been resolved. All services are fully operational.

Read the full incident report →

Started Sep 30, 2026, 10:04 AM UTC· resolved Sep 30, 2026, 11:18 AM UTC

1. investigating Sep 30, 2026, 10:04 AM UTC

   We are currently seeing elevated error rates for real device tests in EU-Central-1 data center. We are investigating.

2. resolved Sep 30, 2026, 11:18 AM UTC

   Real Device test error rates have returned to normal in the EU-Central-1 data center. All services are fully operational.

3. postmortem Oct 14, 2026, 05:12 PM UTC

   ### **Dates:** Wednesday, September 30th 2026, 09:04 UTC - 10:49 UTC. ### **What happened:** Real Device Automated Tests experienced failure to start in the EU Central data center, with error rates reaching up to 25%. ### **Why it happened:** A deployment led to increased cache misses and elevated load on the underlying infrastructure. ### **How we fixed it:** Service was restored by rolling back to a previous version. ### **What we are doing to prevent it from happening again:** Alternative caching strategies are being considered to increase effectiveness of the cache.

Read the full incident report →

Started Sep 28, 2026, 04:04 PM UTC· resolved Sep 28, 2026, 05:19 PM UTC

1. investigating Sep 28, 2026, 04:04 PM UTC

   We are currently seeing elevated error rates for automated real device tests in US-West-1 data center. We are currently investigating.

2. monitoring Sep 28, 2026, 04:56 PM UTC

   The root cause of the elevated error rates for Real Device tests in the US-West-1 data center has been identified. Remediation has been applied, and we are actively monitoring the situation.

3. resolved Sep 28, 2026, 05:19 PM UTC

   After taking remedial action, Real Device test error rates have returned to normal in the US-West-1 data center. All services are fully operational.

4. postmortem Oct 02, 2026, 07:15 PM UTC

   ### **Dates:** Monday, September 28th 2026, 14:19 UTC - 16:52 UTC. ### **What happened:** Real device tests in US West hung and failed to output test artifacts due to slow uploads. ### **Why it happened:** A degraded interconnect link suffered high packet loss from optical errors, but BGP stayed up, preventing automatic failover to healthy paths. ### **How we fixed it:** Manually disabled the degraded port on the edge router, forcing traffic onto healthy alternate interconnects. ### **What we are doing to prevent it from happening again:** Automating link draining based on CRC error thresholds, improving protocols for isolating flapping vendor links, and hardening upload resilience.

Read the full incident report →

Started Sep 23, 2026, 12:05 PM UTC· resolved Sep 23, 2026, 12:35 PM UTC

1. resolved Sep 23, 2026, 12:45 PM UTC

   Between 12:05 UTC and 12:35 UTC, we experienced a high error rate and longer session start times for Desktop and Virtual Device tests in the US-West-1 data center. The issue has been resolved. All services are fully operational.

2. postmortem Oct 21, 2026, 05:00 PM UTC

   ### **Dates:** Wednesday, September 23rd 2026, 12:05 UTC - 12:35 UTC. ### **What happened:** Virtual Device Cloud testing availability gradually decreased in the US-West-1 data center. ### **Why it happened:** A product defect was introduced causing Virtual Device Cloud testing sessions to fail to start. ### **How we fixed it:** Rollback to a stable version. ### **What we are doing to prevent it from happening again:** Strengthen pre-deployment validation and reassess configuration rollout strategies.

Read the full incident report →

Started Sep 16, 2026, 09:52 AM UTC· resolved Sep 16, 2026, 10:34 AM UTC

1. resolved Sep 16, 2026, 11:08 AM UTC

   Between 09:52 UTC and 10:34 UTC, multiple Sauce Labs services in the US-West data center experienced an outage, causing authentication failures and disruptions to testing features. All systems are now fully operational.

Read the full incident report →

Started Sep 02, 2026, 09:39 AM UTC· resolved Sep 02, 2026, 11:53 AM UTC

1. investigating Sep 02, 2026, 09:39 AM UTC

   We are seeing elevated error rates for Real Device tests in EU-Central-1 data center. We are investigating.

2. identified Sep 02, 2026, 11:30 AM UTC

   The root cause of the elevated error rates for Real Device tests in the EU-Central-1 data center has been identified. Remediation has been applied, and we are actively monitoring the situation.

3. resolved Sep 02, 2026, 11:53 AM UTC

   After taking remedial action, Real Device test error rates have returned to normal in the EU-Central-1 data center. All services are fully operational.

4. postmortem Sep 18, 2026, 02:11 AM UTC

   **Dates:** Wednesday, September 2nd 2026, 08:30 UTC - 11:15 UTC. ### **What happened:** Real Device Automated Tests experienced failure to start in the EU Central data center, with elevated error rates reaching nearly 100% for short periods of time. ### **Why it happened:** A deployment led to increased cache misses and elevated load on the underlying infrastructure. ### **How we fixed it:** Service was restored by restarting the application. ### **What we are doing to prevent it from happening again:** Additional resources were allocated to reduce the likelihood of a recurrence in the short term. In the long term, we will optimize the performance of resource intensive tasks and introduce dynamic scaling.

Read the full incident report →

Started Aug 25, 2026, 12:33 PM UTC· resolved Aug 25, 2026, 01:19 PM UTC

1. investigating Aug 25, 2026, 12:33 PM UTC

   We are currently seeing elevated error rates for macOS and iOS tests in the US-West-1 data center. We are currently investigating.

2. resolved Aug 25, 2026, 01:19 PM UTC

   After taking remedial action, all macOS and iOS tests are now performing as expected in the US-West-1 Data center. All services are fully operational.

3. postmortem Sep 03, 2026, 08:23 PM UTC

   ### **Dates:** Tuesday August 25th 2026, 11:38 – 13:10 UTC ### **What happened:** Customers running macOS and iOS tests in our US-West region experienced degraded service. Roughly 50% of the virtual Mac capacity in the region stopped accepting new tests, so tests either queued or failed to start. Remaining capacity came under additional pressure as work shifted onto it, which extended start times for both desktop and simulator tests. ### **Why it happened:** An internal security certificate used by our Mac hosts to reach a supporting cloud service reached its expiry date. Once it lapsed, the hosts could no longer establish a trusted connection to that service and stopped provisioning new test machines. The certificate had been issued manually and had neither automated renewal nor expiry alerting. ### **How we fixed it:** We issued and deployed a replacement certificate, which restored connectivity and returned Mac capacity to normal levels. ### **What we are doing to prevent it from happening again:** We are moving these certificates onto automated renewal and adding alerting so they are replaced well ahead of expiry, along with reviewing the surrounding tooling to make certificate handling safer.

Read the full incident report →

Started Aug 07, 2026, 12:08 PM UTC· resolved Aug 07, 2026, 01:06 PM UTC

1. investigating Aug 07, 2026, 12:08 PM UTC

   We are currently seeing elevated error rates for virtual desktop tests in the US-West-1 datacenter. We are currently investigating.

2. resolved Aug 07, 2026, 01:06 PM UTC

   After taking remedial action, Virtual Desktop tests are starting successfully in the US-West-1 Data center. All services are fully operational. We are closely monitoring the situation

3. postmortem Aug 14, 2026, 04:43 PM UTC

   ### **Dates:** Friday, August 7th 2026, 11:00 UTC - 13:04 UTC. ### **What happened:** Windows and Intel Mac jobs in `us-west1` failed to start due to virtual machine \(VM\) allocation starvation. ### **Why it happened:** A service crash loop left VMs in an allocated but unclaimed state, while a cleanup bug prevented the system from releasing the orphaned capacity to boot new VMs. ### **How we fixed it:** Restored service stability and cleared stale allocations to resume VM provisioning and clear queued jobs. ### **What we are doing to prevent it from happening again:** Fixing allocator cleanup logic, strengthening deployment health checks, and improving capacity accounting for stale allocations.

Read the full incident report →

Started Jul 16, 2026, 01:00 PM UTC· resolved Jul 16, 2026, 05:33 PM UTC

1. resolved Jul 16, 2026, 08:33 PM UTC

   Between July 16th 13:03 UTC and July 16th 17:33 UTC, we experienced a service disruption where symbol archives utilizing multi-part upload protocols failed to process. Incident has been resolved and all services are operational.

2. postmortem Aug 14, 2026, 04:48 PM UTC

   ### **Dates:** Thursday July 16th 2026, 13:03 – 17:33 UTC ### **What happened:** Symbol archives uploaded in multiple parts failed to process. All regions were affected. ### **Why it happened:** Our symbol processing service sent an upload-verification field that our cloud storage provider's API does not accept for multi-part uploads, so those uploads were rejected. A fix for this had already been developed, but it had not yet been included in a released build and the service was not configured to use it. As in the first incident, rejected uploads were retried and accumulated on local disk. ### **How we fixed it:** We deployed a build containing the fix and corrected the service configuration on the affected workers. A large backlog of uploads then processed, which briefly re-filled the disks before draining completely. ### **What we are doing to prevent it from happening again:** We are releasing the fix formally through our build pipeline and persisting the corrected configuration in our configuration management, so it can't be lost. The disk-utilization alerting added after the first incident also covers the disk-exhaustion pattern common to both.

Read the full incident report →

Started Jul 14, 2026, 11:30 PM UTC· resolved Jul 15, 2026, 11:05 PM UTC

1. resolved Jul 16, 2026, 08:30 PM UTC

   Between July 14th 23:45 UTC and July 15th 23:05 UTC, we experienced an issue where symbol archive uploads to Backtrace projects failed with HTTP 400 errors. Incident has been resolved and all services are operational.

2. postmortem Aug 14, 2026, 04:47 PM UTC

   ### **Dates:** Tuesday July 14th 2026, 23:45 UTC – Wednesday July 15th 2026, 23:05 UTC ### **What happened:** Symbol archive uploads to Error Reporting \(Backtrace\) projects failed with HTTP 400 errors. All regions were affected. ### **Why it happened:** The credentials our symbol processing service used to write to cloud storage were no longer valid, so uploads could not be stored. Failed uploads were retried repeatedly and accumulated on local disk until the service ran out of space, at which point it also began rejecting new uploads. ### **How we fixed it:** We reissued the storage credentials, increased disk capacity on the affected workers, and restarted the service. The queued uploads then processed successfully, and we confirmed recovery with affected customers. ### **What we are doing to prevent it from happening again:** We've added disk-utilization monitoring and alerting to this service so we detect the condition ourselves before it affects uploads, rather than relying on customer reports.

Read the full incident report →

Started Jul 14, 2026, 05:19 PM UTC· resolved Jul 14, 2026, 08:23 PM UTC

1. investigating Jul 14, 2026, 05:19 PM UTC

   We are experiencing an issue in the EU Central 1 data center, where iOS ARM Simulator app tests fail with infrastructure errors. We are investigating.

2. investigating Jul 14, 2026, 06:05 PM UTC

   App downloads are taking longer than expected when these infrastructure errors are seen. We are continuing to investigate.

3. monitoring Jul 14, 2026, 06:56 PM UTC

   We have deployed a fix for this issue. Tests should now be performing as expected in the EU Central 1 data center. We are monitoring.

4. monitoring Jul 14, 2026, 07:11 PM UTC

   We are continuing to monitor for any further issues.

5. resolved Jul 14, 2026, 08:23 PM UTC

   After taking remedial action and monitoring the situation, we are seeing iOS ARM application tests perform as expected in the EU Data Center. All services are fully operational.

6. postmortem Jul 29, 2026, 05:40 PM UTC

   ### **Dates:** Tuesday July 14th 2026, 09:00 - 18:28 UTC ### **What happened:** iOS ARM tests running in our EU data center failed. ### **Why it happened:** A fault occurred with our primary network provider in our EU data center. ### **How we fixed it:** We failed over to a secondary network provider from our EU data center. ### **What we are doing to prevent it from happening again:** We've improved our monitoring and alerting to catch issues with third party network providers.

Read the full incident report →

Started Jul 01, 2026, 05:02 PM UTC· resolved Jul 01, 2026, 09:31 PM UTC

1. resolved Jul 01, 2026, 09:37 PM UTC

   On July 1st between 17:02 UTC and 21:31 UTC, macOS 14 tests were failing to start in the US-West and EU-Central data centers. This issue has been resolved. All services are fully operational.

2. postmortem Aug 11, 2026, 08:37 PM UTC

   ### **Dates:** Wednesday July 1st 2026, 17:02 UTC - 21:31 UTC ### **What happened:** macOS 14 tests in the US West and EU Central data centers were unable to start. ### **Why it happened:** An internal datasource was unavailable due to a missing configuration entry. ### **How we fixed it:** The missing entry was replaced. ### **What we are doing to prevent it from happening again:** Safeguards have been put in place to prevent in-use datasource removal from configuration.

Read the full incident report →

Started Jul 01, 2026, 12:12 PM UTC· resolved Jul 01, 2026, 12:58 PM UTC

1. investigating Jul 01, 2026, 12:12 PM UTC

   We are experiencing an issue with accessing Appium Inspector in the US-West-1, EU-Central-1, and US-East-4 data centers. We are investigating.

2. resolved Jul 01, 2026, 12:58 PM UTC

   We have identified the root cause and have deployed a fix for this issue. All services are fully operational.

3. postmortem Jul 20, 2026, 08:34 PM UTC

   ### **Dates:** Wednesday July 1st 2026, 09:37 – 12:46 UTC ### **What happened:** The Appium Inspector feature, used during real device live testing sessions, became unavailable after a scheduled UI deployment. Users who attempted to use the feature during a live test were presented with a 500 error, disrupting their active testing session. Automated test pipelines were not affected. ### **Why it happened:** A routine upgrade of a core frontend library introduced an incompatibility with the Appium Inspector's rendering logic. The previous library version tolerated the pattern, but the updated version did not. The issue was not caught before deployment due to insufficient end-to-end test coverage for this specific feature. ### **How we fixed it:** We rolled back the UI to the last known working version to restore the feature immediately, and then deployed a targeted fix for the incompatibility. ### **What we are doing to prevent it from happening again:** We are adding end-to-end test coverage for the Appium Inspector feature to ensure it is validated automatically before future deployments.

Read the full incident report →

Started Jun 24, 2026, 09:42 AM UTC· resolved Jun 24, 2026, 03:45 PM UTC

1. resolved Jun 24, 2026, 04:39 PM UTC

   On June 24th between 08:28 UTC and 15:32 UTC, we experienced Real Device test session failures impacting Appium and Access API in the US East, US West and EU data centers. This issue has been resolved. All services are fully operational.

2. postmortem Jul 22, 2026, 08:49 PM UTC

   ### **Dates:** Wednesday, June 24th 2026, 09:42 UTC - 15:45 UTC. ### **What happened:** Real Device test sessions using Appium and Access API experienced increased error rates in the US East, US West, and EU data centers. Sessions were timing out after approximately 90 seconds or becoming stuck in the "Connecting" state, preventing them from being closed. ### **Why it happened:** A product defect was introduced, causing Real Device test sessions to fail to start and preventing active sessions from being closed. ### **How we fixed it:** Rollback to a stable version. ### **What we are doing to prevent it from happening again:** Improve monitoring & alerting and enhance post deployment validation.

Read the full incident report →

Started Jun 18, 2026, 04:00 AM UTC· resolved Jun 18, 2026, 06:59 AM UTC

1. investigating Jun 18, 2026, 05:30 AM UTC

   Around 5:26 AM UTC we started experiencing lower iOS device availability in the EU-Central data center. Our team is actively investigating the root cause and working toward a resolution.

2. monitoring Jun 18, 2026, 06:12 AM UTC

   We have been experiencing device unavailability in our EU-Central-1 data center and have identified the root cause. We have taken remedial action and are currently monitoring.

3. resolved Jun 18, 2026, 06:59 AM UTC

   After taking remedial action, we are now seeing real devices available for testing in the EU-Central-1 data center. All services are fully operational.

4. postmortem Jul 06, 2026, 05:17 PM UTC

   ### **Dates:** Thursday June 18 2026, 03:45 UTC - 06:45 UTC. ### **What happened:** Approximately 7% of iOS devices in our EU data center were temporarily unavailable for customer test sessions after a loss of power to the rack hosting them. ### **Why it happened:** The power circuit feeding the affected rack exceeded its capacity, and a protective breaker tripped to safeguard the line - cutting power to the devices on that rack until the circuit was restored. ### **How we fixed it:** The affected circuit was reset and power restored to the rack, returning the devices to customer service. ### **What we are doing to prevent it from happening again:** We are redistributing power load across affected racks, adding capacity monitoring with early-warning alerts ahead of circuit limits, and introducing a capacity review before new devices are deployed to a rack.

Read the full incident report →

Started Jun 01, 2026, 02:02 PM UTC· resolved Jun 01, 2026, 06:24 PM UTC

1. investigating Jun 01, 2026, 02:02 PM UTC

   We are experiencing an issue where the US-EAST data center is currently unavailable. We are investigating.

2. investigating Jun 01, 2026, 02:55 PM UTC

   We are continuing to investigate this issue.

3. investigating Jun 01, 2026, 03:30 PM UTC

   We have identified the root cause and are working on implementing a fix.

4. monitoring Jun 01, 2026, 04:02 PM UTC

   Access has been restored to the US-EAST data center. We are monitoring.

5. resolved Jun 01, 2026, 06:24 PM UTC

   After taking remedial action, access to the US-EAST data center is fully restored. This incident is resolved.

6. postmortem Jun 15, 2026, 11:32 PM UTC

   ### **Dates:** Monday, June 1st 2026, 13:06 UTC - 15:42 UTC. ### **What happened:** Customers served by the US-EAST-4 region were unable to authenticate or start new test sessions because an incomplete TLS certificate chain was deployed to the core directory services. ### **Why it happened:** A certificate extraction script defect silently truncated the certificate chain after the certificate authority transitioned to a longer hierarchy. ### **How we fixed it:** Reverted the certificate rotation and re-applied the previous known ,good certificate to restore authentication services. ### **What we are doing to prevent it from happening again:** Implementing pre-deployment certificate chain validation, adding active monitoring, and fixing the script's chain-length limitations.

Read the full incident report →

Started May 21, 2026, 02:53 PM UTC· resolved May 22, 2026, 09:23 AM UTC

1. resolved May 21, 2026, 02:53 PM UTC

   Between May 13th 18:47 UTC and May 15th 9:23 UTC, we experienced test details not appearing in the dashboard of the US-East Data Center. This issue has been resolved. All services are fully operational.

2. postmortem May 26, 2026, 03:07 PM UTC

   ### **Dates:** Wednesday, May 13th 2026, 18:47 UTC - Friday, May 15th 2026, 09:23 UTC. ### **What happened:** Customers were unable to access test run summaries for Real Device Cloud \(RDC\) jobs because events stopped publishing to the jobs Kafka topic in US-EAST. ### **Why it happened:** An authentication key used by the message producer unexpectedly lost its permissions during an account cleanup. ### **How we fixed it:** Manually restored the required permissions to re-establish the connection and resume service. ### **What we are doing to prevent it from happening again:** Migrating to a permanent service account, implementing a Dead Letter Queue \(DLQ\) for the jobs Kafka topic, and replaying the missing events to restore customer data.

Read the full incident report →

Started Apr 24, 2026, 04:33 PM UTC· resolved Apr 25, 2026, 03:29 PM UTC

1. resolved Apr 24, 2026, 04:33 PM UTC

   Between April 23rd 22:44 and April 24th 15:25 UTC, there was a technical issue that affected video recordings for tests running on macOS 15 and iOS within our EU and US-West Data Center. We identified the issue and deployed a fix. All systems are now fully operational.

2. postmortem May 13, 2026, 03:05 PM UTC

   ### **Dates:** Thursday, April 23rd 2026, 22:43 UTC - Friday, April 24th 2026, 15:29 UTC ### **What happened:** Video assets were missing for virtual iOS simulator tests on ARM and macOS ARM desktop tests in the US-West and EU data centers. ### **Why it happened:** A product defect was introduced resulting in a screen capture failure. ### **How we fixed it:** We performed a rollback to a stable version. ### **What we are doing to prevent it from happening again:** We are improving monitoring & alerting to enhance our post deployment validation.

Read the full incident report →

Started Apr 16, 2026, 12:00 AM UTC· resolved Apr 16, 2026, 09:15 AM UTC

1. resolved Apr 16, 2026, 10:10 AM UTC

   Between 02:00 and 11:15 CEST, live and automated tests on iOS 17.0 simulators were failing to start in the EU and US-West Data Center. We executed a deployment rollback, which restored services. All systems are now fully operational.

2. postmortem May 08, 2026, 06:42 PM UTC

   ### **Dates:** Thursday, April 16th 2026, 00:00 UTC – 09:15 UTC ### **What happened:** Live and automated tests on iOS 17.0 simulators failed to start in both the EU and US-West data centers. Customers running tests on iOS 17.0 Intel-based simulators were unable to execute their tests for approximately 9 hours. ### **Why it happened:** A deployment introduced an incompatibility affecting iOS 17.0 on Intel-based infrastructure. The issue was not caught prior to release due to insufficient post-deployment test coverage for that specific simulator configuration. ### **How we fixed it:** We performed a rollback to the previous deployment, which restored full iOS 17.0 simulator functionality. ### **What we are doing to prevent it from happening again:** We are reviving and expanding automated post-deployment tests to cover a broader range of simulator configurations, including legacy Intel-based iOS versions, to catch incompatibilities before they reach production.

Read the full incident report →

Started Apr 07, 2026, 03:02 PM UTC· resolved Apr 07, 2026, 04:13 PM UTC

1. investigating Apr 07, 2026, 03:02 PM UTC

   We are currently investigating reports of test failures affecting users running tests using SauceCtl in our US-West-1 and EU-Central-1 Data Center. We are investigating.

2. resolved Apr 07, 2026, 04:13 PM UTC

   We have identified the root cause and have deployed a fix for this issue. All services are fully operational.

3. postmortem Apr 10, 2026, 09:56 PM UTC

   ### **Dates:** Monday April 7th 2026, ~11:00 – 15:55 UTC ### **What happened:** Some customers experienced 503 errors when running tests via saucectl. The test-composer service was intermittently unavailable, preventing framework-based test execution. ### **Why it happened:** A stale Docker image was deployed to the test-composer service due to a packaging issue that arose during an internal container registry migration. This caused service pods to crash. ### **How we fixed it:** We identified the stale image and redeployed the correct version, restoring the service. ### **What we are doing to prevent it from happening again:** We are hardening our image deployment pipeline and adding validation checks to ensure container registry migrations do not result in stale or incorrect images being deployed to production.

Read the full incident report →

Started Mar 24, 2026, 09:32 AM UTC· resolved Mar 24, 2026, 03:13 PM UTC

1. resolved Mar 24, 2026, 05:36 PM UTC

   Between 09:32 and 15:13 UTC, we identified a technical issue affecting iOS tests when running with network capture enabled. We've resolved the underlying cause and tests are working as expected. All services are fully operational.

2. postmortem Apr 10, 2026, 11:06 PM UTC

   ### **Dates:** Tuesday, March 24th 2026, 09:32 UTC – 15:13 UTC ### **What happened:** Network calls failed on iOS devices during Real Device Cloud sessions where network capture was enabled. Approximately 12-13% of iOS sessions were affected. Android was not impacted. ### **Why it happened:** A deployment introduced a DNS resolution change that was incompatible with the iOS platform, causing network capture to break. ### **How we fixed it:** Rolled back the deployment to restore service. ### **What we are doing to prevent it from happening again:** Adding synthetic tests to catch network capture regressions before production, and implementing monitoring alerts for faster detection after deployments.

Read the full incident report →

Started Mar 19, 2026, 09:51 AM UTC· resolved Mar 19, 2026, 10:54 AM UTC

1. investigating Mar 19, 2026, 09:51 AM UTC

   Around 4:45 AM UTC we started experiencing lower iOS device availability in the US-West data center. Our team is actively investigating the root cause and working toward a resolution.

2. resolved Mar 19, 2026, 10:54 AM UTC

   This incident has been resolved and our services are fully operational.

3. postmortem Apr 10, 2026, 10:59 PM UTC

   ### **Dates:** Wednesday, March 19 2026, 04:45 UTC - 10:47 UTC. ### **What happened:** Approximately 15% of iOS devices in our US-West data center were temporarily unavailable for customer test sessions due to failed internet connectivity checks. ### **Why it happened:** An automated wireless network optimization feature adjusted transmit power levels on access points serving the affected devices, degrading wireless connectivity and causing devices to fail their availability checks. ### **How we fixed it:** The affected access points were identified and restarted, restoring normal wireless connectivity. ### **What we are doing to prevent it from happening again:** Evaluation of the automated optimization tools and a monitoring improvement.

Read the full incident report →

Started Mar 13, 2026, 02:30 PM UTC· resolved Mar 13, 2026, 03:11 PM UTC

1. resolved Mar 18, 2026, 05:15 PM UTC

   Between 14:43 and 15:11 UTC on March 13, a small subset of Real Devices (iOS and Android) became unavailable across all our data centers. After taking remedial action, the issue was identified and resolved. All services are fully operational.

2. postmortem Apr 10, 2026, 10:57 PM UTC

   ### **Dates:** Friday, March 13th 2026, 14:43 UTC - 15:11 UTC. ### **What happened:** Real Devices \(iOS and Android\) availability gradually decreased across all data centers. ### **Why it happened:** A product defect was introduced resulting in a small subset of Real Devices \(~10%\) failing to maintain required connectivity. ### **How we fixed it:** Rollback to a stable version. ### **What we are doing to prevent it from happening again:** Improve monitoring & alerting, enhance post deployment validation.

Read the full incident report →