2026-September-02 Service Incident

Update timeline

1. investigating Sep 02, 2026, 09:39 AM UTC

   We are seeing elevated error rates for Real Device tests in EU-Central-1 data center. We are investigating.

2. identified Sep 02, 2026, 11:30 AM UTC

   The root cause of the elevated error rates for Real Device tests in the EU-Central-1 data center has been identified. Remediation has been applied, and we are actively monitoring the situation.

3. resolved Sep 02, 2026, 11:53 AM UTC

   After taking remedial action, Real Device test error rates have returned to normal in the EU-Central-1 data center. All services are fully operational.

4. postmortem Sep 18, 2026, 02:11 AM UTC

   **Dates:** Wednesday, September 2nd 2026, 08:30 UTC - 11:15 UTC. ### **What happened:** Real Device Automated Tests experienced failure to start in the EU Central data center, with elevated error rates reaching nearly 100% for short periods of time. ### **Why it happened:** A deployment led to increased cache misses and elevated load on the underlying infrastructure. ### **How we fixed it:** Service was restored by restarting the application. ### **What we are doing to prevent it from happening again:** Additional resources were allocated to reduce the likelihood of a recurrence in the short term. In the long term, we will optimize the performance of resource intensive tasks and introduce dynamic scaling.