TestingBot experienced a major incident on September 5, 2025, lasting —. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Sep 05, 2025, 05:43 AM UTC
Since 1:00 UTC time we have been experiencing elevated wait times, causing some tests to wait for up to 5 minutes before they could start. We deployed a fix and are monitoring the waiting times. They are back to < 10 seconds since 04:30 UTC.
- postmortem Sep 05, 2025, 05:44 AM UTC
Today we experienced long waiting times for Windows \+ macOS machines. The issue was caused by a server issue, where memory issues were causing restarts of our hub \(h[ub.testingbot.com\)](http://hub.testingbot.com) This in turn caused the machines running the tests to fail to communicate with the hub, causing tests from clients to wait up to 5 minutes before a test could start. If your test runner is configured with a http connect timeout < 5 minutes, this may have caused issues where tests would fail to start. The memory issue was resolved and we are looking into a solution to prevent this from happening again.