Ghost Inspector incident

Test runner queue surges

Minor Resolved View vendor source →

Ghost Inspector experienced a minor incident on July 17, 2023 affecting Test Runner, lasting 2h 3m. The incident has been resolved; the full update timeline is below.

Started
Jul 17, 2023, 03:46 PM UTC
Resolved
Jul 17, 2023, 05:49 PM UTC
Duration
2h 3m
Detected by Pingoru
Jul 17, 2023, 03:46 PM UTC

Affected components

Test Runner

Update timeline

  1. investigating Jul 17, 2023, 03:46 PM UTC

    We are currently experiencing a surge in both the Chrome and Firefox queues and are investigating the cause, customers may experience some delays in test runs with both browsers. We will follow up with an update as soon as we have more information

  2. identified Jul 17, 2023, 04:21 PM UTC

    The Firefox capacity issue has been addressed and we are seeing queue levels return to normal, customers should see their enqueued tests complete shortly. Chrome queue volume is still high and we are provisioning additional capacity to help bring levels back to normal. We will update with more information again shortly.

  3. resolved Jul 17, 2023, 05:49 PM UTC

    Queue depths are back to normal and test runners should be operating as expected. any customers still experiencing issues are asked to contact support.