Ghost Inspector experienced a minor incident on February 4, 2020 affecting Test Runner, lasting 2h 58m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Feb 04, 2020, 05:05 PM UTC
We are investigating reports of failing test results with a future completion date.
- identified Feb 04, 2020, 05:23 PM UTC
We've identified the issue. A specific test running host in our fleet encountered an issue that affected its system clock. This resulted in erroneous dates being assigned to the result and issue being encountered during its execution. We've resolved the host issue. We are now working on cleaning up the erroneous results.
- monitoring Feb 04, 2020, 06:48 PM UTC
We've identified 805 test results system-wide that were run with an erroneous future timestamp over the past 12 hours while the troublesome host was in operation. This affected roughly 0.3% of the tests run during that time period. We have purged these erroneous results from the system due to the problems created by the future timestamp. We apologize if you were affected by this issue. If your test is still in a failing state, we suggest re-running the test to generate a fresh, new result. We will continue monitoring this issue throughout the day and will be working to put additional checks in place to prevent an incident like this from happening again in the future.
- resolved Feb 04, 2020, 08:04 PM UTC
This incident has been resolved.