Buildkite incident

Delays in flaky test identification

Minor Resolved View vendor source →

Buildkite experienced a minor incident on June 12, 2025 affecting Web, lasting 40m. The incident has been resolved; the full update timeline is below.

Started
Jun 12, 2025, 02:26 AM UTC
Resolved
Jun 12, 2025, 03:06 AM UTC
Duration
40m
Detected by Pingoru
Jun 12, 2025, 02:26 AM UTC

Affected components

Web

Update timeline

  1. monitoring Jun 12, 2025, 02:26 AM UTC

    We are experiencing delays in identifying flaky test results. We have identified the root cause and are working through the backlog of flaky tests executions and expect this issue to be resolved shortly.

  2. resolved Jun 12, 2025, 03:06 AM UTC

    The backlog of flaky test identification has been processed and we are no longer seeing delays in this service. We will be making changes to our auto-scaling configuration to mitigate against future occurrences of this issue.