Buildkite incident

Increased error rates for Agent API

Minor Resolved View vendor source →

Buildkite experienced a minor incident on May 29, 2025 affecting Agent API, lasting 3h 14m. The incident has been resolved; the full update timeline is below.

Started
May 29, 2025, 01:46 AM UTC
Resolved
May 29, 2025, 05:01 AM UTC
Duration
3h 14m
Detected by Pingoru
May 29, 2025, 01:46 AM UTC

Affected components

Agent API

Update timeline

  1. investigating May 29, 2025, 01:46 AM UTC

    We're seeing elevated error rates for some users and are confirming if this is impacting customers. We will provide a further update in 15 minutes

  2. monitoring May 29, 2025, 02:06 AM UTC

    We have confirmed that the error rate is not significant, but will continue to monitor the situation as we resolve the underlying cause. We will not provide any further updates unless the situation changes.

  3. resolved May 29, 2025, 05:01 AM UTC

    The low level error rates were caused by an increase in database load which is now resolved. We are looking to schedule more regular vacuums to reduce the impact while we work on a long term fix to prevent this issue