Blacksmith incident

Transparent caching degradaded in eu-west

Minor Resolved

Blacksmith experienced a minor incident on May 20, 2026 affecting Actions Cache, lasting 7d 2h. The incident has been resolved; the full update timeline is below.

Started
May 20, 2026, 01:01 PM UTC
Resolved
May 27, 2026, 03:25 PM UTC
Duration
7d 2h
Detected by Pingoru
May 20, 2026, 01:01 PM UTC

Affected components

Actions Cache

Update timeline

  1. monitoring May 20, 2026, 01:01 PM UTC

    We noticed a small uptick in transparent caching failures in our eu-west region, and have applied a potential fix.

  2. identified May 20, 2026, 02:41 PM UTC

    We have identified a bad host in the fleet and are working on replacing it. Client cache interactions are recovering as we rollout this fix.

  3. identified May 20, 2026, 10:59 PM UTC

    We're currently adding more capacity to the cluster. There will be a few minutes of slow cache interactions while the operation completes.

  4. monitoring May 21, 2026, 12:00 AM UTC

    The cluster expansion is complete. Our testing shows that cache interactions are no longer degraded. We're monitoring the cluster to confirm complete recovery.