Blacksmith incident

US west cache failure

Minor Resolved

Blacksmith experienced a minor incident on September 29, 2026 affecting Actions Cache (US West Cache), lasting 1h 32m. The incident has been resolved; the full update timeline is below.

Started
Sep 29, 2026, 05:00 PM UTC
Resolved
Sep 29, 2026, 06:32 PM UTC
Duration
1h 32m
Detected by Pingoru
Sep 29, 2026, 05:00 PM UTC

Affected components

Actions Cache (US West Cache)

Update timeline

  1. investigating Sep 29, 2026, 05:00 PM UTC

    We are investigating failing GitHub Actions cache operations for jobs running in us-west since approximately 16:10 UTC. Affected jobs may see cache restores and saves fail and fall back to a full install, which can make those jobs run longer or fail, while jobs in other regions are not affected.

  2. identified Sep 29, 2026, 05:09 PM UTC

    We have identified the cause of the failures, and we are implementing a fix to restore cache service. Customers with jobs in us-west may still see cache restores and saves fail and fall back to a full install, which can make those jobs run longer or fail, while jobs in other regions are not affected.

  3. monitoring Sep 29, 2026, 05:35 PM UTC

    We have restored the cache infrastructure in us-west, and cache failures have decreased but are not yet back to normal. Jobs in us-west may still see some cache restores and saves fail and fall back to a full install, while the earlier delayed job starts have cleared and other regions are unaffected. We are bringing additional cache capacity online to fully restore service.

  4. monitoring Sep 29, 2026, 06:06 PM UTC

    Cache errors for jobs in us-west have largely subsided. As we complete recovery, some jobs may see a one-time cache miss on their next run. We are monitoring and will provide an update within the next hour.

  5. resolved Sep 29, 2026, 06:32 PM UTC

    This incident is resolved. GitHub Actions cache operations for jobs in us-west have returned to normal, and any jobs that failed during the incident can be re-run.