Blacksmith incident

Hanging apt package installs on us-west runners due to an upstream mirror issue

Minor Resolved

Blacksmith experienced a minor incident on August 12, 2026 affecting US West (us-west x86), lasting 8h 20m. The incident has been resolved; the full update timeline is below.

Started
Aug 12, 2026, 03:30 AM UTC
Resolved
Aug 12, 2026, 11:50 AM UTC
Duration
8h 20m
Detected by Pingoru
Aug 12, 2026, 03:30 AM UTC

Affected components

US West (us-west x86)

Update timeline

  1. investigating Aug 12, 2026, 03:30 AM UTC

    We are currently investigating this incident.

  2. identified Aug 12, 2026, 09:10 AM UTC

    We have identified the affected mirror and are implementing a fix

  3. identified Aug 12, 2026, 10:33 AM UTC

    We are currently deploying a mitigation

  4. monitoring Aug 12, 2026, 11:26 AM UTC

    We implemented a fix and are seeing improvements. We are continuing to monitor the result.

  5. resolved Aug 12, 2026, 11:50 AM UTC

    This incident has been resolved.