h8lio incident

[ci] woodpecker is failing to clone git sources

Major Resolved View vendor source →

h8lio experienced a major incident on April 24, 2023 affecting ci.h8l.io, lasting 5h 57m. The incident has been resolved; the full update timeline is below.

Started
Apr 24, 2023, 09:25 AM UTC
Resolved
Apr 24, 2023, 03:23 PM UTC
Duration
5h 57m
Detected by Pingoru
Apr 24, 2023, 09:25 AM UTC

Affected components

ci.h8l.io

Update timeline

  1. investigating Apr 24, 2023, 09:25 AM UTC

    We are currently investigating this issue.

  2. identified Apr 24, 2023, 12:30 PM UTC

    The last version of Docker used to execute the CI pipelines on the build dedicated instances has a bug on Ubuntu and fails to use the host network and its DNS configuration. We are downgrading the Docker on the CI instances.

  3. monitoring Apr 24, 2023, 12:57 PM UTC

    The CI instances have been downgraded from version 5:23.0.4-1~ubuntu.22.04~jammy to version 5:23.0.1-1~ubuntu.22.04~jammy of Docker. The auto update of the instances has been disabled. We are monitoring the CI instances.

  4. resolved Apr 24, 2023, 03:23 PM UTC

    This incident has been resolved.