Python Package Index incident

Object storage provider outage

Critical Resolved View vendor source →

Python Package Index experienced a critical incident on June 21, 2023 affecting files.pythonhosted.org - Files, lasting 4h 21m. The incident has been resolved; the full update timeline is below.

Started
Jun 21, 2023, 09:38 PM UTC
Resolved
Jun 22, 2023, 01:59 AM UTC
Duration
4h 21m
Detected by Pingoru
Jun 21, 2023, 09:38 PM UTC

Affected components

files.pythonhosted.org - Files

Update timeline

  1. investigating Jun 21, 2023, 09:38 PM UTC

    We are currently investigating an issue with our object storage provider.

  2. identified Jun 21, 2023, 10:46 PM UTC

    We have identified an issue that is impacting latencies for files.pythonhosted.org, resulting in timeouts for a portion of requests.

  3. identified Jun 21, 2023, 10:52 PM UTC

    We're in communication with the object storage provider and are waiting for them to rollout a fix.

  4. identified Jun 21, 2023, 10:57 PM UTC

    We are also implementing our own change to guard against excessive latencies from the provider.

  5. monitoring Jun 21, 2023, 11:45 PM UTC

    Our mitigation has decreased overall latencies, but they remain elevated over normal operation. As our provider's service recovers we expect to see a return to baseline.

  6. resolved Jun 22, 2023, 01:59 AM UTC

    This incident has been resolved.