Python Package Index experienced a major incident on May 24, 2025 affecting pypi.org - Backends, lasting 2d 13h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating May 24, 2025, 10:57 PM UTC
We are working to investigate a partial outage in the PyPI backends.
- identified May 24, 2025, 11:11 PM UTC
We have identified a large increase in CDN cache misses causing excess load on our backends. We are working to determine the reason for this increase.
- identified May 24, 2025, 11:37 PM UTC
We have identified a flood of requests from a cluster using the `python-requests/2.32.3` User-Agent, and attempting to hit all JSON project/releasese en masse. This causes excessive cache misses, which are overwhelming our backends. A temporary block has been put in place.
- monitoring May 24, 2025, 11:54 PM UTC
Our block is in place and systems are stable, but we are not satisfied with having to block such a generic user-agent. If you are aware of a system that began a mass crawl of PyPI's JSON release endpoints starting at around 22:05 UTC 2025-05-25, please contact [email protected].
- monitoring May 25, 2025, 12:16 PM UTC
We have removed the temporary block and are continuing to monitor.
- resolved May 27, 2025, 12:53 PM UTC
We have not seen a return of the traffic that caused this incident after removing the temporary block. We will open a new incident if we need to take action again.