crates.io incident

Database upgrade

Minor Resolved View vendor source →

crates.io experienced a minor incident on May 20, 2024 affecting crates.io, lasting 1h 9m. The incident has been resolved; the full update timeline is below.

Started
May 20, 2024, 11:17 AM UTC
Resolved
May 20, 2024, 12:26 PM UTC
Duration
1h 9m
Detected by Pingoru
May 20, 2024, 11:17 AM UTC

Affected components

crates.io

Update timeline

  1. identified May 20, 2024, 11:17 AM UTC

    We are in the process of upgrading our database servers. This will require a short amount of time in read-only mode and a couple of minutes with degraded performance while our follower database is catching up with the new leader.

  2. monitoring May 20, 2024, 12:14 PM UTC

    We noticed a small issue with our database indexes after the upgrade, resulting in several crates not being found anymore. We've reindexed the relevant tables which appears to have fixed the problem. The new follower database is catching up an will soon be available to improve the API response times again.

  3. monitoring May 20, 2024, 12:26 PM UTC

    Database server upgrade is complete and crates.io should be running at full capacity again now. Hopefully even a bit faster than before due to increased memory on the new database servers.

  4. resolved May 20, 2024, 12:26 PM UTC

    This incident has been resolved.