Storj experienced a major incident on May 22, 2025 affecting AP1 - API and EU1 - API and 1 more component, lasting 3h 20m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating May 22, 2025, 02:09 AM UTC
We are investigating reports of performance impacts to one or more components of our system.
- investigating May 22, 2025, 02:09 AM UTC
We are continuing to investigate this issue.
- monitoring May 22, 2025, 02:47 AM UTC
A fix has been implemented and we are monitoring the results.
- resolved May 22, 2025, 05:13 AM UTC
This incident has been resolved.
- postmortem May 23, 2025, 05:20 PM UTC
On May 22, 2025 from 00:12 to 02:54 UTC Storj experienced a service disruption causing elevated error rates and increased request latency that affected the US1 API, Gateway, Linksharing, and US1 - Select services. We discovered lock contention triggered by high-load delete operations. In the short term, we are artificially slowing down some of these intensive requests to leave needed headroom. We will eliminate the lock contention going forward.