Wasabi experienced a minor incident on April 4, 2025 affecting EU-Central-1 (Amsterdam), lasting 3h 57m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Apr 04, 2025, 10:01 AM UTC
We are currently investigating system errors in our EU-CENTRAL-1 region. Our team is working to isolate the issue and provide a resolution.
- investigating Apr 04, 2025, 10:05 AM UTC
We are continuing to investigate this issue.
- monitoring Apr 04, 2025, 12:34 PM UTC
We have identified and resolved the issue. All affected resources are back in service. We are currently monitoring the status. If you see any error, please reach out to [email protected] for assistance.
- resolved Apr 04, 2025, 01:59 PM UTC
This incident has been resolved.
- postmortem Apr 21, 2025, 03:01 PM UTC
On 04 April 2025 from 06:20 UTC to 12:20 UTC, we experienced system errors in our eu-central-1 \(Amsterdam\) region. This issue affected the S3 service and impacted customer’s ability to read and write to/from their Wasabi buckets within the region. Starting at 05:11 UTC, our Operations Team was notified of a spike in internal system read operations in our eu-central-1 region across multiple storage zones. Approximately 1 hour and 9 minutes after this spike at 06:20 UTC, our storage sub-systems errored to read and write requests, impacting customers. Wasabi’s Operations Team was able to identify a recent upgrade to our internal system as responsible for the spike in storage read operations, that eventually led to the storage system errors. Operations proceeded to rollback this upgrade to a previous stable release, which then restored operations on the storage system, leading to a restoration of service for customers. By 12:20 UTC, all S3 services in the eu-central-1 region were operational.