Wasabi experienced a minor incident on October 31, 2024 affecting EU-Central-1 (Amsterdam), lasting 9h 4m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 31, 2024, 09:44 PM UTC
We have reports of power issues in our EU-CENTRAL-1 region and some users may be experiencing errors when interfacing with their Wasabi buckets hosted there. We will update our status page with more information as it becomes available.
- investigating Oct 31, 2024, 10:21 PM UTC
A facility power distribution issue is impacting several services in the eu-central-1 region. Direct Connect links are impacted as well as a subset of buckets. We continue to work with the facility to restore service.
- identified Oct 31, 2024, 11:15 PM UTC
We have identified the issue and impacted resources. Team continue to work to restore service.
- monitoring Nov 01, 2024, 05:17 AM UTC
The power distribution issue has been resolved and all affected resources are back in service. We are currently monitoring the status. If you see any error, please reach out to [email protected] for assistance.
- resolved Nov 01, 2024, 06:49 AM UTC
The incident has been resolved. Please reach out to [email protected] if you see any issues related to this incident.
- postmortem Nov 08, 2024, 01:24 AM UTC
From 2024-10-31 at 21:10 UTC to 2024-11-01 at 04:00 UTC, we experienced issues with a subset of customer Buckets that were unavailable due to an unexpected power distribution issue while a planned power maintenance window was in progress at the EU-Central-1 region. The maintenance window required to run some servers on a single source and when the switch occurred, the power infrastructure could not keep up with the power draw from some of the infrastructure racks resulting in power down for those racks. The impact resulted in failures in Reads/Write operations and Billing for impacted buckets. Our Operations team worked with the Equinix Datacenter team to restore the power and service availability was re-established as soon as the maintenance was over, and power was completely restored.