Wasabi incident

System Errors to access Wasabi services

Minor Resolved View vendor source →

Wasabi experienced a minor incident on March 27, 2025 affecting US-Central-1 (Texas) and US-East-1 (N. Virginia) and 1 more component, lasting —. The incident has been resolved; the full update timeline is below.

Started
Mar 27, 2025, 10:09 PM UTC
Resolved
Mar 27, 2025, 10:09 PM UTC
Duration
Detected by Pingoru
Mar 27, 2025, 10:09 PM UTC

Affected components

US-Central-1 (Texas)US-East-1 (N. Virginia)US-East-2 (N. Virginia)US-West-1 (Oregon)EU-Central-1 (Amsterdam)EU-Central-2 (Frankfurt)EU-West-2 (Paris)AP-Northeast-1 (Tokyo)AP-Northeast-2 (Osaka)Wasabi Management Console

Update timeline

  1. resolved Mar 28, 2025, 01:53 AM UTC

    Between 10:09PM to 10:14PM UTC, we experienced errors with connecting to the Wasabi Web Console, STS, and IAM services. It was a temporary issue, auto-resolved and Operations Team validated it. There is no longer any further impact, and all services are now running without issue.

  2. postmortem Apr 03, 2025, 09:10 PM UTC

    On 27 March 2025 from 10:09 UTC to approximately 10:14 UTC, Wasabi services in our Ashburn \(us-east-1\), Manassas \(us-east-2\), Hillsboro \(us-west-1\), Plano \(us-central-1\), Amsterdam \(eu-central-1\), Tokyo \(ap-northeast-1\), Osaka \(ap-northeast-2\), London \(eu-west-1\), and Frankfurt \(eu-central-2\) regions were unavailable due to an issue, impacting customer traffic and Web Console connectivity. Wasabi Engineering and Operations teams were notified by our automated monitoring systems of an issue with a spike in connection between internal service worker nodes. This spike led to the inability to open new connections within the services. Wasabi’s Operations Team manually restarted the services, which corrected the issue and allowed for client connections to resume successfully. By 10:14 UTC, all internal services were restarted and all services returned to normal operating conditions.