Wasabi incident

System Errors in CA-CENTRAL-1

Notice Resolved View vendor source →

Wasabi experienced a notice incident on September 18, 2024 affecting CA-Central-1 (Toronto), lasting 40m. The incident has been resolved; the full update timeline is below.

Started
Sep 18, 2024, 06:09 PM UTC
Resolved
Sep 18, 2024, 06:50 PM UTC
Duration
40m
Detected by Pingoru
Sep 18, 2024, 06:09 PM UTC

Affected components

CA-Central-1 (Toronto)

Update timeline

  1. investigating Sep 18, 2024, 06:09 PM UTC

    We are currently experiencing system errors in our CA-CENTRAL-1 region.

  2. monitoring Sep 18, 2024, 06:17 PM UTC

    Our Operations Team identified and corrected the cause of the system errors. Performance to the region is restored and we will continue to monitor the region.

  3. resolved Sep 18, 2024, 06:50 PM UTC

    This incident is now resolved. Please reach out to [email protected] if you continue to see any errors related to this incident.

  4. postmortem Sep 27, 2024, 06:03 PM UTC

    From 2024-09-18 17:55 UTC to 2024-09-18 18:08 UTC, we experienced an issue where client connection attempts to Wasabi Cloud Storage were impacted during an upgrade to our services in the CA-CENTRAL-1 region. The activity had no expectations to create any service impact and began on 2024-09-18 at 13:00 UTC. During the maintenance task, two internal software components had a networking port conflict, causing a delay on the restart process for our load balancer network component. Once our Operations Team identified the condition and cause for the failure, immediate corrective action was taken, and service availability was reestablished.