Crusoe incident

Certain VMs in us-northcentral1-a are unavailable

Major Resolved View vendor source →

Crusoe experienced a major incident on April 28, 2025 affecting us-northcentral1, lasting 8h 26m. The incident has been resolved; the full update timeline is below.

Started
Apr 28, 2025, 03:01 PM UTC
Resolved
Apr 28, 2025, 11:27 PM UTC
Duration
8h 26m
Detected by Pingoru
Apr 28, 2025, 03:01 PM UTC

Affected components

us-northcentral1

Update timeline

  1. investigating Apr 28, 2025, 03:01 PM UTC

    We are currently investigating an issue impacting virtual machine accessibility in us-northcentral1-a. Some users may be unable to access their VMs at this time. We appreciate your patience and will provide updates as more information becomes available.

  2. identified Apr 28, 2025, 06:15 PM UTC

    We are actively working with our storage vendor to address the issue. We have identified the underlying cause and are beginning to see some VMs successfully return to a RUNNING state. Troubleshooting efforts are ongoing, and we will continue to provide updates as progress is made.

  3. monitoring Apr 28, 2025, 07:31 PM UTC

    All impacted instances have now successfully returned to a Running state. No further impact is expected. We appreciate your patience and understanding while we worked through this issue.

  4. resolved Apr 28, 2025, 11:27 PM UTC

    All impacted instances have successfully returned to a running state. A fix has been applied across the us-northcentral1 region to prevent this issue from recurring. Please reach out to [email protected] if you experience any issues.