Crusoe incident
Certain VM's in us-northcentral1-a are unavailable
Crusoe experienced a minor incident on June 30, 2025 affecting Persistent Storage, lasting 1d 23h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jun 30, 2025, 06:05 PM UTC
We are currently investigating a potential service interruption that may impact VMs to be unreachable in the us-northcentral1-a region. If you are experiencing issues in this region, please reach out to [email protected]
- investigating Jun 30, 2025, 10:01 PM UTC
Our teams are working diligently and believe we have identified the root cause and we are working on implementing a resolution. We will continue to provide updates on this page as further developments occur.
- investigating Jul 01, 2025, 01:34 AM UTC
We are continuing our in-depth investigation into the intermittent storage connectivity issues impacting VM reachability in the us-northcentral1-a region. We will provide more information as it becomes available.
- investigating Jul 01, 2025, 05:21 AM UTC
Our team is continuing its in-depth investigation into the intermittent storage connectivity issues impacting VM reachability in the us-northcentral1-a region, in collaboration with our vendor. We will keep you posted on further developments
- investigating Jul 01, 2025, 10:06 AM UTC
We are continuing our dedicated investigation into the intermittent storage connectivity issues impacting VM reachability in us-northcentral1-a. Our teams are working in close collaboration with our storage vendor to analyze the underlying cause. We recognize the impact this is having and have all necessary resources allocated to resolving this.
- investigating Jul 01, 2025, 02:14 PM UTC
We are seeing signs of recovery, and some previously impacted VMs in the us-northcentral1-a region are now available. Our teams are closely monitoring the environment and continuing their work with our vendor to restore full service for all remaining affected instances.
- monitoring Jul 01, 2025, 08:57 PM UTC
A fix has been applied and the outage has been mitigated. Services have returned to normal operations in us-northcentral1-a region. We are continuing to monitor the situation and validate preventive measures. Please contact [email protected] if you are still experiencing any issues.
- resolved Jul 02, 2025, 05:21 PM UTC
This incident has been resolved.