Is Crusoe down?

We’re investigating an issue at one of our data center facilities that is affecting a subset of compute instances in us-east2-a. Some instances are currently unavailable. As a safety precaution, we’re powering down some additional hardware in the affected area. Instances on that hardware will become unavailable during this time. Our teams are actively working on this, and we’ll post another update within 30 minutes.

As a safety precaution, we’ve shut off power to the affected hardware in us-east2-a. Instances running on that hardware are now powered off. Power won’t be restored until the area has been confirmed safe. Our teams are working on recovery, and we’ll post another update within 90 minutes.

We’ve started restoring power to part of the affected hardware in us-east2-a, and some instances will begin to come back online. The rest of the affected hardware will stay powered off while the facility issue is being remediated. Our teams are actively working on recovery, and we’ll post updates as events warrant.

Recovery work for the facility issue in us-east2-a is ongoing. The remaining affected hardware will stay powered off until required facility work and safety inspections are complete. We don’t have an estimated restoration time yet, and we’ll share one as soon as those inspections are done. In the meantime, we’re restoring affected instances on other available capacity in us-east2-a where possible, and a number of instances are already back online. If your instances are affected, our customer success team is in contact with you directly. We’ll post another update by 23:00 UTC

Work to address the facility issue in us-east2-a continues to progress. The hardware that's still affected will stay offline until facility repairs and safety checks are finished, and we don't yet have a time for full restoration. Our teams are still moving impacted instances onto other available capacity in us-east2-a where we can. We will post updates on further developments within the next 24 hours.

We have restored part of the affected capacity in us-east2-a and continue to move impacted instances onto other available capacity where we can. The remaining affected hardware will stay offline until facility repairs and safety checks are finished, and we don’t yet have a time for full restoration. We will post an update in 24 hours, or as events warrant.

Work to address the facility issue in us-east2-a continues. The affected hardware will stay offline until facility repairs and safety checks are complete, and it will be brought back online as that work allows. We don’t yet have a time for full restoration. Our teams continue to move impacted instances onto other available capacity in us-east2-a where we can. We will post an update in 24 hours, or as events warrant.

We have restored additional capacity in us-east2-a and are validating each host before returning it to service. Some affected customers are now fully restored, and we continue to return capacity to the remaining impacted customers. The remaining affected hardware will stay offline until facility repairs and safety checks are finished, and we don’t yet have a time for full restoration. We will post an update in 24 hours, or as events warrant.

Most affected instances in our Ohio region are back in service. A small number are still unavailable while we finish hardware checks and the last repairs. We’ll keep you posted on further developments

This incident has been resolved.

We are currently investigating storage accessibility issues when using Shared Storage, Object Storage and Crusoe Container Registry in eu-norway1. We are working with our storage vendor to resolve the issue as soon as possible and will keep you posted on further developments.

We are seeing early signs of recovery for Shared Storage, Object Storage and Crusoe Container Registry in eu-norway1. We are still working closely with our storage vendor to fully restore service and will share another update as progress continues.

Shared Storage, Object Storage and Crusoe Container Registry in eu-norway1 are available again. We have put a fix in place with our storage vendor and are monitoring the results. We will share another update once we confirm the service is stable.

This incident has been resolved.

We're currently investigating an issue affecting a subset of customer VMs in our us-east2 region . A subset of VMs are showing as UNSPECIFIED in our control plane . We have identified the issue and are actively working on fixing it to bring affected VMs back to a healthy state. We will provide more information as soon as it becomes available.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

We identified general network errors affecting service connectivity in our us-east1 region. The issue has been resolved and services are operating normally.

We are investigating "insufficient capacity" errors when creating/starting VMs or creating scaling CMK clusters in us-east2 region. Existing VMs continue to run normally. We'll share an update as soon as we have more information available.

We continue investigating "insufficient capacity" errors when creating/starting VMs or creating scaling CMK clusters in us-east2 region, and have narrowed down a likely cause. We'll share an update as soon as we have more information available.

We continue investigating "insufficient capacity" errors when creating/starting VMs or creating scaling CMK clusters in us-east2 region. We'll share an update as soon as we have more information available.

We continue investigating "insufficient capacity" errors when creating/starting VMs or creating scaling CMK clusters in us-east2 region. We'll share an update as soon as we have more information available.

We've identified the cause of the errors when creating/starting VMs or creating scaling CMK clusters in us-east2 region and applied a fix. We'll continue monitoring.

This incident has been resolved.