Latitude.sh Outage History

Some devices are being recovered. Our team are still working with the DCO to fully recover the site.

Several servers have been recovered but the full extension of the site is still being worked on at the moment. Our team is still working with the DCO for the full resolution.

We are resolving an increased error rate affecting provisioning actions and virtual machines. It was related to a connection issue with internal services and should be solved within the next 30 minutes. This issue only affects new provisioning action requests. We are truly sorry about the inconvenience.

The issue has been identified and a fix is being implemented.

We are currently experiencing a connectivity issue affecting the dark fiber provider links between our SAO and SAO2 sites. This circuit is responsible for carrying several extended VLANs between the locations, resulting in some devices appearing unavailable. The provider is working on resolving the issue. At the moment, no ETA has been provided. We will continue to monitor and provide updates as soon as new information becomes available.

An ETR has been provided by the dark fiber provider. They are currently working on the issue, and the ETR is 7:00 PM (UTC). The fiber cut was caused by an incident during public road renovation works, which resulted in a major outage affecting several other fiber providers in the region. We will continue to monitor the situation and provide updates as soon as new information becomes available.

We are currently experiencing intermittent issues with server deployments in US regions. The issue affects Ubuntu 24.04, Ubuntu 22.04, and Debian 12 base images, where user data may not be applied correctly during provisioning. Our team is actively investigating and working on a fix. We will provide updates as the situation progresses.

We are continuing to work on a fix for this issue.

We are continuing to work on a fix for this issue.

We have identified the root cause of the issue as an instability on Canonical’s side affecting Ubuntu 24.04 images. This problem impacts new deployments and reinstallations only. Existing servers already running Ubuntu 24.04 are not affected. Our team is monitoring the situation and working on restoring normal deployment operations for Ubuntu 24.04 across the affected regions. As a temporary workaround, we recommend deploying an alternative OS until the issue is fully resolved. Further updates will be shared as more information becomes available.

We are continuing to work on a fix for this issue.

The issue affecting Ubuntu 24.04 deployments has been resolved in the majority of regions. At this time, deployment issues persist only in the ASH (Ashburn) and CHI (Chicago) regions. All other regions are operating normally. Our team continues to work on a fix to fully restore Ubuntu 24.04 deployments in the remaining affected regions. Further updates will be provided as progress is made.

We are continuing to work on a fix for this issue.

We are currently observing increased deployment times and power verification delays in the following regions: US - ASH - LAS - MIA Europe - FRA - LON2 This impact is related to an ongoing global connectivity incident involving the Tier 1 provider Cirion, which is affecting communication paths worldwide. Our engineering team is actively mitigating the issue by adjusting routing within our infrastructure and coordinating route changes with upstream providers. We will continue to share updates as we make progress.

We are aware of an increasing error failure rate for deployments and power actions in Dallas. Our team is already working to solve the issue.

We are continuing to work on a fix for this issue.

We are currently experiencing a connectivity issue affecting the a dark fiber provider links between our SAO and SAO2 sites. Provider has reported overnight issues with their core network equipment, which is causing instability on the inter-site link. This circuit is responsible for carrying several extended VLANs between the locations, resulting in some devices appearing unavailable. Provider is working on resolving the issue, and the estimated time for service restoration is 08:40 BRT. We will continue to monitor and provide updates as soon as new information becomes available.

We are continuing to work on the incident. The dark fiber provider has requested additional time to complete the remediation. We will post another update shortly with a confirmed resolution timeline.

A fix has been implemented and we are currently monitoring the service. Some connections may already be coming back online. We will continue working toward full resolution and provide updates as progress continues.

We are investigating an increased error rate affecting multiple actions in our systems, especially Remote Access, VMs, Custom image deployments, and Databases. It is only affecting the creation of resource actions. Your server is not impacted. We already identified and are working to address this issue as soon as possible, it should take only a few minutes.