Ubicloud Outage History

Ubicloud degraded · 1 active incident View live status →

Ubicloud had 16 outages in the last 2 years totaling 27h 30m of downtime — averaging 0.7 incidents per month.

There were 16 Ubicloud outages since June 5, 2025 totaling 27h 30m of downtime. Each is summarised below — incident details, duration, and resolution information.

Source: https://status.ubicloud.com

Minor May 19, 2026

Degraged GitHub Webhooks

Detected by Pingoru
May 19, 2026, 03:05 PM UTC
Resolved
May 19, 2026, 05:31 PM UTC
Duration
2h 25m
Affected: GitHub Webhooks
Timeline · 2 updates
  1. identified May 19, 2026, 03:05 PM UTC

    At approximately 14:55 UTC, we observed a drop in incoming GitHub webhooks as GitHub temporarily stopped sending events. Delivery has since partially recovered. Our system automatically detects missing webhooks every 5 minutes and self-heals by reconciling the missed events. As a result, you may experience longer-than-usual job assignment times until delivery fully stabilizes. We are continuing to monitor the situation.

  2. resolved May 19, 2026, 05:31 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor May 6, 2026

Increased provisioning times for large runner sizes (16 and 30 vCPUs)

Detected by Pingoru
May 06, 2026, 07:05 PM UTC
Resolved
May 05, 2026, 07:05 PM UTC
Duration
—
Affected: Ubicloud Runners
Timeline · 2 updates
  1. identified May 05, 2026, 10:24 AM UTC

    We're seeing higher than usual demand for our larger runner sizes (16 and 30 vCPUs). This is causing longer than usual provisioning times. We're working to increase our capacity as quickly as possible and mitigate the issue.

  2. resolved May 06, 2026, 03:25 AM UTC

    Following an initial capacity increase, provisioning times have returned to normal. We're continuing to expand our fleet to meet future demand.

Read the full incident report →

Minor March 20, 2026

Networking issue impacting multiple services

Detected by Pingoru
Mar 20, 2026, 10:34 AM UTC
Resolved
Mar 20, 2026, 11:00 AM UTC
Duration
25m
Affected: PostgreSQLNetworking
Timeline · 3 updates
  1. investigating Mar 20, 2026, 10:34 AM UTC

    We are currently investigating issues which mostly impacts Postgres services.

  2. investigating Mar 20, 2026, 10:35 AM UTC

    We are continuing to investigate this issue.

  3. resolved Mar 20, 2026, 11:55 AM UTC

    This incident has been resolved.

Read the full incident report →

Minor March 6, 2026

High job provisioning delays for arm64 GitHub runners

Detected by Pingoru
Mar 06, 2026, 01:58 PM UTC
Resolved
Mar 06, 2026, 02:56 PM UTC
Duration
58m
Affected: Ubicloud Runners
Timeline · 2 updates
  1. identified Mar 06, 2026, 01:58 PM UTC

    Due to an unexpected spike in arm64 demand, there is currently a delay in provisioning arm64 runners. We are working to increase our capacity as quickly as possible to meet the demand and mitigate the issue.

  2. resolved Mar 06, 2026, 02:56 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor February 25, 2026

Occassional DNS Resolution Failures

Detected by Pingoru
Feb 25, 2026, 09:15 PM UTC
Resolved
Feb 25, 2026, 10:50 PM UTC
Duration
1h 34m
Affected: ComputePostgreSQLNetworkingKubernetes
Timeline · 2 updates
  1. identified Feb 25, 2026, 09:37 PM UTC

    DNS queries can fail sporadically, linked to a UDP-related fault at Hetzner https://status.hetzner.com/incident/5e901309-0873-42af-90c5-84f38304f592 We're monitoring and looking to work around this issue.

  2. resolved Feb 25, 2026, 10:55 PM UTC

    Caused by a DDOS attack against Hetzner and its mitigation routines. We will be diversifying our DNS upstreams in the future.

Read the full incident report →

Minor February 13, 2026

Erratic restarts on some VMs and Databases

Detected by Pingoru
Feb 13, 2026, 07:38 PM UTC
Resolved
Feb 13, 2026, 08:16 PM UTC
Duration
38m
Affected: ComputePostgreSQLKubernetes
Timeline · 2 updates
  1. monitoring Feb 13, 2026, 08:13 PM UTC

    Some systems have been restarting erratically due to a recent code change. We backed this out and are recovering.

  2. resolved Feb 13, 2026, 08:16 PM UTC

    All systems operational again. Write to [email protected] about anything unexpected.

Read the full incident report →

Major February 6, 2026

Postgres DNS resolution issues

Detected by Pingoru
Feb 06, 2026, 08:00 PM UTC
Resolved
Feb 06, 2026, 10:13 PM UTC
Duration
2h 13m
Affected: PostgreSQL
Timeline · 5 updates
  1. investigating Feb 06, 2026, 08:33 PM UTC

    There is a defect in Postgres that causes unbound DNS records. Negative caching can exacerbate the incident after our fix. Databases affected: many at the outset, some now. Diagnosing the last records.

  2. monitoring Feb 06, 2026, 09:16 PM UTC

    We've hotfixed the remaining records. Negative caching of DNS can slow recovery, depending on the upstream DNS server. We will remain in contact with all affected customer until all symptoms are resolved.

  3. identified Feb 06, 2026, 09:35 PM UTC

    Some records are proving more difficult to address than we expected.

  4. monitoring Feb 06, 2026, 09:41 PM UTC

    We've identified this problem, and believe negative caching of DNS records to be all that remains. We are going to continue to check for at least thirty minutes, the longest time reported for negative caching so far.

  5. resolved Feb 06, 2026, 10:13 PM UTC

    Cross checking DNS records has been successful for some time. Customers report negative caches have since expired. Likely cause: race condition that caused misapplication of DNS records that was exposed under higher parallelism. We have to analyze it and patch it in the coming days.

Read the full incident report →

Major January 28, 2026

Issues with Postgres databases

Detected by Pingoru
Jan 28, 2026, 11:58 PM UTC
Resolved
Jan 29, 2026, 12:24 AM UTC
Duration
25m
Affected: PostgreSQL
Timeline · 3 updates
  1. investigating Jan 28, 2026, 11:58 PM UTC

    We're currently investigating. Provisioning affected, as is responsiveness to failures. False positives may also cause additional downtime. We will keep you posted.

  2. investigating Jan 29, 2026, 12:07 AM UTC

    The problem was a fault in monitoring. The reporting is improving, we're watching for any systems that have been negatively affected. There may be some extra false positive restarts in this time.

  3. resolved Jan 29, 2026, 12:24 AM UTC

    We believe all errata at this point to be resolved. There are some clear causes and room for improvement, mostly in graceful degradation in some metrics components. Please write in immediately if anything is out of place. Our apologies.

Read the full incident report →

Minor January 19, 2026

A host in eu-central-h1 is experiencing a hardware failure

Detected by Pingoru
Jan 19, 2026, 02:26 PM UTC
Resolved
Jan 19, 2026, 03:13 PM UTC
Duration
47m
Affected: ComputePostgreSQL
Timeline · 2 updates
  1. identified Jan 19, 2026, 02:26 PM UTC

    Instances running on this host are currently unreachable. We are actively working to mitigate the issue and restore services. We will keep you updated as we make progress.

  2. resolved Jan 19, 2026, 03:13 PM UTC

    Our hardware provider has replaced the failed hardware, and all systems are now fully operational.

Read the full incident report →

Minor December 15, 2025

Increased job queue times in Ubicloud runners

Detected by Pingoru
Dec 15, 2025, 01:37 PM UTC
Resolved
Dec 15, 2025, 05:07 PM UTC
Duration
3h 30m
Affected: Ubicloud Runners
Timeline · 3 updates
  1. investigating Dec 15, 2025, 01:37 PM UTC

    Ubicloud runners are currently experiencing a partial outage, resulting in high queue for job requests. We are investigating the incident.

  2. monitoring Dec 15, 2025, 02:11 PM UTC

    We've mitigated the issue and are actively monitoring the results. Queue times have started to improve, but due to the high load caused by the incident, some delays are still expected. We’ll share another update once everything is fully resolved. Thank you for your patience.

  3. resolved Dec 15, 2025, 05:07 PM UTC

    Our provisioning times are back to normal.

Read the full incident report →

Notice November 18, 2025

Ubicloud Website experiencing issues

Detected by Pingoru
Nov 18, 2025, 01:18 PM UTC
Resolved
Nov 18, 2025, 02:55 PM UTC
Duration
1h 37m
Timeline · 3 updates
  1. identified Nov 18, 2025, 01:18 PM UTC

    Ubicloud website uses Cloudflare WAF. The incident https://www.cloudflarestatus.com/incidents/8gmgl950y3h7 is causing downtime on the web pages of Ubicloud.

  2. monitoring Nov 18, 2025, 02:44 PM UTC

    A fix has been implemented and we are monitoring the results.

  3. resolved Nov 18, 2025, 02:55 PM UTC

    This incident has been resolved.

Read the full incident report →

Minor September 26, 2025

Degraded Performance in GitHub API

Detected by Pingoru
Sep 26, 2025, 02:32 PM UTC
Resolved
Sep 26, 2025, 03:39 PM UTC
Duration
1h 6m
Affected: GitHub API Requests
Timeline · 2 updates
  1. investigating Sep 26, 2025, 02:32 PM UTC

    Our monitors have detected an increased number of "503 - github-launch service unavailable" errors from the GitHub API over the past 30 minutes. This affects the time it takes for jobs to be picked up by runners. We believe this may be due to an undeclared GitHub incident and are closely monitoring the situation.

  2. resolved Sep 26, 2025, 03:39 PM UTC

    GitHub API has stopped returning HTTP 503 errors, and operations are back to normal.

Read the full incident report →

Minor September 15, 2025

Elevated reports of pulling images from GitHub Container Registry (ghcr.io)

Detected by Pingoru
Sep 15, 2025, 05:15 PM UTC
Resolved
Sep 16, 2025, 12:13 AM UTC
Duration
6h 57m
Affected: Ubicloud Runners
Timeline · 4 updates
  1. investigating Sep 15, 2025, 09:18 PM UTC

    We've received several reports of degraded performance when pulling Docker images from ghcr.io. Our team is currently investigating the issue.

  2. investigating Sep 15, 2025, 09:31 PM UTC

    We have reproduced the issue on virtual machines from other cloud providers. Several other platform providers have reported active incidents with ghcr.io, primarily affecting European clients.

  3. monitoring Sep 15, 2025, 11:17 PM UTC

    Recent tests show that connectivity with ghcr.io is fast once more. We will continuing monitoring the situation, and if we confirm continued function, will resolve the incident.

  4. resolved Sep 16, 2025, 12:13 AM UTC

    We have been unable to reproduce the issues with repeated speed tests for an hour. Marking as resolved: please write to [email protected] if you are encountering sustained issues.

Read the full incident report →

Minor June 16, 2025

Ubicloud Cache has degraded performance

Detected by Pingoru
Jun 16, 2025, 12:59 PM UTC
Resolved
Jun 16, 2025, 04:00 PM UTC
Duration
3h
Affected: Ubicloud Cache
Timeline · 2 updates
  1. investigating Jun 16, 2025, 12:59 PM UTC

    We’re currently investigating an issue affecting the performance of Ubicloud Cache. You may experience slower cache downloads or occasional timeouts due to degraded performance in the underlying infrastructure. You can email [email protected] if you encounter any issues. We’ll keep you updated as we learn more. Thanks for your patience!

  2. resolved Jun 16, 2025, 09:56 PM UTC

    Due to intermittent failures on some Cloudflare R2 buckets used for saving cached files, a few users experienced occasional timeouts. Operations appear to be back to normal now.

Read the full incident report →

Minor June 5, 2025

High job provisioning delays in GitHub Actions

Detected by Pingoru
Jun 05, 2025, 01:53 PM UTC
Resolved
Jun 05, 2025, 03:44 PM UTC
Duration
1h 50m
Affected: Ubicloud Runners
Timeline · 2 updates
  1. investigating Jun 05, 2025, 01:53 PM UTC

    We are currently investigating this issue.

  2. resolved Jun 05, 2025, 03:44 PM UTC

    Our provisioning times are back to normal.

Read the full incident report →