UTHPC incident

Newly Disclosed Linux kernel Security Incident – Temporary Service Interruptions

Major Resolved

UTHPC experienced a major incident on July 23, 2026 affecting rocket.hpc.ut.ee and UT HPC webservices (hpc.ut.ee) and 1 more component, lasting 23h 30m. The incident has been resolved; the full update timeline is below.

Started
Jul 23, 2026, 07:24 AM UTC
Resolved
Jul 24, 2026, 06:54 AM UTC
Duration
23h 30m
Detected by Pingoru
Jul 23, 2026, 07:24 AM UTC

Affected components

rocket.hpc.ut.eeUT HPC webservices (hpc.ut.ee)UT HPC webservices (docs.hpc.ut.ee)Services (Galaxy)UT HPC webservices (support.hpc.ut.ee)Services (RStudio)Services (Open OnDemand)UT HPC webservices (registry.hpc.ut.ee)

Update timeline

  1. investigating Jul 23, 2026, 07:24 AM UTC

    We are currently responding to a newly disclosed Linux kernel security vulnerability affecting the XFS filesystem. As a precaution, we are implementing mitigation measures across affected infrastructure. **Affected services:** * Web Servers * Rocket Cluster, along with OpenOndemand * SAPU * Cloud * Galaxy To reduce risk while mitigation work is underway, **Cluster access has been temporarily disabled**. You may experience temporary service interruptions across the affected services during this maintenance. Our teams are actively applying mitigations and validating service integrity. We will provide further updates as work progresses and restore normal access as soon as it is safe to do so. Thank you for your patience and understanding.

  2. identified Jul 23, 2026, 07:24 AM UTC

    We are continuing to work on a fix for this incident.

  3. identified Jul 23, 2026, 11:13 AM UTC

    We have made progress in addressing the recently disclosed Linux kernel security vulnerability. The following services have now been restored and are available: * Cloud Service * Web Server (Website Hosting Service) Mitigation work is still ongoing for the following services: * Rocket Cluster and related services such as Galaxy, Open OnDemand * SAPU These services will remain unavailable while we complete the required patching and validation to ensure they can be restored safely. We will continue to provide updates as additional services become available.

  4. resolved Jul 24, 2026, 06:54 AM UTC

    All services previously affected by the Linux kernel security vulnerability impacting the XFS filesystem were restored by midnight on **23 July**. The current service status is as follows: * All **SAPU** systems have been restored and are fully available. * Web servers are up. * Galaxy is available. * OpenOndemand and its services, RStudio and Jupyter, are available. * The **Rocket Cluster** is available for compute workloads. * A small number of older **Rocket Cluster V100 GPU nodes** remain temporarily isolated while they undergo the final upgrade. These nodes are expected to be returned to the queue later today. Thank you for your patience while we implemented the necessary mitigations to ensure the security and stability of our services.