Farm HPC cluster incident
Farm Jobs stuck in SUSPENDED, COMPLETING states
Major Vendor's status page →
Resolved after 1 hour 9 minutes, affecting Storage and Slurm, on September 17, 2024.
This time, the culprit preventing new NFS mounts from happening was identified as nas-4-1. A reboot of that system has brought Farm back into a working state.
Affected components
StorageSlurm
Update timeline
- investigating Sep 17, 2024, 03:09 PM UTC
Jobs are getting stuck in 'COMPLETING' when they should not be. We are currently looking into a root cause.
- resolved Sep 17, 2024, 04:18 PM UTC
This time, the culprit preventing new NFS mounts from happening was identified as nas-4-1. A reboot of that system has brought Farm back into a working state.
Your product runs on more than Farm HPC cluster
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services