Farm HPC cluster incident

nas-4-1 performance issue

Resolved after 1 day 3 hours, affecting Storage, on March 19, 2024.

resolved Mar 20, 2024, 09:14 PM UTC

Additional monitoring shows the performance tuning has evened the IO loads. nas-4-1 may still be occasionally slow as it is being heavily used, but it should be more even over time.

Started
Mar 19, 2024, 05:55 PM UTC
Resolved
Mar 20, 2024, 09:14 PM UTC
Duration
1d 3h
Detected by Pingoru
Mar 19, 2024, 05:55 PM UTC

Affected components

Storage

Update timeline

  1. identified Mar 19, 2024, 05:55 PM UTC

    nas-4-1 is being slammed with traffic. This is impacting everyone who has a home directory or group directory on that server. Admins are working on tunings to reduce the impact.

  2. monitoring Mar 19, 2024, 06:09 PM UTC

    A performance enhancement (faster compression with a slight tradeoff in compression ratio) has been implemented, which is allowing IO to flow more smoothly.

  3. resolved Mar 20, 2024, 09:14 PM UTC

    Additional monitoring shows the performance tuning has evened the IO loads. nas-4-1 may still be occasionally slow as it is being heavily used, but it should be more even over time.

Your product runs on more than Farm HPC cluster

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →