usegalaxy.org datasets and jobs are currently not available

Update timeline

1. investigating Aug 20, 2021, 03:59 AM UTC

   User data is currently not accessible and jobs are not executing on usegalaxy.org. The filesystem housing datasets is currently not accessible to Galaxy systems due to performance problems. This is believed to be due to issues with the networking infrastructure. Engineers at our host (the Texas Advanced Computing Center, TACC) are investigating the problem. We do have an estimate on how long the service will be affected, but will post updates when they are available. Galaxy Europe (usegalaxy.eu) and Galaxy Australia (usegalaxy.org.au) are unaffected and can be used in the interim. These servers use separate accounts and do not share data, so you will need to create accounts and upload data as needed.

2. identified Aug 20, 2021, 04:17 PM UTC

   Engineers at our host believe they have identified some bad network hardware responsible for the performance problems that were occurring, and are working on a solution. We do not yet have a time estimate on how long the service will be affected, but will continue to post updates when they are available.

3. identified Aug 20, 2021, 04:37 PM UTC

   Access to user data has been restored, but job execution is still paused as engineers continue to work on a full resolution to the issue.

4. monitoring Aug 20, 2021, 06:35 PM UTC

   Jobs are executing again and we are monitoring for any repeat issues.

5. resolved Aug 20, 2021, 08:54 PM UTC

   This incident has been resolved.