BCGov incident

CHG0080163 - CITZ MCS GOLDDR - Apply updates and perform maintenance

Minor Resolved View vendor source →

BCGov experienced a minor incident on August 31, 2026, lasting 43d 10h. The incident has been resolved; the full update timeline is below.

Started
Aug 31, 2026, 01:00 PM UTC
Resolved
Oct 13, 2026, 11:50 PM UTC
Duration
43d 10h
Detected by Pingoru
Aug 31, 2026, 01:00 PM UTC

Update timeline

  1. investigating Aug 31, 2026, 12:21 PM UTC

    This work was successfully completed.

  2. scheduled Aug 31, 2026, 01:00 PM UTC

    We will update some software, firmware, and cluster settings and perform a rolling restart of all nodes in the cluster. We will apply the latest firmware on the physical servers. We will upgrade Trident to v26.06.0 and enable concurrency. This has shown great speed improvements to PVC mount/unmount operations in the LABs. Switch the container runtime from runc to crun , as it is the default for OpenShift now. When? Aug 31st to Sept 4th Trident will be upgraded at 6am on the first day of the change. At 9am on the first day of the change we will begin restarting the master nodes first, then the infra nodes, and start on the worker nodes last. Node reboots will continue until 5pm each day, and resume at 9am the next day. Silver will pause over the weekend. Will there be an impact on the Platform apps? As nodes are rebooted pods will be rescheduled to another node. We expect the past issues with Trident taking a long time to start up pods will be solved, but will continue to use slow manual node drains and monitoring of Trident health. Do I need to do anything? Monitor your apps health.