Trustgrid Public experienced a minor incident on January 6, 2026 affecting Software Repository, lasting 3d 3h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jan 06, 2026, 07:47 PM UTC
Out of an abundance of caution, we have made the package repository unavailable while verifying some configuration changes.
- investigating Jan 06, 2026, 10:27 PM UTC
We are continuing to investigate this issue.
- identified Jan 06, 2026, 10:28 PM UTC
We have identified an issue that could cause a small subset of node appliances to fail during updates. As a preventative measure, we have temporarily taken the package repository offline. As a result, the following impacts are expected: - Appliance node upgrades will fail while the repository is unavailable. Customers should avoid initiating upgrades during this time. - Appliance nodes may report Repo Connectivity health issues. These are expected to resolve once the repository is restored. - Container images cannot be pushed or pulled. New containers will not start if they require an image pull. Existing running containers are not impacted and can restart without issue, provided the container version tag is not modified. - Container exec output files will not upload. - Debug logs will not be published. We are actively working to determine the safest path forward and will restore repository access as soon as possible.
- identified Jan 07, 2026, 02:22 AM UTC
We are continuing to work on a fix for this issue.
- monitoring Jan 07, 2026, 02:24 AM UTC
We are beginning to restore access to the repository. As part of this process, access to apt packages will remain deliberately disabled to prevent appliance updates. This approach allows other repository-dependent functionality to recover while we continue to test and validate a fix for the issue impacting a specific subset of appliance models (for example, Lanner NCA-1210 running Ubuntu 22.04 LTS). Current impact during this phase: - Appliance nodes will not be able to install or update apt packages. - Appliance node upgrades will continue to fail and should not be initiated. Other repository-backed functionality will begin to recover as repository access is restored. We will keep apt package access disabled until a validated release is available that resolves the issue for the affected models. Further updates will be provided as testing progresses. This will is expected to be the last status update for the evening.
- monitoring Jan 07, 2026, 03:25 PM UTC
We are continuing to monitor for any further issues.
- monitoring Jan 07, 2026, 03:25 PM UTC
We intend to remain in this state with repository access restored but apt package updates disabled until we can fully test and release a version that addresses the underlying issue impacting the affected appliance models. We expect this state to remain in place for several days while we complete testing and prepare a validated release. During this period: - Apt package installs and appliance upgrades will remain unavailable and should not be initiated. - Appliance nodes will report Repo Connectivity health check failures while apt access is intentionally disabled. - Other repository-dependent functionality will continue to operate normally. We will provide additional updates as testing completes and a validated release becomes available. We appreciate your patience while we work to resolve this issue safely.
- resolved Jan 09, 2026, 11:03 PM UTC
The repository service has now been fully restored. A new node version has been released that prevents issues upgrading the impacted devices. Customers may continue to see Repo Connectivity health check warnings on appliance nodes running older versions of the software. These warnings are expected and may take up to 24 hours to clear as systems fully reconcile and refresh connectivity state. We appreciate your patience during this incident.