The Things Industries incident
The Things Indoor Gateway Pro connectivity issues
The Things Industries experienced a minor incident on October 16, 2025 affecting Europe 1 (eu1) and Europe 2 (eu2) and 1 more component, lasting 1h 31m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 16, 2025, 11:41 AM UTC
We are currently investigating a connectivity issue with TTIG Pro Gateways.
- identified Oct 16, 2025, 11:42 AM UTC
The issue has been identified, and a fix is being implemented.
- monitoring Oct 16, 2025, 12:44 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Oct 16, 2025, 01:12 PM UTC
This incident has been resolved.
- postmortem Oct 16, 2025, 01:14 PM UTC
The Things Gateway Controller was down intermittently for a total duration of 56 minutes between 2025/10/16 13:12 UTC and 14:42 UTC. This also caused some gateways to disconnect from The Things Stack. The issue started after deploying an update to The Things Gateway Controller. When the newly updated services started, all gateways try to reconnect. Due to the amount of The Things Indoor Gateway Pro gateways in the field, the number of database connections opened to connect the gateways simultaneously exceeded the maximum number of allowed database connections. For connecting one gateway, there are several parallel, independent tasks that execute database queries. Because not enough database connections were available, almost all gateways were suffering because one or more tasks could not complete, causing the gateway to disconnect. This happened in a loop. We temporarily increased the number of allowed database connections so that all gateways could connect. In the meantime, we work on a permanent fix to cap the maximum number of database connections and put proper wait and back-off behavior in place.