The Things Industries incident
Webhook failures in TTSC eu1 cluster
The Things Industries experienced a minor incident on December 19, 2025 affecting Europe 1 (eu1), lasting —. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Dec 19, 2025, 02:50 PM UTC
We are investigating reports about webhook failures in our eu1 cluster.
- monitoring Dec 19, 2025, 03:26 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Dec 19, 2025, 04:53 PM UTC
A significant increase in downlink queue operations within a short time window and this caused internal database connections to accumulate as each request waited for exclusive access to device resources. This connection buildup exhausted the available connection pool, resulting in degraded performance for other operations across the Application Server. Unrelated requests experienced increased latency and timeouts as they competed for the limited available connections. Measures were taken so that this incident do not happen again.