The Things Industries incident
Applications Links Failing in eu1 cluster of TTES CH
The Things Industries experienced a major incident on August 25, 2020 affecting Europe 1 (eu1), lasting 7h 10m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Aug 25, 2020, 03:07 AM UTC
We are investigating application linking issues in the Europe (eu1) cluster of our Cloud Hosted service.
- monitoring Aug 25, 2020, 03:49 AM UTC
We're monitoring as applications are reconnecting, and traffic in our Europe (eu1) cluster is recovering.
- identified Aug 25, 2020, 07:27 AM UTC
After further investigation of the incident, we have found that invalid responses from an external service made our Application Server enter an invalid state. At this time all services are operational, but we are monitoring the situation closely while our colleagues are working on a fix.
- resolved Aug 25, 2020, 10:17 AM UTC
We have fixed the issue that caused the Application Server to enter an invalid state when receiving invalid responses from the external service. The fix has been deployed to all clusters.