Tornado Datacenter experienced a critical incident on August 13, 2025 affecting Public Grid Power Supply, lasting 24m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Aug 13, 2025, 12:17 PM UTC
Between 14:04:30 CEST and 14:07:30 CEST, we experienced a total blackout of the public power grid in Langen. The UPSes took over the load, we're currently checking the environment.
- identified Aug 13, 2025, 12:29 PM UTC
Public Grid is back since 14:07:30. Temperature increased by up to 28,3°C for 90 seconds until cold water generation returned operational, currently the temperature is down again to around 22°C.
- resolved Aug 13, 2025, 12:42 PM UTC
The UPS batteries are full again. Air Conditioners are operating normally. We're closing this incident. Onsite technicans have fully verified the infrastructure to be normally operational.
- postmortem Aug 13, 2025, 12:56 PM UTC
The last longer going power outage happened in August 2023. Today, whole city Langen was without power from 14:04 till 14:07 CEST. This affected everything which has no backup supply. Tornado Datacenter has diverse UPS for this case, one on A one on B Feed connected to central datacenter power distribution, which can change power sources from public grid \(default mode\) to generator \(backup mode\). Also a emergency generator is available to take over the loads in case of a longer going outage, providing sufficient spare fuel. Load transfer can happen after 20 seconds public grid outage, until that, the UPSes are taking over IT load. Generator fuel delivery is SLA guaranteed. Air Conditioning devices arent UPS protected, which is best practice in a modern datacenter environment. Not to mention the power draw of these, slowness of air temperature changes allow for restarts of cold air / water generation units, without breaching SLA or causing dangerous situations. A restart is caused whenever power is lost and/or transferred. During todays power outage, customers may have noticed a short spike in temperature, which was still below 30°C. Beside of the restart of air conditioning equipment, the infrastructure remained operational.