TRMNL experienced a critical incident on January 28, 2026 affecting Display API, lasting 1h 1m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jan 28, 2026, 06:19 PM UTC
Today at 12:46pm ET we deployed a domain switchover and our services are catching up.
- identified Jan 28, 2026, 06:48 PM UTC
CPU usage on our API server began spiking at 9a ET for approximately 4 hours and our DNS changes may have exacerbated the latency. We've disabled some logging and are seeing a return to normal levels.
- resolved Jan 28, 2026, 07:21 PM UTC
An infrastructure change to our underlying cache system was made around 9 AM Eastern time, which caused one of our cache entries to start growing without bounds. Eventually this entry got so big it ate up all the CPU time just serializing this value and writing it back out to the cache. The cache for this value was disabled and performance has returned to normal.