pool.ntp.org experienced a notice incident on September 26, 2025 affecting Management Portal and Public website and 1 more component, lasting 212d 7h. The incident has been resolved; the full update timeline is below.
Affected components
Management PortalPublic websiteDNS updatesGeoDNS serversGlobal NTP ServiceMonitoring System
Update timeline
- investigating Sep 26, 2025, 12:52 AM UTC
The newly reset database cluster (from last night) decided to blow itself up again. I think the extra query load and data from the new monitoring system is making the setup fragile. Investigating.
- identified Sep 26, 2025, 01:25 AM UTC
With a bit of help the kubernetes operator got the cluster working again, at least for now.
- identified Sep 27, 2025, 09:40 PM UTC
The database croaked again! I'm restoring from a ~20 minute old backup.
- monitoring Sep 28, 2025, 01:37 AM UTC
ok, we are stable again -- hopefully.
- resolved Apr 26, 2026, 08:05 AM UTC
System has been migrated to the new infrastructure.