Resolved after 212 days 7 hours, affecting Management Portal and Public website and 1 more, on September 26, 2025.
ok, we are stable again -- hopefully.
Affected components
Management PortalPublic websiteDNS updatesGeoDNS serversGlobal NTP ServiceMonitoring System
Update timeline
- investigating Sep 26, 2025, 12:52 AM UTC
The newly reset database cluster (from last night) decided to blow itself up again. I think the extra query load and data from the new monitoring system is making the setup fragile. Investigating.
- identified Sep 26, 2025, 01:25 AM UTC
With a bit of help the kubernetes operator got the cluster working again, at least for now.
- identified Sep 27, 2025, 09:40 PM UTC
The database croaked again! I'm restoring from a ~20 minute old backup.
- monitoring Sep 28, 2025, 01:37 AM UTC
ok, we are stable again -- hopefully.
- resolved Apr 26, 2026, 08:05 AM UTC
System has been migrated to the new infrastructure.
Your product runs on more than pool.ntp.org
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services