pool.ntp.org experienced a minor incident on December 19, 2020 affecting Management Portal and DNS updates, lasting 2h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Dec 18, 2020, 01:15 AM UTC
Equinix Metal had a power outage (https://status.equinixmetal.com/incidents/pfgmgy1fnjcp) in the data center where the monitoring and web servers are hosted. One too many of our servers went down making the distributed file system (barely) unavailable. Power is back and the servers should recover soon. When it comes up we'll review if we can increase replication for the critical services. DNS service was unaffected other than having delayed updates from the monitoring system.
- identified Dec 23, 2020, 07:00 PM UTC
This has was all settled earlier; no data was lost but monitoring wasn't functional for a number of hours.
- resolved Dec 23, 2020, 07:01 PM UTC
This incident has been resolved.