Cadmium experienced a critical incident on October 8, 2024 affecting Registration, lasting 5h 36m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 08, 2024, 09:57 AM UTC
Our team is aware there is a service disruption with the registration module and is working swiftly to identify the root cause.
- identified Oct 08, 2024, 10:58 AM UTC
The root cause has been identified, and our team is working as quickly as possible to fix the issue. Thank you for your patience! We will provide an update as soon as we have more information.
- identified Oct 08, 2024, 01:50 PM UTC
Our team is still working on putting a fix in place. Some registration services may be available. We will provide an update as soon as we have more information.
- monitoring Oct 08, 2024, 02:22 PM UTC
A fix has been put in place, and service has been restored. Our team is continuing to monitor the system.
- resolved Oct 08, 2024, 03:33 PM UTC
The issue has been resolved.
- postmortem Oct 09, 2024, 03:48 PM UTC
This outage occurred when a vendor disabled the Eventscribe Registration Solutions’s Cloudflare account without advance notice. Cadmium staff was alerted immediately upon the sites going offline. To resolve the issue as quickly as possible, DNS records were updated to bypass the Cloudflare account. Once the DNS records propagated across the internet, the registration solution came back online. We apologize for any disruption this caused.