SchemeServe incident

SchemeServe connectivity issues

Critical Resolved View vendor source →

SchemeServe experienced a critical incident on January 6, 2024 affecting 🎩 SchemeServe and 🔗 SchemeServe API and 1 more component, lasting 45m. The incident has been resolved; the full update timeline is below.

Started
Jan 06, 2024, 12:40 AM UTC
Resolved
Jan 06, 2024, 01:25 AM UTC
Duration
45m
Detected by Pingoru
Jan 06, 2024, 12:40 AM UTC

Affected components

🎩 SchemeServe🔗 SchemeServe API🔗 Legacy API🗃️ Batch Renewals📈 Reports

Update timeline

  1. investigating Jan 06, 2024, 12:40 AM UTC

    SchemeServe is currently experiencing connectivity issues affecting all services.

  2. resolved Jan 06, 2024, 01:25 AM UTC

    This incident has been fully resolved and SchemeServe is fully functioning again. A full post mortem will follow shortly.

  3. postmortem Jan 06, 2024, 01:27 AM UTC

    SchemeServe’s primary edge SSL certificates were due to enter their renewal period this evening 30 days before expiry, this process did not function correctly and was unable to renew the certificate. If this occurs the process should log the event and retry 24 hours later, however, instead the certificate was automatically removed from circulation. Upon identification of the issue, a new certificate was issued and all affected services were accessible again. We will be investigating why the process did not execute correctly. ‌ **Issued first occurred:** 00:00:17 **Issue identified:** 00:38:10 **Issue remediated:** 01:09:47