Litium incident

Certificate issues on serverless Litium Search service

Major Resolved View vendor source →

Litium experienced a major incident on February 26, 2024 affecting Litium Search, lasting 6h 54m. The incident has been resolved; the full update timeline is below.

Started
Feb 26, 2024, 07:49 AM UTC
Resolved
Feb 26, 2024, 02:44 PM UTC
Duration
6h 54m
Detected by Pingoru
Feb 26, 2024, 07:49 AM UTC

Affected components

Litium Search

Update timeline

  1. identified Feb 26, 2024, 07:49 AM UTC

    The issue has been identified and a fix is being implemented.

  2. monitoring Feb 26, 2024, 08:10 AM UTC

    The issue was due to an expired certificate in the chain. We are working on gathering all the facts on how this happened and how we can ensure it does not happen again.

  3. resolved Feb 26, 2024, 02:44 PM UTC

    The issue has been resolved. The fault was due to an error in third-party software responsible for automatically renewing certificates used for the service. We have implemented additional monitoring and alerts to prevent similar occurrences in the future. Configuration changes have also been made to the software in question. We apologize for any inconvenience caused and are committed to ensuring this does not happen again.