Litium incident

Serverless Cloud Site Disruptions

Major Resolved View vendor source →

Litium experienced a major incident on December 17, 2024 affecting Cloud API and Litium Search and 1 more component, lasting 16h 26m. The incident has been resolved; the full update timeline is below.

Started
Dec 17, 2024, 03:06 PM UTC
Resolved
Dec 18, 2024, 07:33 AM UTC
Duration
16h 26m
Detected by Pingoru
Dec 17, 2024, 03:06 PM UTC

Affected components

Cloud APILitium SearchDatabase services

Update timeline

  1. investigating Dec 17, 2024, 03:06 PM UTC

    We are currently experiencing issues affecting some sites in our serverless cloud environment. Our team is actively investigating the cause and working to resolve the problem. We will provide an update as soon as more information is available. Thank you for your patience.

  2. investigating Dec 17, 2024, 03:53 PM UTC

    We are still actively working to resolve the issue affecting some sites in our serverless cloud environment. Our team is making progress, and we will share further updates as soon as possible. Thank you for your continued patience and understanding.

  3. identified Dec 17, 2024, 04:58 PM UTC

    We have identified the cause of the issue affecting some sites in our serverless cloud environment and are now working on implementing a solution. We will provide further updates as soon as we have more information. Thank you for your patience and understanding.

  4. identified Dec 17, 2024, 06:18 PM UTC

    We keep working on resolving the issue affecting sites in our serverless cloud. We will provide an update as soon as more information is available. Thank you for your patience.

  5. identified Dec 17, 2024, 07:28 PM UTC

    Sites are up and running since 19:46 but might still experience some intermittent issues. We are currently working on stabilizing the services. We will provide an update as soon as more information is available. Thank you for your patience.

  6. monitoring Dec 17, 2024, 07:58 PM UTC

    The services are fully operational and back to a normal state. We will continue to monitor the situation and investigate the root cause of the incident. If no further issues arise, the next update will be provided tomorrow morning. Thank you for your patience.

  7. resolved Dec 18, 2024, 07:33 AM UTC

    We have been monitoring overnight, and all services are now normal. The issue was related to an unexpected problem with the control plane in our main application cluster. We will continue to investigate the root cause to ensure this does not happen again. We apologize for the inconvenience this has caused and thank you for your patience during this incident.