Cribl experienced a major incident on June 23, 2025 affecting Cribl Lake, lasting 3d. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jun 23, 2025, 06:24 PM UTC
We are currently investigating this issue.
- investigating Jun 23, 2025, 06:26 PM UTC
We're currently investigating an issue with high CPU utilization on small Lakehouses. No customer action required at this time. If you are impacted, you will see delayed or failing searches to Lakehouse.
- investigating Jun 23, 2025, 07:35 PM UTC
Update: We have restarted the impacted Lakehouses, which is allowing them to slowly return to a normal state. There is still no Customer action required at this time. Investigating: We are still in the process of investigating a long-term resolution, and the incident is still active.
- identified Jun 23, 2025, 08:46 PM UTC
The issue has been identified and a fix is being implemented.
- monitoring Jun 23, 2025, 08:48 PM UTC
A fix has been implemented and we are monitoring the results.
- monitoring Jun 24, 2025, 04:01 PM UTC
Update: The impacted tenants have been restarted, and are being actively monitored for the impact-thresholds. We will proactively intervene if necessary, but tenants should be performing as expected. Investigating: Engineering has identified the trigger and is finalizing long-term remediation plans.
- identified Jun 26, 2025, 12:56 PM UTC
The issue has been identified and a fix is being implemented.
- identified Jun 26, 2025, 06:08 PM UTC
Cribl will be performing an Emergency Maintenance at 2pm ET and 4pm ET in order to remediate this Incident. During this time, Customers should expect small windows of slow searches from Cribl Search to Lakehouse datasets during this time. If your Lakehouse does not perform as expected after this time, please contact Cribl Support for assistance.
- monitoring Jun 26, 2025, 06:29 PM UTC
A fix has been implemented and we are monitoring the results.
- resolved Jun 26, 2025, 06:29 PM UTC
This incident has been resolved.