Splunk incident

Incident Review | CINC-56605 Major Outage

Critical Resolved View vendor source →

Splunk experienced a critical incident on May 23, 2024 affecting Search, lasting 2h 36m. The incident has been resolved; the full update timeline is below.

Started
May 23, 2024, 09:41 PM UTC
Resolved
May 24, 2024, 12:18 AM UTC
Duration
2h 36m
Detected by Pingoru
May 23, 2024, 09:41 PM UTC

Affected components

Search

Update timeline

  1. investigating May 23, 2024, 05:28 PM UTC

    We are investigating an issue where the error “Unknown error: Failed to fetch from KV Store” is occurring on the Investigations tab of the Enterprise Security app for several Splunk cloud platform customers. We will provide an update within the next 30 minutes.

  2. identified May 23, 2024, 05:46 PM UTC

    We have identified the source of the issue and are currently working on implementing a fix within the next 2 hours. Impacted customers may continue to see error: “Unknown error: Failed to fetch from KV Store” occurring on the Investigations tab of the Enterprise Security application. Within the next 2 hours, customers may also see improvement in their environment. Your patience is greatly appreciated and we will provide more updates as we implement the fix.

  3. identified May 23, 2024, 07:17 PM UTC

    We are continuing to make progress towards resolution and will provide more updates as we implement the fix.

  4. identified May 23, 2024, 08:40 PM UTC

    We are continuing to work on a fix for this issue.

  5. identified May 23, 2024, 09:41 PM UTC

    We are continuing to work on a fix for this issue.

  6. identified May 23, 2024, 10:42 PM UTC

    We are continuing to work on a fix for this issue.

  7. monitoring May 23, 2024, 11:34 PM UTC

    A fix has been implemented and we are monitoring the results.

  8. resolved May 24, 2024, 12:18 AM UTC

    This incident has been resolved.