Splunk Observability Cloud US2 incident

Splunk RUM Troubleshooting MetricSets and Monitoring Metricsets are being dropped

Major Resolved View vendor source →

Splunk Observability Cloud US2 experienced a major incident on February 2, 2025 affecting Splunk RUM API and Splunk RUM Monitoring MetricSets and 1 more component, lasting 2h 20m. The incident has been resolved; the full update timeline is below.

Started
Feb 02, 2025, 08:21 PM UTC
Resolved
Feb 02, 2025, 10:42 PM UTC
Duration
2h 20m
Detected by Pingoru
Feb 02, 2025, 08:21 PM UTC

Affected components

Splunk RUM APISplunk RUM Monitoring MetricSetsSplunk RUM Troubleshooting MetricSets

Update timeline

  1. investigating Feb 02, 2025, 08:54 PM UTC

    A degradation in the Splunk RUM metrics processing pipeline is causing Monitoring MetricSets to be dropped. Alerts, charts, and detectors built from Monitoring MetricSets are impacted. A degradation in the performance of the Splunk RUM trace processing pipeline is causing Troubleshooting MetricSets to be delayed by more than fifteen minutes. As a result, the RUM Troubleshooting experience does not have access to the most recent data.

  2. investigating Feb 02, 2025, 08:59 PM UTC

    We are continuing to investigate this issue.

  3. identified Feb 02, 2025, 09:16 PM UTC

    The issue has been identified and a fix is being implemented.

  4. identified Feb 02, 2025, 09:52 PM UTC

    We are continuing to work on a fix for this issue.

  5. identified Feb 02, 2025, 10:07 PM UTC

    We are continuing to work on a fix for this issue.

  6. identified Feb 02, 2025, 10:22 PM UTC

    We are continuing to work on a fix for this issue.

  7. monitoring Feb 02, 2025, 10:35 PM UTC

    A fix has been implemented and we are monitoring the results.

  8. resolved Feb 02, 2025, 10:42 PM UTC

    This incident has been resolved.