SignalFx US1 incident

Splunk RUM Metrics, Monitoring MetricSets, Session Data and Trace Data being Dropped

Major Resolved View vendor source →

SignalFx US1 experienced a major incident on June 5, 2025 affecting Splunk RUM Ingest and Splunk RUM Metrics and 1 more component, lasting 6h 7m. The incident has been resolved; the full update timeline is below.

Started
Jun 05, 2025, 01:02 PM UTC
Resolved
Jun 05, 2025, 07:10 PM UTC
Duration
6h 7m
Detected by Pingoru
Jun 05, 2025, 01:02 PM UTC

Affected components

Splunk RUM IngestSplunk RUM MetricsSplunk RUM Monitoring MetricSetsSplunk RUM Session DetailsSplunk RUM Troubleshooting MetricSetsSession Replay Ingest

Update timeline

  1. investigating Jun 05, 2025, 02:01 PM UTC

    A degradation of the ingest path for Splunk RUM processing pipeline is causing Monitoring MetricSets, Session Data, Trace data and other data related to RUM to be dropped and lost. We are investigating the issue and will provide an update as soon as possible.

  2. investigating Jun 05, 2025, 02:06 PM UTC

    We are continuing to investigate this issue.

  3. investigating Jun 05, 2025, 02:12 PM UTC

    We are continuing to investigate this issue.

  4. investigating Jun 05, 2025, 02:39 PM UTC

    We are continuing to investigate this issue.

  5. investigating Jun 05, 2025, 02:48 PM UTC

    We are continuing to investigate this issue.

  6. investigating Jun 05, 2025, 03:01 PM UTC

    We are continuing to investigate this issue.

  7. investigating Jun 05, 2025, 03:27 PM UTC

    We are continuing to investigate this issue.

  8. identified Jun 05, 2025, 03:32 PM UTC

    The issue has been identified and a fix is being implemented.

  9. monitoring Jun 05, 2025, 03:32 PM UTC

    A fix has been implemented and we are monitoring the results. Data related to Splunk RUM is no longer being dropped but may be delayed.

  10. monitoring Jun 05, 2025, 04:06 PM UTC

    We are continuing to monitor for any further issues.

  11. investigating Jun 05, 2025, 04:23 PM UTC

    We are currently investigating this issue.

  12. investigating Jun 05, 2025, 04:45 PM UTC

    We are continuing to investigate this issue.

  13. investigating Jun 05, 2025, 04:55 PM UTC

    We are continuing to investigate this issue.

  14. investigating Jun 05, 2025, 05:18 PM UTC

    We are continuing to investigate this issue.

  15. investigating Jun 05, 2025, 05:33 PM UTC

    We are continuing to investigate this issue.

  16. investigating Jun 05, 2025, 05:44 PM UTC

    We are continuing to investigate this issue.

  17. monitoring Jun 05, 2025, 05:51 PM UTC

    A fix has been implemented and we are monitoring the results.

  18. monitoring Jun 05, 2025, 06:15 PM UTC

    We are continuing to monitor for any further issues.

  19. monitoring Jun 05, 2025, 06:46 PM UTC

    We are continuing to monitor for any further issues.

  20. monitoring Jun 05, 2025, 06:56 PM UTC

    We are continuing to monitor for any further issues.

  21. resolved Jun 05, 2025, 07:10 PM UTC

    This incident has been resolved.