Splunk APM Trace Data Ingestion Delayed

Update timeline

1. investigating Mar 25, 2026, 07:04 PM UTC

   A degradation in the performance of the Splunk APM data ingestion pipeline is causing the processing and storage of raw trace data to be delayed by more than fifteen minutes. No data is being lost at this time and MetricSets are not impacted but the most recent data may not be available in trace search results.

2. identified Mar 25, 2026, 08:23 PM UTC

   We are actively working to mitigate the performance degradation impacting the Splunk APM data ingestion pipeline. Efforts are underway to stabilize the environment and reduce delays in processing raw trace data. Customers may continue to experience delays in trace data availability, while MetricSets remain unaffected. We will provide further updates as mitigation progresses and system performance improves.

3. identified Mar 25, 2026, 09:56 PM UTC

   We are continuing to work on mitigating the issue and stabilizing the environment. Customers may still experience delays of more than 15 minutes in trace data availability. In addition, Troubleshooting MetricSets (TMS) and Monitoring MetricSets (MMS) are also experiencing similar delays. We will provide further updates as progress continues.

4. identified Mar 25, 2026, 11:16 PM UTC

   After applying some mitigations, they have not yielded the intended results. We continue to work on mitigating the issue and stabilizing the environment. Customers may still experience delays of more than 15 minutes in trace data availability. In addition, Troubleshooting MetricSets (TMS) and Monitoring MetricSets (MMS) are also experiencing similar delays. We will provide further updates as progress continues.

5. identified Mar 25, 2026, 11:48 PM UTC

   We are actively mitigating the issue and monitoring as delays improve. Customers may still experience delays of over 15 minutes in trace data, as well as in Troubleshooting and Monitoring MetricSets. We will share further updates as progress is made.

6. identified Mar 26, 2026, 12:09 AM UTC

   We are continuing to mitigate the issue and are seeing gradual improvement; customers may still experience delays of over 15 minutes in data processing and availability, and we will provide further updates as recovery progresses

7. identified Mar 26, 2026, 12:28 AM UTC

   We are seeing gradual recovery as mitigation efforts take effect; customers may continue to experience delays in data processing and availability, though performance is steadily improving

8. monitoring Mar 26, 2026, 12:46 AM UTC

   The latency issues have been resolved, and we are now monitoring the service to ensure continued stability. Thank you for your patience

9. resolved Mar 26, 2026, 01:54 AM UTC

   The incident has been resolved, and data processing performance is now stable. Thank you for your patience.