Splunk OnCall incident

Splunk On-Call Reporting - On-Call Review Report data is degraded

Minor Resolved View vendor source →

Splunk OnCall experienced a minor incident on October 3, 2025 affecting Reporting, lasting 98d 6h. The incident has been resolved; the full update timeline is below.

Started
Oct 03, 2025, 03:11 PM UTC
Resolved
Jan 09, 2026, 09:30 PM UTC
Duration
98d 6h
Detected by Pingoru
Oct 03, 2025, 03:11 PM UTC

Affected components

Reporting

Update timeline

  1. identified Oct 03, 2025, 03:11 PM UTC

    We are currently experiencing inaccurate reporting data in the On-Call Review report (OCR) within the web client. This is a known issue, and we're working on a fix as soon as possible. If you are affected by this reporting feature, please reach out to On-Call Support. Contact Us: https://help.splunk.com/en/splunk-enterprise/alert-and-respond/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support Our apologies for this issue! Please stay tuned to this status page for updates! Thank you for your patience while we work towards a resolution.

  2. identified Oct 04, 2025, 04:16 AM UTC

    Update: Please bear with us as we continue to work on a fix for this issue. If you are affected and require immediate On-Call Review reporting data, please reach out to Support via a support ticket or through in-product live chat. We'll gladly provide an accurate, manually generated CSV of the data you are searching for in the On-Call Review report upon request. Contact Us: https://help.splunk.com/en/splunk-enterprise/alert-and-respond/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support Again, our apologies for this issue! Please stay tuned to this status page for further updates! Thank you for your patience while we work towards a resolution.

  3. identified Oct 07, 2025, 12:25 AM UTC

    We continue to work on a fix for this issue. If you are affected and require immediate attention, please reach out to Support via a support ticket or through in-product live chat. We've upgraded this incident to Major, as we are currently unable to provide an accurate CSV for the On-Call Review at this time. We've prioritized this issue with our Engineering teams. They are addressing the issue with urgency. Contact Us: https://help.splunk.com/en/splunk-enterprise/alert-and-respond/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support Our apologies again for this issue! Please stay tuned to this status page for updates! Thank you for your patience.

  4. identified Oct 07, 2025, 09:22 PM UTC

    Update: We continue to work on a fix for the On-Call Review reporting feature. The Splunk On-Call Engineering team and senior Support leadership continue to work towards providing you with accurate On-Call Review reporting. Please note that this is a complex problem involving data from many sources. Our teams are working urgently and diligently, with accurate On-Call-Review data as our top priority. If you are affected by this reporting feature, please reach out to On-Call Support. Contact Us: https://help.splunk.com/en/splunk-enterprise/alert-and-respond/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support Thank you again for your patience while we work to resolve this incident and the accuracy of the On-Call Review report.

  5. identified Oct 08, 2025, 10:23 PM UTC

    Update: Please bear with us as we continue to work on a fix for the On-Call Review reporting feature. The Splunk On-Call Engineering team and Senior Support leadership continue to work towards providing you with accurate On-Call Review reporting. ** Please note that this is a complex problem involving data from different sources. Our teams are working urgently and diligently, with accurate On-Call-Review data as our top priority. If you are affected by this reporting feature, please reach out to On-Call Support. Contact Us: https://help.splunk.com/en/splunk-enterprise/alert-and-respond/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support Thank you again for your patience while we work to resolve this incident and the accuracy of the On-Call Review report.

  6. identified Oct 09, 2025, 09:23 PM UTC

    Update: Please bear with us as we continue to work on a fix for the On-Call Review reporting feature. The Splunk On-Call Engineering team and Senior Support leadership continue to work towards providing you with accurate On-Call Review reporting. ** Please note that this is a complex problem involving data from different sources. Our teams are working urgently and diligently, with accurate On-Call-Review data as our top priority. If you are affected by this reporting feature, please reach out to On-Call Support. Contact Us: https://help.splunk.com/en/splunk-enterprise/alert-and-respond/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support Thank you again for your patience while we work to resolve this incident and the accuracy of the On-Call Review report.

  7. identified Oct 10, 2025, 10:12 PM UTC

    Update: Our Engineering and Support teams continue to give our full attention to resolving this issue as soon as possible. ** Please note that this is a complex problem involving data from different sources. Our teams are working urgently and diligently, with accurate On-Call-Review data as our top priority. If you are affected by this reporting feature, please reach out to On-Call Support. Contact Us: https://help.splunk.com/en/splunk-enterprise/alert-and-respond/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support Thank you again for your patience while we work to resolve this incident and the accuracy of the On-Call Review report.

  8. identified Oct 13, 2025, 10:53 PM UTC

    Splunk On-Call Engineering and Support teams continue to give our full attention to resolving this issue as soon as possible. ** Please note that this is a complex problem. Our teams are working urgently and diligently towards providing our customers with accurate On-Call-Review data. It is our top priority. If you are affected by this reporting feature, please reach out to On-Call Support. Contact Us: https://help.splunk.com/en/splunk-enterprise/alert-and-respond/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support Thank you again for your patience while we work to resolve this incident and the accuracy of the On-Call Review report.

  9. identified Oct 14, 2025, 11:35 PM UTC

    Splunk On-Call Engineering and Support teams continue to give our full attention to resolving this issue as soon as possible. ** Please note that this is a complex problem. Our teams are working urgently and diligently towards providing our customers with accurate On-Call-Review data. It is our top priority. If you are affected by this reporting feature, please reach out to On-Call Support. Contact Us: https://help.splunk.com/en/splunk-enterprise/alert-and-respond/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support Thank you again for your patience while we work to resolve this incident and the accuracy of the On-Call Review report.

  10. monitoring Oct 17, 2025, 08:46 PM UTC

    We want to update you on the status of the On-Call Review report issue. Our team has implemented several fixes to address the root cause, and as a result, we are reducing the incident severity to "Minor" and updating the status to "Monitoring" while we complete final data validation. We recognize that some issues with how On-Call Reporting data appears in the UI are still present. Please know that our engineering team is working diligently to fully resolve these remaining problems. If this issue is impacting your access to On-Call Review reporting data and you need information urgently, please contact our Support team by submitting a ticket or using the in-product live chat. Upon request, we’re happy to provide a manually generated CSV containing the accurate data you need from the On-Call Review report. Thank you for your understanding and patience as we work to fully resolve this incident. Contact Us: https://help.splunk.com/en/splunk-enterprise/alert-and-respond/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support

  11. monitoring Oct 31, 2025, 10:21 PM UTC

    Issues with On-Call Reporting data still appear in the UI. Please know that our engineering team continues to work on these remaining problems. If this issue is impacting your access to On-Call Review reporting data and you need information, please contact our Support team by submitting a ticket or using the in-product live chat. Upon request, we’re happy to provide a manually generated CSV containing the accurate data you need from the On-Call Review report. Thank you for your understanding and patience as we continue our work to fully resolve this incident. Contact Us: https://help.splunk.com/en/splunk-enterprise/alert-and-respond/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support

  12. resolved Jan 09, 2026, 09:30 PM UTC

    Issue Summary: An issue was introduced during an early-notification feature rollout, which prevented certain on-call and off-call entries from being saved to our primary database. This resulted in inaccurate reporting, with on-call hours appearing either higher or lower than actual values for affected users. Contact Us: https://help.splunk.com/en/splunk-enterprise/alert-and-respond/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support Again, our apologies for this issue! Thank you for being so patient while we worked towards a resolution.