Splunk OnCall had 11 outages in the last 2 years totaling 2412h 44m of downtime — averaging 0.5 incidents per month.
There were 11 Splunk OnCall outages since September 2, 2024 totaling 2412h 44m of downtime. Each is summarised below — incident details, duration, and resolution information.
Splunk On-Call experienced loading issues affecting the portal.victorops.com web interface and the Android and iOS mobile applications. The disruption lasted approximately one hour and has been resolved. We apologize for any inconvenience this disruption may have caused.
Splunk On-Call experienced an isolated network disruption between approximately 08:47 UTC - 09:12 UTC earlier today, December 5th. The functionality has since returned to operational. We apologize for any disruption or adverse impact this caused. Please contact Splunk On-Call support with any questions - https://help.splunk.com/en/splunk-enterprise/alert-and-respond/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support
Splunk On-Call Email Alert Ingestion appears to be disrupted. Thanks for your patience while we investigate this issue. Contact Us: https://help.splunk.com/en/splunk-enterprise/alert-and-respond/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support
We are now seeing successful delivery of email alerts in our system. We are moving status to monitoring while we investigate the cause for this disruption. Thanks for your patience while we continue to investigate. Updates to follow. Contact Us: https://help.splunk.com/en/splunk-enterprise/alert-and-respond/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support
We've identified this issue as stemming from a release by our Email Alert provider. Their release has been rolled back and now Email Alert delivery to On-Call is fully operational. Our apologies for this disruption. Contact Us: https://help.splunk.com/en/splunk-enterprise/alert-and-respond/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support
Please bear with us while we troubleshoot this issue; updates will follow. Note, this issue DOES NOT affect the functionality of previously downloaded iOS apps. Android is unaffected. For questions or concerns regarding the iOS Mobile App availability, please reach out to Splunk On-Call Support: https://help.splunk.com/en/splunk-observability-cloud/splunk-on-call/introduction-to-splunk-on-call/contact-splunk-on-call-support
This incident has been resolved. The iOS Mobile App for On-Call is once again available in the Apple App Store. Apologies for any inconvenience! Please send any questions regarding this issue to Splunk On-Call Support: https://help.splunk.com/en/splunk-observability-cloud/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support
The Splunk On-Call portal is currently unavailable via web and mobile. We are actively working to resolve this issue as quickly as possible. If you have any immediate questions or concerns, our support team is standing by to respond: 1 (855) 775-8657 https://help.victorops.com/knowledge-base/how-to-contact-splunk-on-call-support/
We've identified this issue related to a deploy inadvertently impacting our databases and are working to stabilize. The Web Client is coming back on line now. Thank you for your patience while we coming back on line.
Update: A fix has been implemented and we are monitoring the issue. The On-Call timeline is still experiencing loading issues. We are seeing high alert-lag across the platform.
We continue to monitor the situation. We are still experiencing high alert lag. The Web Client and Mobile Apps are now accessible. Thanks for your patience while we continue to work this issue to resolution.
This incident is now resolved. The Splunk On-Call web client and mobile applications are fully operational. We apologize for the inconvenience. Please reach out to the Splunk On-Call support team with any questions. https://help.splunk.com/en/splunk-observability-cloud/splunk-on-call/introduction-to-splunk-on-call/getting-started-guide-for-splunk-on-call-admins/contact-splunk-on-call-support
Emails sent to the Generic Email Endpoint Integration are not being ingested by On-Call. These emails are not creating incidents or other alerts as expected. All other Integrations and On-Call incident operations are fully operational at this time. We are currently investigating this issue. Please monitor this Statuspage for the most up to date information, we will post another update shortly
We have identified the source of this issue and are actively working to deploy a fix.
A fix has been implemented and we are monitoring the results. Stand by for confirmation
This incident is now resolved. Alert Ingestion - Inbound email is now fully operational. We apologize for this inconvenience. If you continue to experience issues with your inbound email alerts, please reach out to On-Call Support as soon as possible! https://docs.splunk.com/observability/en/sp-oncall/admin/get-started/spoc-support.html
Please bear with us while we investigate this issue—impacted components Single Sign On (SSO) Updates to follow. We apologize for this inconvenience. Please reach out to On-Call Support with any questions. https://help.victorops.com/knowledge-base/how-to-contact-splunk-on-call-support/
The issue has been identified and a fix is being implemented.
A fix has been implemented and we are monitoring the solution. SSO is back to an operational state.
This incident has been resolved. Single Sign On (SSO) is fully operational. We apologize for this inconvenience. If you continue to experience issues with your SSO connections, please reach out to On-Call Support as soon as possible! https://help.victorops.com/knowledge-base/how-to-contact-splunk-on-call-support/
We are investigating reports of issues with our MS Teams integration, specifically posts from On-Call to MS Teams. Please bear with us as we continue to look into this issue.
The issue has been identified and resolved; apologies for any inconvenience! Please direct any questions regarding this incident to On-Call Support: https://help.victorops.com/knowledge-base/how-to-contact-splunk-on-call-support/