SignalFx US0 Outage History

A fix has been implemented and we are monitoring the results.

We are continuing to monitor for any further issues.

Apr 17, 10:46 PDT Resolved - This incident has been resolved. Apr 17, 06:38 PDT Monitoring - The mitigation has been completed and unified identity login access has been restored in this realm. We are no longer seeing the login failures. We will continue to monitor to ensure stability. Apr 17, 06:17 PDT Update - We are still continuing to work on a fix for this issue. Customers may still experience login failures with unified identity in this realm while we validate recovery. We will provide another update as soon as more information is available. Apr 17, 05:58 PDT Update - We are continuing to work on a fix for this issue. Customers may still experience login failures with unified identity in this realm while we validate recovery. We will provide another update as soon as more information is available. Apr 17, 05:31 PDT Identified - We have identified the issue and are working on identifying a correct mitigation. Customers will still experience login failures with unified identity in this realm. We will share another update as soon as more information is available. Apr 17, 05:18 PDT Update - Customers are unable to login into Splunk Observability using Splunk Cloud (SCS) unified identify login page. Customers are seeing an internal server error HTTP 500 during login. We are actively investigating this incident. Apr 17, 01:30 PDT Investigating - Customers are unable to login into Splunk Observability using Splunk Cloud (SCS) unified identify login page. Customers are seeing an internal server error HTTP 500 during login. We are actively investigating this incident.

Apr 5, 12:29 PDT Resolved - This incident has been resolved. Apr 5, 12:14 PDT Monitoring - A fix has been implemented and we are monitoring the results. Apr 5, 11:59 PDT Update - Investigation into the root cause of the Unified Identity issue is ongoing. Apr 5, 11:41 PDT Update - Our team is currently working to resolve an issue impacting Unified Identity services. Apr 5, 11:06 PDT Update - We are continuing to investigate the root cause of this issue. Apr 5, 10:22 PDT Update - We are continuing to investigate this issue. Apr 5, 09:36 PDT Investigating - We are investigating a major outage of Splunk cloud involving logging in via Unified Identity. Ingest is not impacted at this time. We will provide an update as soon as possible. Splunk Log Observer may not be available for unified identity customers.

Mar 24, 11:19 PDT Update - The issue has been identified as a regional cloud provider outage that is impacting our ability to maintain Synthetics testing in specific areas. Due to the uncertainty regarding the resolution timeline for this issue, we have made the decision to temporarily remove the following locations from our Synthetics testing network: Bahrain Muscat UAE We recommend reviewing your current test configurations and selecting alternative testing locations if necessary to ensure continued coverage. Our engineering team is monitoring the situation closely. We will provide further updates as more information becomes available from the cloud provider. We apologize for any inconvenience this may cause and appreciate your patience as we work to maintain the reliability of our services. Mar 23, 17:56 PDT Update - The issue has been identified as an outage affecting the Bahrain region due to a Cloud Provider outage. We recommend that users temporarily run their Synthetic Monitoring tests from an alternative location while this issue is being resolved by the Cloud Provider. Mar 23, 16:30 PDT Update - We are continuing to work on a fix for this issue. Mar 23, 16:27 PDT Identified - The issue has been identified and is due to a Cloud Provider outage in the Bahrain region.

Mar 19, 17:00 PDT Resolved - This incident has been resolved. Mar 19, 16:12 PDT Update - We are continuing to monitor for any further issues. Mar 19, 15:27 PDT Monitoring - We have successfully addressed an infrastructure issue and the environment is now stable. Our team is continuing to monitor the situation to ensure everything remains fully operational. Mar 19, 14:59 PDT Investigating - Charts and Detectors are intermittently failing. Datapoint ingest is not affected. We are currently investigating.

Mar 17, 06:48 PDT Resolved - The incident has been resolved. Mar 17, 06:23 PDT Monitoring - The fix has been implemented and charts and detectors, web interface and Splunk Synthetics Monitoring Interface have changed to operational. Teams are monitoring for stability and performance. Mar 17, 06:10 PDT Identified - The issue has been identified and a fix is being implemented, the impact on in charts and detectors, web interface and Splunk Synthetics Monitoring Interface has changed to degraded performance. Mar 17, 06:00 PDT Update - Customers may be experiencing an outage in charts and detectors, web interface and Splunk Synthetics Monitoring Interface. Datapoint ingest is not affected. Splunk technical teams are investigating and will continue to provide updates. Mar 17, 05:04 PDT Update - Customers may be experiencing delays in some charts and detectors. Datapoint ingest is not affected, but web interface shows access issues. Splunk is investigating the incident. Mar 17, 04:04 PDT Investigating - Customers may be experiencing delays in some charts and detectors. Datapoint ingest is not affected. We are investigating and will provide an update shortly.

Mar 16, 18:12 PDT Resolved - This incident has been resolved and the system is stable Mar 16, 18:02 PDT Monitoring - The system has fully recovered and the Operations team is actively monitoring to ensure continued stability and performance Mar 16, 17:05 PDT Identified - We've identified and resolved the underlying issue, the system is recovering and we expect the lag to be fully resolved within an hour. Mar 16, 16:00 PDT Update - The Operations team is currently investigating an issue affecting the rendering of alert modal charts and latency within the alerts tab. Please note that alert creation, delivery, and data ingestion remain fully functional. There is no risk of data loss. Mar 16, 14:37 PDT Investigating - We are currently investigating this issue.

Mar 5, 11:23 PST Resolved - Monitoring over the past several days has not identified any impact to Splunk Observability Cloud services related to the previously reported cloud service provider disruption in the Middle East. Splunk Observability Cloud services continue to operate normally. We will continue to monitor the situation and will notify customers if conditions change. Mar 3, 09:14 PST Update - Cisco is aware of an ongoing cloud service provider outage in the Middle East affecting multiple cloud offers and services. As a result, O11y customers ingesting metrics and/or metadata from impacted regions may experience significant delays and/or interruptions in data delivery. Customers who do not rely on metrics or metadata from these regions are not affected. We will continue to closely monitor this issue as it unfolds. Mar 2, 12:18 PST Update - Cisco is aware of an ongoing cloud service provider outage in the Middle East affecting multiple cloud offers and services. As a result, O11y customers ingesting metrics and/or metadata from impacted regions may experience significant delays and/or interruptions in data delivery. Customers who do not rely on metrics or metadata from these regions are not affected. We will continue to closely monitor this issue as it unfolds. Mar 2, 00:33 PST Monitoring - Cisco is aware of an ongoing cloud service provider outage in the Middle East affecting multiple cloud offers and services. As a result, O11y customers ingesting metrics and/or metadata from impacted regions may experience significant delays and/or interruptions in data delivery. Customers who do not rely on metrics or metadata from these regions are not affected. We will continue to closely monitor this issue as it unfolds. Mar 2, 00:33 PST Investigating - Cisco is aware of an ongoing cloud service provider outage in the Middle East affecting multiple cloud offers and services. As a result, O11y customers ingesting metrics and/or metadata from impacted regions may experience significant delays and/or interruptions in data delivery. Customers who do not rely on metrics or metadata from these regions are not affected. We will continue to closely monitor this issue as it unfolds.

Feb 28, 00:00 PST Completed - The scheduled maintenance has been completed. Feb 9, 00:00 PST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Feb 2, 09:42 PST Scheduled - Splunk Synthetic Monitoring will begin updating Google Chrome to version 142 for Browser tests on Monday, February 9th at 12:00 AM PT. We periodically auto-update to newer versions of Google Chrome when available. Due to differences between browser versions, test behavior or timings can sometimes change and may require updates to your test steps. This rollout will be done gradually over two weeks, after which a new runner version for Private Location customers will be made available.

Feb 27, 01:00 PST Completed - The scheduled maintenance has been completed. Feb 27, 00:00 PST In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary. Feb 24, 17:05 PST Update - We will be undergoing scheduled maintenance during this time. Feb 27, 00:00 PST Update - We will be undergoing scheduled maintenance during this time. Feb 27, 00:00 PST Update - We will be undergoing scheduled maintenance during this time. Feb 19, 09:36 PST Scheduled - There will be a planned maintenance window for Splunk APM in Splunk Observability Cloud. During this time, data in the Trace Analyzer, Trace Waterfall and Exemplar features may not be displayed (10 seconds to 2 minutes). All other functionalities, including MMS and TMS-powered charts and graphs, will remain unaffected. No action is needed. Thank you for your understanding.

Feb 20, 08:59 PST Resolved - This incident has been resolved. Feb 20, 08:49 PST Monitoring - A fix has been implemented and we are monitoring the results. Feb 20, 08:33 PST Update - We are continuing to investigate this issue. Feb 20, 08:11 PST Update - We are investigating intermittent failures affecting the Splunk APM and Synthetics landing pages and charts. We will provide an update as soon as possible. Feb 20, 08:10 PST Update - We are investigating intermittent failures affecting the Splunk APM and Synthetics landing pages and charts. We will provide an update as soon as possible. Feb 20, 07:44 PST Investigating - We are investigating the intermittent failures of the APM landing page and charts. We will provide an update as soon as possible.

Feb 19, 21:05 PST Resolved - This incident now stands resolved and no further updates are required. Feb 19, 20:34 PST Update - We are continuing to investigate this issue. Feb 19, 19:51 PST Investigating - We are investigating the intermittent failures of the APM landing page. We will provide an update as soon as possible.

Feb 17, 13:09 PST Resolved - This incident has been resolved. Feb 17, 13:03 PST Monitoring - A fix has been implemented and we are monitoring the results. Feb 17, 12:05 PST Update - Restoration is still in progress. Retrying to load the Data Management Page should yield positive results for some customers. Our teams continue to work diligently to resolve the issue and will provide further updates as progress is made. Feb 17, 11:28 PST Update - Our teams are actively working to restore services. Some customers may be seeing improvement at this time. We are committed to resolving the issue as quickly as possible and will provide updates as progress is made. Feb 17, 10:50 PST Identified - The issue has been identified and a fix is being implemented. Feb 17, 10:09 PST Investigating - Customers may be experiencing errors while loading Data Management Page.