Netskope Outage History

We are investigating a service impacting incident at the FR4 datacenter(s) for the following service(s) Data Protection, Incident Management System. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

The incident has been resolved

We are investigating a service impacting incident at the BOM1 datacenter(s) for the following service(s) NPA Gateway, NPA Stitcher, Private Access. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

Netskope has identified the root cause and is performing optimisations on the Load Balancer component to restore optimal performance. These changes will be performed for the next hour starting 9:30 UTC. During this window, the publishers connected to BOM1 point of presence might experience a disconnection.

All services have auto recovered. To prevent any further disruption, the optimisations will be performed off business hours. This incident is now resolved.

The incident has been resolved

We are investigating a service impacting incident at the SJC1 datacenter(s) for the following service(s) Incident Management System. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

Netskope continues to investigate for a potential root cause. Details will continue to be provided within the support portal.

Netskope continues to investigate at this time. A root cause has not yet been identified.

Netskope is implementing a potential mitigation strategy. Updates will continue to be provided as they become available.

Netskope has completed executing the mitigation strategy on one of the impacted service hosts. Positive results have been observed. The strategy is now being implemented across all other hosts. Updates will be provided as they become available. Estimated completion time is currently 1 to 1.5 hours.

The errors have reappeared on the first host where the first mitigation attempt was performed. Root cause investigation remains underway at this time.

Netskope continues the root cause investigation at this time. Updates will be provided as they become available.

Netskope continues to investigate for a root cause.

Netskope continues to analyze the underlying cause of this incident.

Netskope continues to investigate the origin of this incident, we have confirmation that there are no more errors reported.

Netskope continues to monitor system stability and investigate the root cause. All services are currently functioning within normal parameters with no new errors reported. We will provide a final update once the monitoring period is complete.

Netskope has observed no recurrence during the monitoring period, and system performance remains stable. Monitoring will continue through the completion of NAM business hours, after which a final update will be shared.

Netskope monitoring reports that system performance continues to be stable and all services are currently functioning within normal parameters, active monitoring remains in place

The incident has been resolved

Netskope has formalized a confident hypothesis on root cause. The incident is being moved to a resolved status. Further details can be found within the support portal.

We are investigating a service impacting incident at the FRA2 datacenter(s) for the following service(s) NSClient download. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

Netskope has identified the root cause and is implementing a potential mitigation strategy.

The incident has been resolved

We are investigating a service impacting incident at the AM2 datacenter(s) for the following service(s) Incident Management System. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

The incident has been resolved

We are investigating a service impacting incident at the All DPs datacenter(s) for the following service(s) IPSEC. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

Netskope has started the rollback process and expects it to be completed in the next 2 hours. Next update will be shared at 11:30 AM UTC.

Netskope has completed the rollback across 50% of the PoPs. The remaining rollback activities are expected to conclude within the next three hours. We will provide the next status update at 3:00 PM UTC.

Netskope has successfully completed the rollback across all Points of Presence (PoPs). A temporary surge in IPsec tunnel re-establishment requests triggered DoS protection mechanisms on certain PoPs, resulting in some tunnels going down. Netskope is actively working to stabilize the environment and restore full service continuity.

Following the rollback of IPsec nodes across all PoPs, a temporary tunnel imbalance has been observed. Netskope is actively executing remediation actions to restore balanced tunnel distribution. Customers may notice brief tunnel flapping during this window. A limited number of PoPs have also experienced continued failures after rollback and will be temporarily taken out of service for continued mitigation. Currently working is being performed in SAO4 and NYC1. Additional POPs will be provided if required.

Work continues to redeploy the IPSec service in some impacted data centers. Tunnel rebalancing efforts also continues at this time. During the rebalance activity, customers may experience transient tunnel flaps.

Rebalance activity for all Day 3 POPs has been completed. Day 2 is currently underway. Details on which POPs are targeted under each can be viewed in the support portal deployment notifications. Detail on how to access the deployment schedule is included in the support portal article. All work is expected to completed within the next 30 minutes.

Rebalance activity for all Day 2 POPs has been completed. Day 1 is currently underway. SAO4 data center work has also been completed. Further details are in the support portal.

Rebalance activity for all Day 1 POPs has been completed. Netskope is now conducting a thorough audit to ensure the IPSec service is fully operational in all data centers.

The incident has been resolved

We are investigating a service impacting incident at the SIN2 datacenter(s) for the following service(s) Event Logging. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

Netskope has identified the root cause and currently devising a mitigation strategy to mitigate the impact.

Netskope has applied the fix to mitigate the impact. It is anticipated to take close to 3 hours for the lag to completely drain. The next update will be shared at 1:45 PM UTC.

The incident has been resolved

We are investigating a service impacting incident at the MEL2 datacenter(s) for the following service(s) Advanced Analytics, DEM. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

The incident has been resolved

We are investigating a service impacting incident at the All MPs datacenter(s) for the following service(s) Cloud Access Security Broker. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

The vendor continues with their investigation as a root cause has not yet been identified. A temporary mitigation solution has been implemented by Netskope to eliminate the risk of interruption. The incident will be marked resolved once the vendor has provided a final solution.

The vendor released a fix based on a code regression that was identified after a recent deployment within their environment. The fix was successful in addressing all API subscription errors. The incident is now being marked resolved.

The incident has been resolved

We are investigating a service impacting incident at the all Management Plane(s) for the following service(s) Netskope Client. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

Netskope continues to investigate the issue.

Netskope continues to actively investigate the issue and is working to identify the root cause.

Netskope has identified the root cause as an endpoint shutdown occurring during the upgrade process, which can result in the client becoming uninstalled or entering a non-functional state. As a mitigation, a clean installation of the Netskope client is recommended to restore normal functionality.

Netskope continues to monitor for any future reoccurrences and is waiting for confirmation that the recommendation resolves the issue.

The incident has been resolved

We are investigating a service impacting incident at the SV5 datacenter(s) for the following service(s) CASB API Data Protection, Event Logging, Management Console WebUI. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

Netskope is actively working on implementing the mitigation strategy

Netskope continues to actively implement a mitigation strategy.

All affected nodes have been restored to an online status.

The incident has been resolved

We are investigating a service impacting incident at the ICN1 datacenter(s) for the following service(s) Secure Web Gateway. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

Netskope has identified the root cause of the issue and is currently developing a comprehensive remediation plan to ensure a full resolution.

Netskope has engaged the relevant transit partners for ICN1 to expedite a resolution. We are closely monitoring the restoration efforts.

We continue to work toward a resolution in coordination with our transit partners. Our next scheduled update will be shared at 12:00 PM UTC.

Netskope is continuing to work toward resolving the issue in coordination with its transit partners. The next update will be provided at 18:00 UTC.

The incident has been resolved

Netskope has confirmed that the issue has been resolved by the carrier partner. Following validation, the PoP ICN1 has been successfully reinstated into the NewEdge traffic management system.

We are investigating a service impacting incident for the following service(s) Secure Web Gateway. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

Netskope has identified the root cause of the issue and currently devising a mitigation strategy to mitigate the impact.

Mitigation steps have been successfully implemented on the MEL2 MP, and we have verified that the issue is resolved at that location. Netskope is now proceeding with these steps across all remaining MPs. A follow-up update will be provided at 8:45 AM UTC.

The incident has been resolved

Mitigation steps have been successfully completed across all MPs. Final verification confirms that the issue is fully resolved and all systems are performing as expected. This incident is now resolved.

We are investigating a service impacting incident at the FRA2, SJC2, DFW3, ZUR2, SIN2 datacenter(s) for the following service(s) Incident Management System. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

During the ongoing monitoring, we identified that the issue was not limited to FRA2 and also impacted San Jose, California, USA (SJC2), Dallas, Texas, USA (DFW3), Zurich, Switzerland (ZUR2) and Singapore (SIN2). Our teams are actively engaged in implementing the fix across the other impacted management planes to ensure full service stability.

The incident has been resolved

We are investigating a service impacting incident at the LON2 datacenter(s) for the following service(s) GRE, IPSEC, Netskope Client. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

LON2 has been removed from NewEdge traffic management system to mitigate the impact.

LON2 data center has been re-added to the NewEdge traffic management system. Netskope will continue to monitor before marking this incident as fully mitigated.

The incident has been resolved

We are investigating a service impacting incident at the ZUR2 datacenter(s) for the following service(s) Management Console WebUI. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

The incident has been resolved

We are investigating a service impacting incident at the IAD3 datacenter(s) for the following service(s) Management Console WebUI. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

The incident has been resolved

We are investigating a service impacting incident at the DFW3 datacenter(s) for the following service(s) Advanced Analytics, Borderless WAN, CASB API Data Protection, Cloud Security Posture Management, Core Services, DEM, DSPM, Event Logging, Incident Management System, Management Console WebUI, SaaS Security Posture Management. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

Netskope is working on the recovery of the infrastructure before proceeding with service restoration

Netskope is actively engaged in the recovery of the internal databases

The WebUI service has been restored.

Services recovered successfully. SkopeIT is currently processing a backlog of events

The incident has been resolved

We are investigating a service impacting incident at the TLV1 datacenter(s) for the following service(s) NPA Gateway, NPA Stitcher, Private Access. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

Netskope has implemented further mitigation actions. Further details on what was done can be found in the Support Portal

The incident has been resolved

We are investigating a service impacting incident at the BOM1, BOM2, CCU1, DEL1, DEL2, MAA1 datacenter(s) for the following service(s) NPA Gateway, NPA Stitcher. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

The incident has been resolved

We are investigating a service impacting incident at the YYZ2 datacenter(s) for the following service(s) All Services, Advanced Analytics, CASB API Data Protection, Core Services, Event Logging, Incident Management System, Management Console WebUI. Additional details on the scope of impact are currently being assessed and will be provided within the support portal once an article is provided.

The incident has been resolved

Netskope has received reports from customers that are missing recent malware scan events in SkopeIT.

Netskope continues to investigate this issue.

Netskope continues to investigate the issue.

Netskope is in the process of implementing a fix. More details can be found in the support portal.

Netskope has begun deploying a fix. More details can be found in the support portal.

This incident has been resolved. More details can be found in the support portal.

The incident has been resolved

Netskope has successfully rolled back the changes that caused the recent impact. Customers no longer need to use the temporary workaround.