MindTouch Outage History

Started Oct 20, 2026, 05:53 PM UTC· resolved Oct 20, 2026, 08:37 PM UTC

1. investigating Oct 20, 2026, 05:53 PM UTC

   CXone Knowledge Management Service Degradation: Site Degradation. The CXone Knowledge Management Engineering team is investigating reports of site unavailability.

2. investigating Oct 20, 2026, 06:01 PM UTC

   We are continuing to investigate this issue.

3. monitoring Oct 20, 2026, 06:16 PM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved Oct 20, 2026, 08:37 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 19, 2026, 06:31 PM UTC· resolved Oct 19, 2026, 07:16 PM UTC

1. investigating Oct 19, 2026, 06:31 PM UTC

   CXone Knowledge Management Service Degradation: Sites slow to load or unavailable. The CXone Knowledge Management Engineering team is investigating reports of site unavailability

2. monitoring Oct 19, 2026, 06:45 PM UTC

   CXone Knowledge Management - Fix Deployed - All Services Running Normally. The CXone Knowledge Management Engineering team has deployed a fix and all services are running normally. We are currently monitoring sites for deployment stability. Event duration 15 mins

3. resolved Oct 19, 2026, 07:16 PM UTC

   CXone Knowledge Management - Service Disruption Resolved - All Services Running Normally. The CXone Mpower Expert Engineering team has deployed a fix and monitored the deployment to make sure sites are stable. The issue is now resolved at this time. Event duration 15 mins

Read the full incident report →

Started Jul 23, 2026, 11:43 AM UTC· resolved Jul 23, 2026, 09:40 PM UTC

1. investigating Jul 23, 2026, 11:43 AM UTC

   CXone Knowledge Management Service Degradation. The CXone Knowledge Management Engineering team is investigating reports of site degradation.

2. investigating Jul 23, 2026, 12:07 PM UTC

   CXone Knowledge Management Service Degradation. The CXone Knowledge Management Engineering team is investigating reports of site degradation.

3. investigating Jul 23, 2026, 12:09 PM UTC

   We are continuing to investigate this issue.

4. investigating Jul 23, 2026, 12:25 PM UTC

   CXone Knowledge Management Service Degradation. The CXone Knowledge Management Engineering team is investigating reports of site degradation.

5. investigating Jul 23, 2026, 12:30 PM UTC

   CXone Knowledge Management Service Degradation. The CXone Knowledge Management Engineering team is investigating reports of site degradation.

6. investigating Jul 23, 2026, 12:48 PM UTC

   We are continuing to investigate this issue.

7. monitoring Jul 23, 2026, 12:59 PM UTC

   A fix has been implemented and we are monitoring the results.

8. resolved Jul 23, 2026, 09:40 PM UTC

   This incident has been resolved.

9. postmortem Aug 04, 2026, 04:05 PM UTC

   **Impact Start Time \(UTC\) 07/23/2026 07:31 AM UTC** **Impact End Time \(UTC\) 07/23/2026 01:01 PM UTC** **Incident Summary** Updated on 08/04/2026 - On 07/23/2026, some NiCE CXone Mpower customers reported experiencing page formatting failures and API call errors within the CXone Mpower Expert service. The incident was caused by incorrect Web Application Firewall \(WAF\) control rules that were deployed as part of a scheduled software release. The issue was resolved after the Engineering team reverted the incorrect WAF rules that had been implemented across the affected regions, restoring normal service operation. **Root Cause** The incident was caused by incorrect WAF control rules that were deployed as part of a scheduled software release. These rules classified and blocked legitimate web and Application Programming Interface \(API\) requests as bot traffic, resulting in service degradation and errors for affected customers. Although a corrective configuration had been identified and merged during pre-deployment validation activities, a build pipeline checkout failure prevented the corrected configuration from being included in the deployed release. As a result, the unintended WAF configuration remained active, causing legitimate customer traffic to be blocked. **Corrective Actions** * **Detection:** NiCE Internal support teams received reports from customers experiencing page formatting failures and API call errors within the CXone Mpower Expert service. * **Remediation:** The issue was resolved after the Engineering team reverted the incorrect WAF rules that had been implemented across the affected regions, restoring normal service operation. Completed on 07/23/2026. * **Prevention:** The Engineering team has updated the build pipeline configuration to address the checkout failure and improve build reliability. Completed on 07/30/2026. The Engineering team has strengthened release deployment criteria by implementing stricter release guidelines to ensure all required builds are available and verified before deployment. Completed on 08/03/2026. The Engineering team is evaluating monitoring and alerting enhancement opportunities to improve visibility and enable earlier detection. An update will be provided by End of Day MT on 08/14/2026. **Incident Timeline \(UTC\)** **07/23/2026 07:31 AM \(UTC\)** - The first customer case was opened, and Tech Support Engineers \(TSEs\) began their initial validations and troubleshooting investigation. **07/23/2026 11:37 AM \(UTC\)** - TSEs notified the Network Operations Center \(NOC\) engineers about the reported customer impact; a major incident was proposed and confirmed. **07/23/2026 12:31 PM \(UTC\)** - Engineers identified a suspected cause and began remediation steps. **07/23/2026 01:01 PM \(UTC\)** - The issue was resolved after the Engineering team reverted the incorrect WAF rules that had been implemented across the affected regions, restoring normal service operation. Following successful validations, the major incident was marked as resolved.

Read the full incident report →

Started Jul 01, 2026, 08:40 AM UTC· resolved Jul 02, 2026, 02:49 PM UTC

1. investigating Jul 01, 2026, 08:40 AM UTC

   CXone Knowledge Management Service Degradation: Sites unavailable. The CXone Knowledge Management Engineering team is investigating reports of site unavailability.

2. investigating Jul 01, 2026, 08:59 AM UTC

   CXone Knowledge Management Service Degradation: Sites unavailable. The CXone Knowledge Management Engineering team is investigating reports of site unavailability.

3. investigating Jul 01, 2026, 09:25 AM UTC

   CXone Knowledge Management Service Degradation: Sites unavailable. The CXone Knowledge Management Engineering team is investigating reports of site unavailability.

4. investigating Jul 01, 2026, 09:49 AM UTC

   CXone Knowledge Management Service Degradation: Sites unavailable. The CXone Knowledge Management Engineering team is investigating reports of site unavailability.

5. investigating Jul 01, 2026, 10:03 AM UTC

   CXone Knowledge Management Service Degradation: Sites unavailable. The CXone Knowledge Management Engineering team is investigating reports of site unavailability.

6. investigating Jul 01, 2026, 10:26 AM UTC

   CXone Knowledge Management Service Degradation: Sites unavailable. The CXone Knowledge Management Engineering team is investigating reports of site unavailability.

7. monitoring Jul 01, 2026, 10:31 AM UTC

   A fix has been implemented and we are monitoring the results.

8. identified Jul 01, 2026, 02:25 PM UTC

   CXone Knowledge Management Service Degradation: Sites unavailable or slow to load. The issue has been identified and a fix is being worked on for deployment.

9. identified Jul 01, 2026, 02:47 PM UTC

   CXone Knowledge Management Service Degradation: Sites unavailable or slow to load. The issue has been identified and a fix is being worked on for deployment.

10. identified Jul 01, 2026, 03:07 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable or slow to load. The issue has been identified and a fix is being worked on for deployment.

11. identified Jul 01, 2026, 03:27 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable or slow to load. The issue has been identified and a fix is being worked on for deployment.

12. identified Jul 01, 2026, 04:03 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable or slow to load. The issue has been identified and a fix is being worked on for deployment.

13. identified Jul 01, 2026, 04:35 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable or slow to load. The issue has been identified and a fix is being worked on for deployment.

14. identified Jul 01, 2026, 04:57 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable or slow to load. The issue has been identified and a fix is being worked on for deployment.

15. identified Jul 01, 2026, 05:20 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable or slow to load. The issue has been identified and a fix is being worked on for deployment.

16. identified Jul 01, 2026, 05:39 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable or slow to load. The issue has been identified and a fix is being worked on for deployment.

17. monitoring Jul 01, 2026, 05:47 PM UTC

    CXone Knowledge Management - Fix Deployed - All Services Running Normally. The CXone Knowledge Management Engineering team has deployed a fix and all services are running normally. We are currently monitoring sites for deployment stability. Event duration 3h 11m

18. identified Jul 01, 2026, 06:01 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable. The issue has been identified and a fix has been released. Sites may still be experiencing degraded performance but will be coming fully back online shortly.

19. identified Jul 01, 2026, 06:20 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable. The issue has been identified and a fix has been released. Sites may still be experiencing degraded performance but will be coming fully back online shortly.

20. identified Jul 01, 2026, 06:37 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable. The issue has been identified and a fix has been released. Sites may still be experiencing degraded performance but will be coming fully back online shortly.

21. identified Jul 01, 2026, 07:01 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable. The issue has been identified and a fix has been released. Sites may still be experiencing degraded performance but will be coming fully back online shortly.

22. identified Jul 01, 2026, 07:17 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable. The issue has been identified and a fix has been released. Sites may still be experiencing degraded performance but will be coming fully back online shortly.

23. identified Jul 01, 2026, 07:35 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable. The issue has been identified and a fix has been released. Sites may still be experiencing degraded performance but will be coming fully back online shortly.

24. identified Jul 01, 2026, 07:59 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable. The issue has been identified and a fix has been released. Sites may still be experiencing degraded performance but will be coming fully back online shortly.

25. monitoring Jul 01, 2026, 08:16 PM UTC

    A fix has been implemented and we are monitoring the results.

26. identified Jul 01, 2026, 08:28 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable. The issue has been identified and a fix has been released. Sites may still be experiencing degraded performance but will be coming fully back online shortly.

27. identified Jul 01, 2026, 08:46 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable. The issue has been identified and a fix has been released. Sites may still be experiencing degraded performance but will be coming fully back online shortly.

28. monitoring Jul 01, 2026, 08:52 PM UTC

    A fix has been implemented and we are monitoring the results.

29. resolved Jul 02, 2026, 02:49 PM UTC

    This incident has been resolved.

30. postmortem Jul 07, 2026, 05:37 PM UTC

    **Impact Start Time \(UTC\) 07/01/2026 07:55 AM UTC** **Impact End Time \(UTC\) 07/01/2026 01:21 PM UTC** **Incident Summary:** On 07/01/2026, some NiCE CXone Mpower customers reported service degradation while accessing and using the CXone Mpower Expert knowledge portal. The recurring service degradation was caused by a Domain Name System \(DNS\) resolution bottleneck within the platform infrastructure during periods of elevated traffic. The impact was mitigated after engineers scaled up resources and implemented a node local DNS cache. Service stability was further improved as traffic volumes decreased, allowing the platform to recover and return to normal operation. ## Root Cause The recurring service degradation was caused by a DNS resolution bottleneck within the platform infrastructure during periods of elevated traffic. Increased DNS request volume exceeded infrastructure processing limits, resulting in intermittent DNS lookup failure. These failures triggered application retries, increased latency, elevated “5xx” errors, and customer-facing service degradation. The issue was traced to an existing configuration gap that prevented application workloads from utilizing the platform's DNS caching capability. As a result, DNS requests continued to follow the standard resolution path, creating unnecessary network overhead and contributing to DNS request saturation under higher traffic volumes. While this condition had existed for some time without widespread impact, continued platform growth, increased workload density, and infrastructure scaling changes increased DNS traffic and exposed the underlying limitation. Once the DNS processing limits were reached, requests were intermittently dropped, resulting in cascading application failures and customer-facing service disruptions. During the investigation, engineers implemented several mitigations to reduce customer impact while continuing to identify the underlying cause. However, because these actions addressed symptoms rather than the root issue, the condition recurred and resulted in multiple major incidents before a permanent corrective solution was implemented. ## Corrective Actions **Detection:** The failure condition triggered system alarms, prompting engineers to initiate incident response procedures to validate, investigate, and reproduce the issue. While the investigation was underway, some customers reported service degradation when accessing and using the CXone Mpower Expert knowledge portal. **Remediation:** The impact was mitigated after engineers scaled up resources and implemented a node local DNS cache. Service stability was further improved as traffic volumes decreased, allowing the platform to recover and return to normal operation. Completed on 07/01/2026. **Prevention:** Engineers developed a dashboard to monitor local DNS cache performance, improving visibility into DNS health and enabling earlier detection, investigation, and remediation of potential issues before they impact service availability or customer experience. Completed on 07/01/2026. The Engineering team implemented an emergency change to correctly enable local DNS caching for applicable workloads, reducing DNS traffic, preventing node-level DNS saturation, and restoring the intended DNS architecture. This improved platform stability and DNS performance while reducing the risk of similar incidents. Additional infrastructure optimizations were also implemented to improve operational efficiency and minimize service impact during scaling activities. Completed on 07/02/2026. Engineers will conduct a comprehensive validation of DNS configurations across all workloads to ensure consistent use of node local DNS cache. While the immediate issue has been remediated, a fleet-wide review will identify and correct any remaining configuration gaps, helping to optimize DNS performance, reduce infrastructure load, and prevent similar service degradation events in the future. In addition, the Engineering team will continue extended monitoring of platform health and stability to verify the long-term effectiveness of the corrective actions and proactively identify any emerging issues. An update will be provided by End of Day MT on 07/17/2026. NiCE resiliency teams remain focused on enhancing system reliability, stability, platform-wide resilience, monitoring capabilities, and overall performance. These ongoing efforts are aligned with the Operational Resiliency Excellence initiative, driving continuous improvement, strengthening operational maturity, and delivering measurable business outcomes. ## Incident Timeline \(UTC\) 07/01/2026 07:55 AM \(UTC\) - Engineers received a service degradation alert and immediately initiated incident response procedures to investigate, validate, and reproduce the issue to determine its scope and potential customer impact. 07/01/2026 08:15 AM \(UTC\) - The first customer case was opened, and Tech Support \(TS\) engineers began the troubleshooting investigation. 07/01/2026 08:39 AM \(UTC\) - TS engineers notified the Network Operations Center \(NOC\) engineers about the reported customer impact; a major incident was proposed and confirmed. 07/01/2026 09:54 AM \(UTC\) - Engineers identified a suspected cause and Incident Timeline \(UTC\) began remediation steps. 07/01/2026 10:31 AM \(UTC\) - A fix was implemented, and engineers closely monitored system performance. While the remediation reduced the impact, it did not fully restore service stability. Engineers continued their investigation and identification of further remediation actions. 07/01/2026 01:21 PM \(UTC\) - The impact was resolved after engineers scaled up resources and implemented a node local DNS cache, while traffic volume returned to normal levels. Following successful validations, the major incident was marked as resolved. Engineering teams continued extended monitoring of platform performance and kept the service disruption notification active on the Status page while validating the long-term effectiveness of the remediation. ‌ **Impact Start Time \(UTC\) 07/01/2026 02:11 PM UTC** **Impact End Time \(UTC\) 07/01/2026 07:18 PM UTC** **Incident Summary:** On 07/01/2026, some NiCE CXone Mpower customers reported service degradation while accessing and using the CXone Mpower Expert knowledge portal. The recurring service degradation was caused by a Domain Name System \(DNS\) resolution bottleneck within the platform infrastructure during periods of elevated traffic. The impact was mitigated after engineers scaled up cluster resources and updated application workloads to utilize node local DNS cache, reducing DNS traffic and preventing nodes from reaching DNS query processing limits. ## Root Cause The recurring service degradation was caused by a DNS resolution bottleneck within the platform infrastructure during periods of elevated traffic. Increased DNS request volume exceeded infrastructure processing limits, resulting in intermittent DNS lookup failure. These failures triggered application retries, increased latency, elevated “5xx” errors, and customer-facing service degradation. The issue was traced to an existing configuration gap that prevented application workloads from utilizing the platform's DNS caching capability. As a result, DNS requests continued to follow the standard resolution path, creating unnecessary network overhead and contributing to DNS request saturation under higher traffic volumes. While this condition had existed for some time without widespread impact, continued platform growth, increased workload density, and infrastructure scaling changes increased DNS traffic and exposed the underlying limitation. Once the DNS processing limits were reached, requests were intermittently dropped, resulting in cascading application failures and customer-facing service disruptions. During the investigation, engineers implemented several mitigations to reduce customer impact while continuing to identify the underlying cause. However, because these actions addressed symptoms rather than the root issue, the condition recurred and resulted in multiple major incidents before a permanent corrective solution was implemented. ## Corrective Actions **Detection:** The failure condition triggered system alarms, prompting engineers to initiate incident response procedures to validate, investigate, and reproduce the issue. While the investigation was underway, some customers reported service degradation when accessing and using the CXone Mpower Expert knowledge portal. **Remediation:** The impact was mitigated after engineers scaled up cluster resources and updated application workloads to utilize node local DNS cache, reducing DNS traffic and preventing nodes from reaching DNS query processing limits. These actions restored normal service behavior and improved overall platform stability. Completed on 07/01/2026. **Prevention:** Engineers developed a dashboard to monitor local DNS cache performance, improving visibility into DNS health and enabling earlier detection, investigation, and remediation of potential issues before they impact service availability or customer experience. Completed on 07/01/2026. The Engineering team implemented an emergency change to correctly enable local DNS caching for applicable workloads, reducing DNS traffic, preventing node-level DNS saturation, and restoring the intended DNS architecture. This improved platform stability and DNS performance while reducing the risk of similar incidents. Additional infrastructure optimizations were also implemented to improve operational efficiency and minimize service impact during scaling activities. Completed on 07/02/2026. Engineers will conduct a comprehensive validation of DNS configurations across all workloads to ensure consistent use of node local DNS cache. While the immediate issue has been remediated, a fleet-wide review will identify and correct any remaining configuration gaps, helping to optimize DNS performance, reduce infrastructure load, and prevent similar service degradation events in the future. In addition, the Engineering team will continue extended monitoring of platform health and stability to verify the long-term effectiveness of the corrective actions and proactively identify any emerging issues. An update will be provided by End of Day MT on 07/17/2026. NiCE resiliency teams remain focused on enhancing system reliability, stability, platform-wide resilience, monitoring capabilities, and overall performance. These ongoing efforts are aligned with the Operational Resiliency Excellence initiative, driving continuous improvement, strengthening operational maturity, and delivering measurable business outcomes. ## Incident Timeline \(UTC\) 07/01/2026 02:11 PM \(UTC\) - Engineers observed service degradation alert and the first customer case was opened. Engineers began their initial validation and troubleshooting investigation. 07/01/2026 02:55 PM \(UTC\) - TS engineers notified the Network Operations Center \(NOC\) engineers about the reported customer impact; a major incident was proposed and confirmed. 07/01/2026 03:14 PM \(UTC\) - Engineers identified a suspected cause and began remediation steps. 07/01/2026 03:49 PM \(UTC\) - Engineers began increasing cluster capacity Incident Timeline \(UTC\) and redistributed workloads to use the node local DNS cache. 07/01/2026 05:29 PM \(UTC\) - Sites became accessible but engineers were still seeing some “500” errors. Engineers planned to implement additional configuration changes to fully restore the service. 07/01/2026 05:55 PM \(UTC\) - Engineers continued resetting pods to stabilize and fully restore service. 07/01/2026 06:40 PM \(UTC\) - Problematic pods were removed from the affected nodes as part of ongoing recovery actions. 07/01/2026 07:18 PM \(UTC\) - The impact was resolved after engineers increased cluster resources and updated the application service with the node local DNS cache. Following successful validation, the major incident was marked as resolved. Since implementation of the fix, engineers have observed significant improvements in platform stability and continue to monitor performance to ensure sustained effectiveness.

Read the full incident report →

Started Jun 29, 2026, 03:25 PM UTC· resolved Jun 29, 2026, 09:51 PM UTC

1. investigating Jun 29, 2026, 03:25 PM UTC

   CXone Knowledge Management Service Degradation. The CXone Knowledge Management Engineering team is investigating reports of degraded performance

2. investigating Jun 29, 2026, 03:33 PM UTC

   We are continuing to investigate this issue.

3. investigating Jun 29, 2026, 03:46 PM UTC

   CXone Knowledge Management Service Degradation. The CXone Knowledge Management Engineering team is investigating reports of degraded performance

4. investigating Jun 29, 2026, 04:02 PM UTC

   CXone Knowledge Management Service Degradation. The CXone Knowledge Management Engineering team is investigating reports of degraded performance

5. investigating Jun 29, 2026, 04:29 PM UTC

   CXone Knowledge Management Service Degradation. The CXone Knowledge Management Engineering team is investigating reports of degraded performance

6. monitoring Jun 29, 2026, 04:34 PM UTC

   A fix has been implemented and we are monitoring the results.

7. resolved Jun 29, 2026, 09:51 PM UTC

   This incident has been resolved.

8. postmortem Jul 07, 2026, 05:27 PM UTC

   **Impact Start Time \(UTC\) 06/29/2026 01:22 PM UTC** **Impact End Time \(UTC\) 06/29/2026 04:31 PM UTC** **Incident Summary:** On 06/29/2026, some NiCE CXone Mpower customers reported service degradation while accessing and using the CXone Mpower Expert knowledge portal. The recurring service degradation was caused by a Domain Name System \(DNS\) resolution bottleneck within the platform infrastructure during periods of elevated traffic. The impact was resolved after traffic volumes returned to normal levels. ## Root Cause The recurring service degradation was caused by a DNS resolution bottleneck within the platform infrastructure during periods of elevated traffic. Increased DNS request volume exceeded infrastructure processing limits, resulting in intermittent DNS lookup failure. These failures triggered application retries, increased latency, elevated “5xx” errors, and customer-facing service degradation. The issue was traced to an existing configuration gap that prevented application workloads from utilizing the platform's DNS caching capability. As a result, DNS requests continued to follow the standard resolution path, creating unnecessary network overhead and contributing to DNS request saturation under higher traffic volumes. While this condition had existed for some time without widespread impact, continued platform growth, increased workload density, and infrastructure scaling changes increased DNS traffic and exposed the underlying limitation. Once the DNS processing limits were reached, requests were intermittently dropped, resulting in cascading application failures and customer-facing service disruptions. During the investigation, engineers implemented several mitigations to reduce customer impact while continuing to identify the underlying cause. However, because these actions addressed symptoms rather than the root issue, the condition recurred and resulted in multiple major incidents before a permanent corrective solution was implemented. ## Corrective Actions **Detection:** Corrective Actions The failure condition triggered system alarms, prompting engineers to initiate incident response procedures to validate, investigate, and reproduce the issue. While the investigation was underway, some customers reported service degradation when accessing and using the CXone Mpower Expert knowledge portal. **Remediation:** The impact was resolved after traffic volumes returned to normal levels. Completed on 06/29/2026. **Prevention:** Engineers developed a dashboard to monitor local DNS cache performance, improving visibility into DNS health and enabling earlier detection, investigation, and remediation of potential issues before they impact service availability or customer experience. Completed on 07/01/2026. The Engineering team implemented an emergency change to correctly enable local DNS caching for applicable workloads, reducing DNS traffic, preventing node-level DNS saturation, and restoring the intended DNS architecture. This improved platform stability and DNS performance while reducing the risk of similar incidents. Additional infrastructure optimizations were also implemented to improve operational efficiency and minimize service impact during scaling activities. Completed on 07/02/2026. Engineers will conduct a comprehensive validation of DNS configurations across all workloads to ensure consistent use of node local DNS cache. While the immediate issue has been remediated, a fleet-wide review will identify and correct any remaining configuration gaps, helping to optimize DNS performance, reduce infrastructure load, and prevent similar service degradation events in the future. In addition, the Engineering team will continue extended monitoring of platform health and stability to verify the long-term effectiveness of the corrective actions and proactively identify any emerging issues. An update will be provided by End of Day MT on 07/17/2026. NiCE resiliency teams remain focused on enhancing system reliability, stability, platform-wide resilience, monitoring capabilities, and overall performance. These ongoing efforts are aligned with the Operational Resiliency Excellence initiative, driving continuous improvement, strengthening operational maturity, and delivering measurable business outcomes. ## Incident Timeline \(UTC\) 06/29/2026 01:22 PM \(UTC\) - Engineers received a service degradation alert and immediately initiated incident response procedures to investigate, validate, and reproduce the issue to determine its scope and potential customer impact. 06/29/2026 01:30 PM \(UTC\) - The first customer case was reported, and Technical Support \(TS\) engineers began validation and troubleshooting activities. 06/29/2026 03:34 PM \(UTC\) - TS engineers notified the Network Operations Center \(NOC\) engineers about the reported customer impact; a major incident was proposed and confirmed. 06/29/2026 03:39 PM \(UTC\) - Engineers identified a suspected contributing factor and initiated mitigation efforts to stabilize the service. 06/29/2026 04:31 PM \(UTC\) - Service performance recovered as traffic volumes returned to normal levels. Following successful validation of platform

Read the full incident report →

Started May 11, 2026, 08:38 AM UTC· resolved May 12, 2026, 02:50 PM UTC

1. investigating May 11, 2026, 08:38 AM UTC

   CXone Expert Service Degradation. The CXone Expert Engineering team is investigating reports of site unavailability.

2. investigating May 11, 2026, 08:52 AM UTC

   CXone Expert Service Degradation. The CXone Expert Engineering team is investigating reports of site unavailability.

3. monitoring May 11, 2026, 09:25 AM UTC

   A fix has been implemented and we are monitoring the results.

4. identified May 11, 2026, 07:08 PM UTC

   CXone Knowledge Management Service Degradation: Sites unavailable. The issue has been identified and a fix is being worked on for deployment.

5. identified May 11, 2026, 07:47 PM UTC

   CXone Knowledge Management Service Degradation: Sites unavailable. The issue has been identified and a fix is being worked on for deployment.

6. identified May 11, 2026, 08:04 PM UTC

   CXone Knowledge Management Service Degradation: Sites unavailable. The issue has been identified and a fix is being worked on for deployment.

7. identified May 11, 2026, 08:24 PM UTC

   CXone Knowledge Management Service Degradation: Sites unavailable. The issue has been identified and a fix is being worked on for deployment.

8. identified May 11, 2026, 08:45 PM UTC

   CXone Knowledge Management Service Degradation: Sites unavailable. The issue has been identified and a fix is being worked on for deployment.

9. identified May 11, 2026, 09:19 PM UTC

   CXone Knowledge Management - Fix Deployed - All Services Running Normally. The CXone Knowledge Management Engineering team has deployed a fix and all services are running normally. We are currently monitoring sites for deployment stability.

10. resolved May 12, 2026, 02:50 PM UTC

    This incident has been resolved.

11. postmortem May 27, 2026, 03:58 PM UTC

    **Major Incident# 02796435** **Impact Start Time \(UTC\) 05/12/2026 02:50 PM UTC** **Impact End Time \(UTC\) 05/12/2026 03:36 PM UTC** ### Incident Summary Updated on 05/22/2026 - On 05/12/2026, some NiCE CXone Knowledge customers experienced slowness when accessing sites, while others were unable to access the platform entirely with a “504 Gateway Timeout” error within the CXone Knowledge portal. The recurring service degradation incidents were caused by increased traffic volumes combined with performance limitations in certain backend processes, primarily impacting the US regional platform due to higher demand. The impact was mitigated after restarting the affected pods, which restored platform stability ### Root Cause The recurring service degradation incidents were caused by increased traffic volumes combined with performance limitations in certain backend processes, primarily impacting the US regional platform due to higher demand. Under elevated traffic conditions, including automated crawler activity, some requests followed less optimized processing paths, increasing system load. This was further amplified by legacy or complex page content requiring more intensive processing. While scaling actions helped restore capacity, they also introduced temporary overhead that contributed to intermittent performance degradation. Additionally, although autoscaling functioned as designed, it reached its limits and was insufficient to address constraints related to per-pod Central Processing Unit \(CPU\) capacity. ### Corrective Actions **Detection:** Internal support teams detected partial service failures through proactive alerting and monitoring. Subsequently, engineers received customer reports of slowness when accessing sites within the CXone Knowledge portal. **Remediation:** The impact was mitigated after restarting the affected pods, which restored platform stability. Completed on 05/12/2026. **Prevention:** The Engineering team implemented interim mitigation measures to maintain system stability and ensure consistent performance while permanent improvements are finalized and deployed. Completed on 05/12/2026. Baseline system capacity was increased by raising the minimum number of pods and allocating higher CPU resources. This ensures sufficient resources are consistently available, reduces reliance on dynamic scaling, and improves overall system stability during periods of increased demand. Completed on 05/12/2026. The Engineering team will implement targeted software optimizations, including improvements to a specific endpoint that previously introduced cascading effects during scaling events. These enhancements are designed to reduce resource contention and improve system efficiency under high load conditions. The changes are currently undergoing validation and will be deployed following comprehensive testing. Completed on 05/21/2026. The Engineering team will enhance alert notification delivery to ensure alarms are reliably triggered and routed to the appropriate response teams as expected, enabling faster detection and more timely corrective action when issues arise. Completed on 05/22/2026. ### Incident Timeline \(UTC\) 5/12/2026 02:50 PM \(UTC\) - Engineers identified partial service failures through proactive alerting and monitoring. A corresponding service disruption notification was promptly published on the Status Health page while engineers initiated their investigation and remediation efforts to prevent customer impact. 5/12/2026 03:09 PM \(UTC\) - The first customer case was opened, and Tech Support \(TS\) engineers began their initial validation and troubleshooting investigation. 5/12/2026 03:27 PM \(UTC\) - TS engineers notified the Network Operations Center \(NOC\) engineers about the reported customer impact; a major incident was proposed and confirmed. 5/12/2026 03:36 PM \(UTC\) - The impact was resolved after engineers restarted the affected pods and validated recovery through internal testing, marking the end of customer impact. Following confirmation of sustained stability and no further customer impact, the incident was subsequently marked as resolved.

Read the full incident report →

Started May 11, 2026, 06:31 AM UTC· resolved May 11, 2026, 07:19 AM UTC

1. investigating May 11, 2026, 06:31 AM UTC

   CXone Expert Service Degradation. The CXone Expert Engineering team is investigating reports of Service Degradation.

2. investigating May 11, 2026, 06:53 AM UTC

   CXone Expert Service Degradation. The CXone Expert Engineering team is investigating reports of Service Degradation.

3. monitoring May 11, 2026, 07:05 AM UTC

   A fix has been implemented and we are monitoring the results.

4. resolved May 11, 2026, 07:19 AM UTC

   This incident has been resolved.

5. postmortem May 27, 2026, 03:53 PM UTC

   **Major Incident# 02795089** **Impact Start Time \(UTC\) 05/11/2026 06:02 AM UTC** **Impact End Time \(UTC\) 05/11/2026 06:25 AM UTC** ### Incident Summary Updated on 05/22/2026 - On 05/11/2026, a NiCE CXone Mpower customer experienced slowness when accessing sites, while others were unable to access the platform entirely with a “504 Gateway Timeout” error within the CXone Mpower Expert knowledge portal. The recurring service degradation incidents were caused by increased traffic volumes combined with performance limitations in certain backend processes, primarily impacting the US regional platform due to higher demand. The impact was mitigated after restarting the affected pods, which restored platform stability ### Root Cause The recurring service degradation incidents were caused by increased traffic volumes combined with performance limitations in certain backend processes, primarily impacting the US regional platform due to higher demand. Under elevated traffic conditions, including automated crawler activity, some requests followed less optimized processing paths, increasing system load. This was further amplified by legacy or complex page content requiring more intensive processing. While scaling actions helped restore capacity, they also introduced temporary overhead that contributed to intermittent performance degradation. Additionally, although autoscaling functioned as designed, it reached its limits and was insufficient to address constraints related to per-pod Central Processing Unit \(CPU\) capacity. ### Corrective Actions **Detection:** Although built in alerting mechanisms were in place to detect this type of condition, alerts did not consistently reach the responsible teams as expected. In some cases, alerts were grouped or suppressed, delaying timely visibility of the issue. As a result, internal teams became aware of the impact primarily through a customer report of slowness when accessing sites within the CXone Mpower Expert knowledge portal. **Remediation:** The impact was mitigated after restarting the affected pods, which restored platform stability. Completed on 05/11/2026. **Prevention:** The Engineering team implemented interim mitigation measures to maintain system stability and ensure consistent performance while permanent improvements are finalized and deployed. Completed on 05/12/2026. Baseline system capacity was increased by raising the minimum number of pods and allocating higher CPU resources. This ensures sufficient resources are consistently available, reduces reliance on dynamic scaling, and improves overall system stability during periods of increased demand. Completed on 05/12/2026. The Engineering team will implement targeted software optimizations, including improvements to a specific endpoint that previously introduced cascading effects during scaling events. These enhancements are designed to reduce resource contention and improve system efficiency under high load conditions. The changes are currently undergoing validation and will be deployed following comprehensive testing. Completed on 05/21/2026. The Engineering team will enhance alert notification delivery to ensure alarms are reliably triggered and routed to the appropriate response teams as expected, enabling faster detection and more timely corrective action when issues arise. Completed on 05/22/2026. ### Incident Timeline \(UTC\) 05/11/2026 06:02 AM \(UTC\) - A customer case was opened, and Tech Support \(TS\) engineers began their initial validation and troubleshooting investigation. 05/11/2026 06:21 AM \(UTC\) - TS engineers notified the Network Operations Center \(NOC\) of the reported customer impact, and internal validation confirmed that site load times had degraded to approximately 15 seconds. 05/11/2026 06:22 AM \(UTC\) - Engineers identified a suspected cause and began remediation steps. 05/11/2026 06:25 AM \(UTC\) - The impact was resolved after engineers restarted the affected pods and validated recovery through internal testing, marking the end of customer impact. 05/11/2026 07:41 AM \(UTC\) - After extended validation and monitoring confirmed that the platform had stabilized, a major incident was formally declared to document the service impact. Following confirmation of sustained stability and no further customer impact, the incident was subsequently marked as resolved.

Read the full incident report →

Started May 07, 2026, 11:52 AM UTC· resolved May 07, 2026, 06:21 PM UTC

1. investigating May 07, 2026, 11:52 AM UTC

   CXone Mpower Expert Service Degradation. The CXone Mpower Expert Engineering team is investigating reports of Service Degradation.

2. investigating May 07, 2026, 12:08 PM UTC

   We are continuing to investigate this issue.

3. investigating May 07, 2026, 12:23 PM UTC

   We are continuing to investigate this issue.

4. investigating May 07, 2026, 12:40 PM UTC

   We are continuing to investigate this issue.

5. investigating May 07, 2026, 12:54 PM UTC

   We are continuing to investigate this issue.

6. investigating May 07, 2026, 01:18 PM UTC

   We are continuing to investigate this issue.

7. investigating May 07, 2026, 01:42 PM UTC

   We are continuing to investigate this issue.

8. monitoring May 07, 2026, 01:52 PM UTC

   A fix has been implemented and we are monitoring the results.

9. investigating May 07, 2026, 04:21 PM UTC

   CXone Mpower Expert Service Degradation: Sites Service Degradation. The CXone Mpower Expert Engineering team is investigating reports of Site Service Degradation.

10. identified May 07, 2026, 05:40 PM UTC

    CXone Knowledge Management Service Degradation: Sites unavailable. The issue has been identified and a fix is being worked on for deployment.

11. monitoring May 07, 2026, 05:56 PM UTC

    CXone Knowledge Management - Fix Deployed - All Services Running Normally. The CXone Knowledge Management Engineering team has deployed a fix and all services are running normally. We are currently monitoring sites for deployment stability.

12. resolved May 07, 2026, 06:21 PM UTC

    CXone Knowledge Management - Service Disruption Resolved - All Services Running Normally. The NiCE Knowledge Management Engineering team has deployed a fix and monitored the deployment to make sure sites are stable. The issue is now resolved at this time. Event duration - 5hr 48m

13. postmortem May 14, 2026, 03:01 PM UTC

    **Major Incident# 02793015** **Impact Start Time \(UTC\) 05/07/2026 11:43 AM UTC** **Impact End Time \(UTC\) 05/07/2026 02:07 PM UTC** ### Incident Summary On 05/07/2026, some NiCE CXone Mpower customers experienced slowness when accessing sites, while others were unable to access the platform entirely with a “504 Gateway Timeout” error within the CXone Mpower Expert knowledge portal. The recurring service degradation incidents were caused by increased traffic volumes combined with performance limitations in certain backend processes, primarily impacting the US regional platform due to higher demand. The impact was resolved after scaling up pod resources and restarting the proxy pods, which restored platform stability. ### Root Cause The recurring service degradation incidents were caused by increased traffic volumes combined with performance limitations in certain backend processes, primarily impacting the US regional platform due to higher demand. Under elevated traffic conditions, including automated crawler activity, some requests followed less optimized processing paths, increasing system load. This was further amplified by legacy or complex page content requiring more intensive processing. While scaling actions helped restore capacity, they also introduced temporary overhead that contributed to intermittent performance degradation. Additionally, although autoscaling functioned as designed, it reached its limits and was insufficient to address constraints related to per-pod Central Processing Unit \(CPU\) capacity. Overall, evolving traffic patterns exposed underlying performance limitations, highlighting the need for targeted code optimizations and increased per service capacity. ### Corrective Actions **Detection:** Although built in alerting mechanisms were in place to detect this type of condition, alerts did not consistently reach the responsible teams as expected. In some cases, alerts were grouped or suppressed, delaying timely visibility of the issue. As a result, internal teams became aware Corrective Actions of the impact primarily through customer reports of slowness when accessing sites within the CXone Mpower Expert knowledge portal. **Remediation:** The impact was resolved after scaling up pod resources and restarting the proxy pods, which restored platform stability. Completed on 05/07/2026. **Prevention:** The Engineering team enhanced traffic filtering rules at the Web Application Firewall \(WAF\) layer to identify and block a significant portion of automated bot traffic contributing to elevated system load. These actions reduced unnecessary requests and improved overall platform stability. Completed on 05/07/2026. The Engineering team implemented interim mitigation measures to maintain system stability and ensure consistent performance while permanent improvements are finalized and deployed. Completed on 05/12/2026. Baseline system capacity was increased by raising the minimum number of pods and allocating higher CPU resources. This ensures sufficient resources are consistently available, reduces reliance on dynamic scaling, and improves overall system stability during periods of increased demand. Completed on 05/12/2026. The Engineering team will implement targeted software optimizations, including improvements to a specific endpoint that previously introduced cascading effects during scaling events. These enhancements are designed to reduce resource contention and improve system efficiency under high load conditions. The changes are currently undergoing validation and will be deployed following comprehensive testing. An update will be provided by End of Day \(EOD\) MT on 05/22/2026. The Engineering team will enhance alert notification delivery to ensure alarms are reliably triggered and routed to the appropriate response teams as expected, enabling faster detection and more timely corrective action when issues arise. An update will be provided by EOD MT on 05/22/2026. ### Incident Timeline \(UTC\) 05/07/2026 11:43 AM \(UTC\) - The first customer case opened, and Tech Support \(TS\) engineers began the troubleshooting investigation. 05/07/2026 11:44 AM \(UTC\) - TS engineers notified the Network Operations Center \(NOC\) engineers about the reported customer impact; a major incident was proposed and confirmed. 05/07/2026 12:09 PM \(UTC\) - Engineers identified a suspected cause and increased the resources of the web pods to improve system performance. 05/07/2026 12:18 PM \(UTC\) - Engineers also scaled up resources for the Application Programming Interface \(API\) pods to further stabilize performance. 05/07/2026 01:30 PM \(UTC\) - The platform continued to catch-up and engineers were already seeing improvements in system performance. 05/07/2026 01:52 PM \(UTC\) - Engineers restarted proxy pods, resulting in continued performance improvements while monitoring system stability. 05/07/2026 02:00 PM \(UTC\) - Platform performance returned to normal levels, with continued validation and monitoring underway. 05/07/2026 02:07 PM \(UTC\) - The platform stabilized fully. The impact was resolved following resource scaling, and after successful validation, the major incident was marked as resolved.

Read the full incident report →

Started Apr 29, 2026, 11:19 AM UTC· resolved Apr 29, 2026, 12:13 PM UTC

1. investigating Apr 29, 2026, 11:19 AM UTC

   CXone Knowledge Management Service Degradation: Sites unavailable. The CXone Knowledge Management Engineering team is investigating reports of site unavailability.

2. investigating Apr 29, 2026, 11:38 AM UTC

   CXone Knowledge Management Service Degradation: Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

3. investigating Apr 29, 2026, 11:54 AM UTC

   CXone Knowledge Management - Fix Deployed - All Services Running Normally. The CXone Mpower Expert Engineering team has deployed a fix and all services are running normally. We are currently monitoring sites for deployment stability. Event duration 51 mins

4. monitoring Apr 29, 2026, 11:55 AM UTC

   A fix has been implemented and we are monitoring the results.

5. resolved Apr 29, 2026, 12:13 PM UTC

   Service Disruption Resolved - All Services Running Normally. The CXone Knowledge Management Engineering team has deployed a fix and monitored the deployment to make sure sites are stable. The issue is now resolved at this time. Event duration 51 mins

6. postmortem May 05, 2026, 06:32 PM UTC

   **Impact Start Time \(UTC\) 04/29/2026 10:31 AM UTC** **Impact End Time \(UTC\) 04/29/2026 11:56 AM UTC** **Incident Summary** On 04/29/2026, some NiCE CXone Mpower customers initially experienced slowness when accessing sites within the CXone Mpower Expert knowledge portal. The issue subsequently worsened, leading to complete inaccessibility, with users encountering “503 Service Unavailable” and “504 Gateway Timeout” errors. The service disruption was caused by a short-term resource saturation within one of the platform’s backend Application Programming Interface \(API\)\* components. The issue was resolved by scaling up multiple services, increasing Central Processing Unit \(CPU\) allocation per-pod to better handle traffic spikes, and restarting the affected API services to apply the updated resource configurations, restoring normal service operation. **Root Cause** The service disruption was caused by a short-term resource saturation within one of the platform’s backend API components. Under certain conditions, a burst of long running requests placed higher than expected demand on processing resources, leading to slower response times and errors for some customers. Certain subset of API requests required significantly more CPU than others. When many of these requests were processed concurrently, CPU utilization on affected pods became saturated, causing requests to slow down or time out. This saturation cascaded to upstream services, more broadly impacting site availability. Although a built in autoscaling mechanism was in place and functioned as designed, it reached its maximum limit and proved insufficient because the underlying issue was related to per-pod CPU constraints rather than the number of pods. The observed workload pattern required additional per-service processing capacity to compensate for these limitations and restore stable operation. ### Corrective Actions: **Detection:** Although a builtin alerting mechanism was in place to detect the failure condition, the alarm did not trigger to the responsible team as expected. Subsequently, internal support teams began receiving customer reports of slowness when accessing sites within the CXone Mpower Expert knowledge portal. The condition later escalated to complete service inaccessibility, with users encountering “503 Service Unavailable” and “504 Gateway Timeout” errors. **Remediation:** The issue was resolved by scaling up multiple services, increasing CPU allocation per-pod to better handle traffic spikes, and restarting the affected API services to apply the updated resource configurations, restoring normal service operation. Completed on 04/29/2026. **Prevention:** The Engineering team will enhance alert notification delivery to ensure alarms are reliably triggered and routed to the appropriate response teams as expected, enabling faster detection and more timely corrective action when issues arise. An update will be provided by End of Day \(EOD\) MT on 05/15/2026. While the risk of recurrence is currently low following the increase in resource capacity, the Engineering team will implement targeted performance improvements to the affected API component that contributed to this incident. In addition, code level safeguards will be introduced to ensure that, under similar conditions, inefficient requests are terminated sooner, and system resources are released more quickly. These measures are intended to enable faster failure handling and prevent prolonged CPU saturation. An update will be provided by EOD MT on 06/05/2026. **Incident Timeline \(UTC\)** 04/29/2026 10:31 AM \(UTC\) - Telemetry detected initial slow responses and timeout errors, marking the start of customer impact. 04/29/2026 10:51 AM \(UTC\) - The first customer case opened, and Tech Support \(TS\) engineers began the troubleshooting investigation. 04/29/2026 11:20 AM \(UTC\) - TS engineers notified the Network Operations Center \(NOC\) engineers about the reported customer impact; a major incident was proposed and confirmed. 04/29/2026 11:41 AM \(UTC\) - Engineers identified a suspected cause and began remediation steps. 04/29/2026 11:56 AM \(UTC\) - The issue was resolved after scaling up resources and restarting the affected API services. Following successful validation, the major incident was marked as resolved.

Read the full incident report →

Started Apr 24, 2026, 10:35 PM UTC· resolved Apr 24, 2026, 10:58 PM UTC

1. investigating Apr 24, 2026, 10:35 PM UTC

   MindTouch Engineering is looking into issues related to slow load time on sites.

2. monitoring Apr 24, 2026, 10:49 PM UTC

   A fix has been implemented and we are monitoring the results.

3. resolved Apr 24, 2026, 10:58 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Mar 16, 2026, 01:10 PM UTC· resolved Mar 16, 2026, 02:18 PM UTC

1. investigating Mar 16, 2026, 01:10 PM UTC

   CXone Mpower Expert Service Degradation: Site Degradation. The CXone Mpower Expert Engineering team is investigating reports of site degradation.

2. investigating Mar 16, 2026, 01:26 PM UTC

   CXone Mpower Expert Service Degradation: Site Degradation. The CXone Mpower Expert Engineering team is investigating reports of site degradation.

3. investigating Mar 16, 2026, 01:41 PM UTC

   CXone Mpower Expert Service Degradation: Site Degradation. The CXone Mpower Expert Engineering team is investigating reports of site degradation.

4. monitoring Mar 16, 2026, 01:45 PM UTC

   A fix has been implemented and we are monitoring the results.

5. resolved Mar 16, 2026, 02:18 PM UTC

   This incident has been resolved.

6. postmortem Mar 25, 2026, 05:09 PM UTC

   **Impact Start Time \(UTC\)** - 03/16/2026 12:55 PM UTC **Impact End Time \(UTC\)** - 03/16/2026 01:44 PM UTC ## **Incident Summary** Updated on 03/24/2025 - On 03/16/2026, some NiCE CXone Mpower customers experienced degraded performance and intermittent errors within the CXone Mpower Expert application. The impact was caused by elevated system utilization resulting from increased traffic load, which placed excessive demand on the high-availability load-balancing layer. This led to increased response times and reduced request processing efficiency. The impact was resolved when engineers increased the pod capacity of both the web application tier and the load balancer later, thereby restoring system throughput, stabilizing response times, and returning the platform to normal operating conditions. ## **Root Cause** The impact was caused by elevated system utilization resulting from increased traffic load, which placed excessive demand on the high-availability load-balancing layer. This led to increased response times and reduced request processing efficiency. Further investigation determined that the performance degradation was caused by a combination of constrained application processing capacity and insufficient load-balancing scale under peak traffic conditions. Web worker throughput was limited due to a known performance constraint, which reduced the rate at which incoming requests could be processed. This condition was further amplified by the load balancing layer operating below its expected production capacity, resulting in suboptimal traffic distribution and increased system latency during periods of elevated demand. Engineering teams are implementing targeted preventive actions to improve auto-scaling responsiveness and ensure the timely promotion of critical performance fixes to production. ## **Corrective Actions** **Detection:** Internal teams received alerts indicating increased of high availability load balancer response times. Shortly after, customers reported degraded performance and intermittent errors within the CXone Mpower Expert application. **Remediation:** The impact was resolved when engineers increased the pod capacity of both the web application tier and the load balancer later, thereby restoring system throughput, stabilizing response times, and returning the platform to normal operating conditions. Completed on 03/16/2026. **Prevention:** Engineers will adjust the auto-scaling policies to improve responsiveness and enable more aggressive scale-out behavior during peak traffic periods, ensuring capacity aligns with demand in near real time. An update will be provided by End of Day \(EOD\) MT on 03/26/2026. Engineers will implement auto-scaling for the high-availability load-balancing layer to ensure traffic distribution capacity scales proportionally with application workload. An update will be provided by EOD MT on 03/26/2026. ## **Incident Timeline \(UTC\)** 03/16/2026 12:55 PM \(UTC\) - First customer case opened, and Tech Support \(TS\) engineers began the troubleshooting investigation. 03/16/2026 01:07 PM \(UTC\) - TS engineers notified the Network Operations Center \(NOC\) engineers about the reported customer impact; a major incident was proposed and confirmed. 03/16/2026 01:44 PM \(UTC\) - Impact was resolved after engineers made an increase in available resources and internal tests were successful. Impact and major incident resolved.

Read the full incident report →

Started Mar 03, 2026, 04:37 PM UTC· resolved Mar 03, 2026, 06:34 PM UTC

1. investigating Mar 03, 2026, 04:37 PM UTC

   CXone Mpower Expert Service Degradation: Degradation Performance. The CXone Mpower Expert Engineering team is investigating reports of site Degradation Performance.

2. investigating Mar 03, 2026, 04:54 PM UTC

   CXone Mpower Expert Service Degradation: Degraded Performance. The CXone Mpower Expert Engineering team is investigating reports of degraded performance.

3. investigating Mar 03, 2026, 05:09 PM UTC

   CXone Mpower Expert Service Degradation: Degraded Performance. The CXone Mpower Expert Engineering team is investigating reports of degraded performance.

4. monitoring Mar 03, 2026, 05:22 PM UTC

   A fix has been implemented and we are monitoring the results.

5. resolved Mar 03, 2026, 06:34 PM UTC

   This incident has been resolved.

6. postmortem Mar 16, 2026, 10:10 PM UTC

   **Impact Start Time** 03/03/2026 02:24 PM UTC **Impact End Time** 03/03/2026 05:00 PM UTC **Incident Summary** Updated on 03/11/2026 - On 3/3/2026, some NiCE CXone Mpower customers elevated latency within the CXone Expert application, resulting in delayed retrieval of knowledge-base articles and impacting agents’ ability to efficiently access customer support content. The impact stemmed from a processing latency issue within the application layer. The impact was resolved when engineers restored normal service by scaling up the infrastructure components responsible for handling inbound web requests, ensuring sufficient processing capacity to stabilize throughput and eliminate request bottlenecks. **Root Cause** The root cause stemmed from a processing latency issue for the application layer. The system briefly took longer than usual to handle requests. For a short period, more activity came in than the system typically manages at once, which slowed things down. As a result, some requests took longer to finish and a few timed out, leading to the performance issues customers noticed. **Corrective Actions** _Detection_: Internal support teams received a customer reported issue and confirmed an ongoing latency issue with the CXone Expert application. _Remediation_: Engineers restored normal service by scaling up the infrastructure components responsible for handling inbound web requests, ensuring sufficient processing capacity to stabilize throughput and eliminate request bottlenecks. Completed on 03/03/2026. _Prevention_: The Engineering team developed and deployed a code fix to address an underlying issue with instance scaling. Completed on 03/05/2026. **Incident Timeline \(UTC\)** 03/03/2026 02:24 PM \(UTC\) - First customer case opened, and Tech Support \(TS\) engineers began the troubleshooting investigation. 03/03/2026 04:31 PM \(UTC\) - TS engineers notified the Network Operations Center \(NOC\) engineers about the reported customer impact; a major incident was proposed and confirmed. 03/03/2026 04:35 PM \(UTC\) - Engineers identified a suspected cause and began remediation steps. 03/03/2026 05:00 PM \(UTC\) - Impact was resolved after engineers increased capacity for components responsible for processing web requests and internal tests were successful. Impact and major incident resolved.

Read the full incident report →

Started Feb 06, 2026, 01:39 PM UTC· resolved Feb 06, 2026, 10:24 PM UTC

1. investigating Feb 06, 2026, 01:39 PM UTC

   CXone Knowledge Management Service Degradation: Sites are experiencing slow load times and sometimes unavailable. The CXone Knowledge Management Engineering team is investigating reports of slow load times and sometimes not resolving.

2. investigating Feb 06, 2026, 01:55 PM UTC

   CXone Knowledge Management Service Degradation: The Knowledge Management Engineering team is continuing to investigate reports of slow load times and sometimes not resolving.

3. investigating Feb 06, 2026, 02:11 PM UTC

   CXone Knowledge Management Service Degradation: The Knowledge Management Engineering team is continuing to investigate reports of slow load times and sometimes not resolving.

4. investigating Feb 06, 2026, 02:26 PM UTC

   CXone Knowledge Management Service Degradation: The Knowledge Management Engineering team is continuing to investigate reports of slow load times and sometimes not resolving.

5. identified Feb 06, 2026, 02:42 PM UTC

   CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

6. identified Feb 06, 2026, 02:58 PM UTC

   CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

7. identified Feb 06, 2026, 03:13 PM UTC

   CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

8. identified Feb 06, 2026, 03:30 PM UTC

   CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

9. identified Feb 06, 2026, 03:46 PM UTC

   CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

10. identified Feb 06, 2026, 04:00 PM UTC

    CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

11. identified Feb 06, 2026, 04:15 PM UTC

    CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

12. identified Feb 06, 2026, 04:30 PM UTC

    CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

13. identified Feb 06, 2026, 05:09 PM UTC

    CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

14. identified Feb 06, 2026, 05:24 PM UTC

    CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

15. identified Feb 06, 2026, 05:44 PM UTC

    CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

16. identified Feb 06, 2026, 06:01 PM UTC

    CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

17. identified Feb 06, 2026, 06:17 PM UTC

    CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

18. identified Feb 06, 2026, 06:41 PM UTC

    CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

19. identified Feb 06, 2026, 06:57 PM UTC

    CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

20. identified Feb 06, 2026, 07:12 PM UTC

    CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

21. identified Feb 06, 2026, 07:27 PM UTC

    CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

22. identified Feb 06, 2026, 07:44 PM UTC

    CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

23. identified Feb 06, 2026, 08:00 PM UTC

    CXone Knowledge Management Service Degradation: The issue has been identified and a fix is being worked on for deployment.

24. monitoring Feb 06, 2026, 08:22 PM UTC

    CXone Knowledge Management – Service Degradation: Stable All U.S. sites are currently in a stable state, and our teams are monitoring to ensure continued recovery.

25. monitoring Feb 06, 2026, 09:01 PM UTC

    CXone Knowledge Management – Service Degradation: Stable All U.S. sites are currently in a stable state, and our teams are monitoring to ensure continued recovery.

26. monitoring Feb 06, 2026, 09:21 PM UTC

    CXone Knowledge Management – Service Degradation: Stable All U.S. sites are currently in a stable state, and our teams are monitoring to ensure continued recovery.

27. monitoring Feb 06, 2026, 09:39 PM UTC

    CXone Knowledge Management – Service Degradation: Stable All U.S. sites are currently in a stable state, and our teams are monitoring to ensure continued recovery.

28. monitoring Feb 06, 2026, 09:55 PM UTC

    CXone Knowledge Management – Service Degradation: Stable All U.S. sites are currently in a stable state, and our teams are monitoring to ensure continued recovery.

29. monitoring Feb 06, 2026, 10:13 PM UTC

    CXone Knowledge Management – Service Degradation: Stable All U.S. sites are currently in a stable state, and our teams are monitoring to ensure continued recovery.

30. resolved Feb 06, 2026, 10:24 PM UTC

    This incident has been resolved.

Read the full incident report →

Started Feb 05, 2026, 06:26 PM UTC· resolved Feb 06, 2026, 04:10 AM UTC

1. investigating Feb 05, 2026, 06:26 PM UTC

   CXone Mpower Knowledge Management Service Degradation: Sites are loading slow or unavailable. The CXone Knowledge Management Engineering team is investigating reports of slow load times and occasional site unavailability.

2. investigating Feb 05, 2026, 06:41 PM UTC

   CXone Knowledge Management Service Degradation: Slow Load Times and Unavailability. The Engineering team is still investigating at this time.

3. investigating Feb 05, 2026, 06:57 PM UTC

   CXone Knowledge Management Service Degradation: Slow Load Times and Unavailability. The Engineering team is still investigating at this time.

4. investigating Feb 05, 2026, 07:12 PM UTC

   CXone Knowledge Management Service Degradation: Slow Load Times and Unavailability. The Engineering team is still investigating at this time.

5. investigating Feb 05, 2026, 07:29 PM UTC

   CXone Knowledge Management Service Degradation: Slow Load Times and Unavailability. The Engineering team is still investigating at this time.

6. identified Feb 05, 2026, 07:45 PM UTC

   The issue has been identified and a fix is being worked on for deployment.

7. identified Feb 05, 2026, 08:01 PM UTC

   The issue has been identified and a fix is being worked on for deployment.

8. identified Feb 05, 2026, 08:17 PM UTC

   The issue has been identified and a fix is being worked on for deployment.

9. identified Feb 05, 2026, 08:33 PM UTC

   The issue has been identified and a fix is being worked on for deployment.

10. identified Feb 05, 2026, 08:53 PM UTC

    The issue has been identified and a fix is being worked on for deployment.

11. identified Feb 05, 2026, 09:08 PM UTC

    We are continuing to work on a fix for this issue.

12. identified Feb 05, 2026, 09:24 PM UTC

    We are continuing to work on a fix for this issue.

13. identified Feb 05, 2026, 09:39 PM UTC

    We are continuing to work on a fix for this issue.

14. monitoring Feb 05, 2026, 10:33 PM UTC

    A fix has been implemented and we are monitoring the results.

15. monitoring Feb 05, 2026, 10:48 PM UTC

    We are continuing to monitor for any further issues.

16. monitoring Feb 05, 2026, 11:05 PM UTC

    We are continuing to monitor for any further issues.

17. monitoring Feb 05, 2026, 11:31 PM UTC

    We are continuing to monitor for any further issues.

18. monitoring Feb 05, 2026, 11:47 PM UTC

    We are continuing to monitor for any further issues.

19. monitoring Feb 06, 2026, 12:04 AM UTC

    We are continuing to monitor for any further issues.

20. monitoring Feb 06, 2026, 12:19 AM UTC

    We are continuing to monitor for any further issues.

21. monitoring Feb 06, 2026, 12:34 AM UTC

    We are continuing to monitor for any further issues.

22. monitoring Feb 06, 2026, 12:50 AM UTC

    We are continuing to monitor for any further issues.

23. monitoring Feb 06, 2026, 01:05 AM UTC

    We are continuing to monitor for any further issues.

24. monitoring Feb 06, 2026, 01:20 AM UTC

    We are continuing to monitor for any further issues.

25. monitoring Feb 06, 2026, 01:36 AM UTC

    We are continuing to monitor for any further issues.

26. monitoring Feb 06, 2026, 01:51 AM UTC

    We are continuing to monitor for any further issues.

27. monitoring Feb 06, 2026, 02:06 AM UTC

    We are continuing to monitor for any further issues.

28. monitoring Feb 06, 2026, 02:21 AM UTC

    We are continuing to monitor for any further issues.

29. monitoring Feb 06, 2026, 02:36 AM UTC

    We are continuing to monitor for any further issues.

30. monitoring Feb 06, 2026, 02:52 AM UTC

    We are continuing to monitor for any further issues.

31. monitoring Feb 06, 2026, 03:09 AM UTC

    We are continuing to monitor for any further issues.

32. monitoring Feb 06, 2026, 03:25 AM UTC

    We are continuing to monitor for any further issues.

33. monitoring Feb 06, 2026, 03:52 AM UTC

    We are continuing to monitor for any further issues.

34. resolved Feb 06, 2026, 04:10 AM UTC

    This incident has been resolved.

Read the full incident report →

Started Feb 03, 2026, 06:15 PM UTC· resolved Feb 04, 2026, 12:00 AM UTC

1. investigating Feb 03, 2026, 06:15 PM UTC

   CXone Knowledge Service Degradation: Slow Load times and occasional page loading failure. The CXone Knowledge Engineering team is investigating reports of slow load times and occasional page load failure.

2. monitoring Feb 03, 2026, 06:25 PM UTC

   CXone Knowledge - Fix Deployed - All Services Running Normally. The CXone Knowledge Engineering team has deployed a fix and all services are running normally. We are currently monitoring sites for deployment stability. Event duration 9 minutes

3. identified Feb 03, 2026, 08:20 PM UTC

   CXone Knowledge Service Degradation: Slow Load times and occasional page loading failure. The CXone Knowledge Engineering team is investigating reports of slow load times and occasional page load failure.

4. identified Feb 03, 2026, 08:53 PM UTC

   We are continuing to work on a fix for this issue.

5. identified Feb 03, 2026, 09:09 PM UTC

   We are continuing to work on a fix for this issue.

6. identified Feb 03, 2026, 09:24 PM UTC

   We are continuing to work on a fix for this issue.

7. identified Feb 03, 2026, 09:40 PM UTC

   We are continuing to work on a fix for this issue.

8. monitoring Feb 03, 2026, 09:43 PM UTC

   A fix has been implemented and we are monitoring the results.

9. monitoring Feb 03, 2026, 09:59 PM UTC

   We are continuing to monitor for any further issues.

10. monitoring Feb 03, 2026, 10:16 PM UTC

    CXone Knowledge - Fix Deployed - All Services Running Normally. The CXone Knowledge Engineering team has deployed a fix and all services are running normally. We are currently monitoring sites for deployment stability. Event duration 9 minutes

11. monitoring Feb 03, 2026, 10:31 PM UTC

    CXone Knowledge - Fix Deployed - All Services Running Normally. The CXone Knowledge Engineering team has deployed a fix and all services are running normally. We are currently monitoring sites for deployment stability. Event duration 9 minutes

12. resolved Feb 04, 2026, 12:00 AM UTC

    This incident has been resolved.

Read the full incident report →

Started Jan 27, 2026, 03:32 PM UTC· resolved Jan 27, 2026, 06:09 PM UTC

1. investigating Jan 27, 2026, 03:32 PM UTC

   CXone Mpower Expert Service Degradation: Search unavailable. The CXone Mpower Expert Engineering team is investigating reports of search unavailability.

2. investigating Jan 27, 2026, 04:03 PM UTC

   CXone Mpower Expert Service Degradation: Search unavailable. The CXone Mpower Expert Engineering team is investigating reports of search unavailability.

3. investigating Jan 27, 2026, 04:27 PM UTC

   CXone Mpower Expert Service Degradation: Search unavailable. The CXone Mpower Expert Engineering team is investigating reports of search unavailability.

4. monitoring Jan 27, 2026, 04:38 PM UTC

   A fix has been implemented and we are monitoring the results.

5. resolved Jan 27, 2026, 06:09 PM UTC

   This incident has been resolved.

6. postmortem Mar 16, 2026, 11:05 PM UTC

   **Impact Start Time \(UTC\):** 01/27/2026 03:16 PM UTC **Impact End Time \(UTC\):** 01/27/2026 04:20 PM UTC ‌ **Incident Summary:** Updated on 02/02/2026 - On 1/27/2026, some NiCE CXone Mpower customers reported significant functional and performance degradation within the CXone Mpower Expert application. Affected users experienced pronounced latency, recurring Hyper Text Transfer Protocol \(HTTP\) 503 service unavailable errors, and search related faults. These issues collectively resulted in Knowledge Base \(KB\) pages failing to load, along with intermittent authentication issues. The impact stemmed from a severe socket exhaustion between the Application Programming Interface \(API\) service and the proxy tier. The issue was fully mitigated after engineers executed a rolling update of the API backend services, which re-established socket connections and restored normal functionality. ‌ **Root Cause:** The incident was traced to socket exhaustion occurring between the API service and the proxy layer, driven by a combination of prolonged network timeouts and the application’s connection retry behavior. Intermittent network latency caused existing socket connections to remain in a hung or half-open state, preventing them from closing and being returned to the connection pool. When the application initiated retry attempts, it failed to reuse these open sockets and instead created additional outbound connections. This pattern progressively consumed all available socket resources allocated by the proxy tier. Once socket capacity was saturated, the API service could no longer establish new connections to downstream backend services, resulting in significant degradation of search operations. Under these constrained conditions, only a small subset of search requests successfully completed, while the majority failed with service level errors due to connection unavailability. Key contributing factors included insufficient observability into socket consumption and proxy connection thresholds, which delayed detection and mitigation. In addition, the proxy’s connection limits were not provisioned to accommodate the increased load generated by retry amplification during periods of network instability. ‌ **Corrective Actions:** _Detection_: Internal support teams received a customer reported issue and confirmed an ongoing functional and performance degradation within the CXone Mpower Expert application. _Remediation_: Engineers executed a rolling update of the API backend services, which re-established socket connections and restored normal functionality. Completed on 01/27/2026. _Prevention_: The Engineering team is implementing a monitoring enhancement and alerting to proactively detect abnormal connection usage and capacity saturation before customer impact occurs. Internal teams are implementing an increase in connection and service capacity to better handle peak traffic and retry scenarios. Additionally, improvements are being made to capacity planning to account for traffic spikes and cascading retry behavior. An update will be provided by EOD MT of 03/02/2026. ‌ **Incident Timeline \(UTC\):** 01/27/2026 03:16 PM \(UTC\) - First customer case opened, and Tech Support \(TS\) engineers began the troubleshooting investigation. 01/27/2026 03:41 PM \(UTC\) - TS engineers notified the Network Operations Center \(NOC\) engineers about the reported customer impact; a major incident was proposed and confirmed. 01/27/2026 04:20 PM \(UTC\) - Impact was resolved after engineers restarted services and internal tests were successful. Impact and major incident resolved.

Read the full incident report →

Started Jan 13, 2026, 01:13 AM UTC· resolved Jan 13, 2026, 02:50 AM UTC

1. investigating Jan 13, 2026, 01:13 AM UTC

   CXone Expert Service Degradation: Sites unavailable. The CXone Expert Engineering team is investigating reports of site unavailability.

2. investigating Jan 13, 2026, 01:28 AM UTC

   CXone Expert Service Degradation: Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

3. investigating Jan 13, 2026, 01:44 AM UTC

   CXone Expert Service Degradation: Sites unavailable. The CXone Expert Engineering team is investigating reports of site unavailability.

4. investigating Jan 13, 2026, 01:59 AM UTC

   CXone Expert Service Degradation: Sites unavailable. The CXone Expert Engineering team is investigating reports of site unavailability

5. identified Jan 13, 2026, 02:08 AM UTC

   CXone Expert Service Degradation: Sites unavailable. The issue has been identified and a fix is being worked on for deployment.

6. monitoring Jan 13, 2026, 02:17 AM UTC

   CXone Expert - Fix Deployed - All Services Running Normally. The CXone Mpower Expert Engineering team has deployed a fix and all services are running normally. We are currently monitoring sites for deployment stability. Event duration (1h 3m)

7. monitoring Jan 13, 2026, 02:34 AM UTC

   CXone Expert - Fix Deployed - All Services Running Normally. The CXone Mpower Expert Engineering team has deployed a fix and all services are running normally. We are currently monitoring sites for deployment stability. Event duration (1h 3m)

8. resolved Jan 13, 2026, 02:50 AM UTC

   CXone Expert - Service Disruption Resolved - All Services Running Normally. The CXone Expert Engineering team has deployed a fix and monitored the deployment to make sure sites are stable. The issue is now resolved at this time. Event duration (1h 3m)

9. postmortem Jan 20, 2026, 08:47 PM UTC

   **Impact Start Time** \(UTC\) 01/13/2026 01:05 AM UTC **Impact End Time** \(UTC\) 01/13/2026 02:15 AM UTC **Incident Summary** Updated on 01/20/2026 - On 01/13/2026, some NiCE CXone Mpower customers reported experiencing service disruptions when accessing the CXone Mpower Expert knowledge portal. During this time, incorrect authentication settings in the production environment prevented users from successfully signing in. The issue was traced to an unexpected behaviour on a deployment tool that caused a component to be deployed to multiple environments, including production, at the same time. The impact was resolved when engineers corrected the configuration by redeploying the correct resources and configuration to the production environment. Following the fix, user access, cluster health, and overall platform functionality were validated, confirming full restoration of service. **Root Cause** During routine deployment testing, a commandline interface \(CLI\) tool encountered an edgecase condition that caused part of a deployment script to run across several environments simultaneously. Although the script commands were valid, the tool incorrectly mixed environment context, one role was targeting a lower environment while another component pointed to production. This resulted in unintended changes to Identity and Access Management \(IAM\) settings in the production environment. The incorrect IAM configuration disrupted authentication workflows, making the portal temporarily inaccessible. * Detection: Internal support teams received reports from customer experiencing service disruptions when accessing the CXone Mpower Expert knowledge portal. * Remediation: The impact was resolved when engineers corrected the configuration by redeploying the correct resources and configuration to the production environment. Following the fix, user access, cluster health, and overall platform functionality were validated, confirming full restoration of service. Completed on 01/13/2026. **Corrective Actions** Prevention: * While the risk of recurrence is low, the Engineering team is working in standardizing the command-line interface tool’s environment initialization to prevent implicit profiles, regions, or cluster context leakage. This will include improved visibility into environment-targeted deployments by adding clearer logging of the target account, region, and cluster prior to execution to prevent similar incidents in the future. An update will be provided by End of Day \(EOD\) MT on 01/30/2026. * The Engineering team will implement separate deployment workflows for application resources and for cluster and IAM control plane components, minimizing the risk of accidental deployments to unintended environments due to configuration errors. An update will be provided by EOD MT on 01/30/2026 **Incident Timeline \(UTC\)** 01/13/2026 01:05 AM \(UTC\) - The first customer case opened, and Tech Support \(TS\) engineers began their initial validations and troubleshooting investigation. 01/13/2026 01:14 AM \(UTC\) - Engineers notified the Network Operations Center \(NOC\) engineers about the reported customer impact; a major incident was proposed and confirmed. The service disruption was also posted on the Status page for customers’ awareness. 01/13/2026 02:10 AM \(UTC\) - Engineers identified a suspected cause and began remediation steps. 01/13/2026 02:15 AM \(UTC\) - The impact was resolved after engineers corrected the configuration on the affected component. Following successful validations, the major incident was marked as resolved.

Read the full incident report →

Started Dec 31, 2025, 02:10 AM UTC· resolved Dec 31, 2025, 09:21 AM UTC

1. investigating Dec 31, 2025, 02:10 AM UTC

   CXone Mpower Expert Service Degradation: Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

2. investigating Dec 31, 2025, 02:29 AM UTC

   CXone Mpower Expert Service Degradation: Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

3. investigating Dec 31, 2025, 03:10 AM UTC

   CXone Mpower Expert Service Degradation: Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

4. investigating Dec 31, 2025, 03:30 AM UTC

   CXone Mpower Expert Service Degradation: Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

5. monitoring Dec 31, 2025, 03:34 AM UTC

   A fix has been implemented and we are monitoring the results.

6. investigating Dec 31, 2025, 03:52 AM UTC

   CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

7. investigating Dec 31, 2025, 04:16 AM UTC

   CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

8. investigating Dec 31, 2025, 04:32 AM UTC

   CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

9. investigating Dec 31, 2025, 04:51 AM UTC

   CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

10. investigating Dec 31, 2025, 05:06 AM UTC

    CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

11. investigating Dec 31, 2025, 05:23 AM UTC

    CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

12. investigating Dec 31, 2025, 05:38 AM UTC

    CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

13. investigating Dec 31, 2025, 05:53 AM UTC

    CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

14. investigating Dec 31, 2025, 06:11 AM UTC

    CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

15. investigating Dec 31, 2025, 06:25 AM UTC

    CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

16. investigating Dec 31, 2025, 06:41 AM UTC

    CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

17. investigating Dec 31, 2025, 06:56 AM UTC

    CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

18. investigating Dec 31, 2025, 07:11 AM UTC

    CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

19. investigating Dec 31, 2025, 07:53 AM UTC

    CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

20. investigating Dec 31, 2025, 08:24 AM UTC

    CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

21. investigating Dec 31, 2025, 08:42 AM UTC

    CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

22. investigating Dec 31, 2025, 08:59 AM UTC

    CXone Mpower Expert Service Degradation: US Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

23. resolved Dec 31, 2025, 09:21 AM UTC

    This incident has been resolved.

24. postmortem Jan 07, 2026, 10:42 PM UTC

    Impact Start Time \(UTC\) 12/31/2025 01:45 AM UTC Impact End Time \(UTC\) 12/31/2025 09:01 AM UTC **Incident Summary** Updated on 01/07/2026 - On 12/30/2025, some NiCE CXone Mpower customers reported experiencing intermittent service disruptions when utilizing the CXone Mpower Expert knowledge portal. The issue stemmed from a misconfigured network proxy, which prevented a critical cluster component from starting correctly on newly provisioned or restarted nodes. The impact was resolved after correcting the network proxy configuration and restarting the affected component pods, which fully restored service functionality. **Root Cause** The issue stemmed from a misconfigured network proxy, which prevented a critical cluster component from starting correctly on newly provisioned or restarted nodes. This led to node-specific failures in service routing and Domain Name System \(DNS\) resolution. Because the network proxy is responsible for programming node-level routing, the affected nodes were unable to reliably resolve or route traffic. The impact was limited to specific nodes and only affected workloads running on them, which accounts for the intermittent nature of the failures. Engineers later determined that the misconfiguration was introduced after enabling a component for enhanced logging due to an oversight. ## Corrective Actions **Detection:** Internal support teams received reports from customers indicating intermittent service disruptions when utilizing the CXone Mpower Expert knowledge portal. **Remediation:** The impact was resolved after correcting the network proxy configuration and restarting the affected component pods, which fully restored service functionality. Completed on 12/31/2025. **Prevention:** The Engineering team will implement enhanced monitoring and alerting for network proxy crash loops to proactively detect similar issues and enable immediate remediation, preventing customer impact. An update will be provided by End of Day \(EOD\) MT on 01/23/2026. The Engineering team is implementing an automated validation process for network proxy configuration changes to detect issues early and prevent future service disruptions. An update will be provided by EOD MT on 01/30/2026. **Incident Timeline \(UTC\)** 12/31/2025 01:45 AM \(UTC\) - The first customer case was opened, and Tech Support \(TS\) engineers began their initial validations troubleshooting investigation. 12/31/2025 02:31 AM \(UTC\) - TS engineers notified the Network Operations Center \(NOC\) engineers about the reported customer impact; a major incident was proposed and confirmed. 12/31/2025 03:00 AM \(UTC\) - Engineers observed that sites were restored but were still experiencing latency. 12/31/2025 04:40 AM \(UTC\) - Engineers restarted the pods to allow the system to create new pods. 12/31/2025 04:54 AM \(UTC\) - The affected sites were working fine but engineers were still seeing proxy errors on the backend. 12/31/2025 05:22 AM \(UTC\) - The proxy errors were still present, and engineers found DNS failures. Investigation and troubleshooting continued. 12/31/2025 06:09 AM \(UTC\) - Engineers restarted the deploy and infrastructure pipelines. 12/31/2025 08:00 AM \(UTC\) - Engineers reported that the automated deployment of pipelines failed and will need to perform manual deployment. 12/31/2025 09:01 AM \(UTC\) - The impact was resolved after engineers corrected the proxy configuration and restarted the pods. Following successful test validations, the major incident was marked as resolved.

Read the full incident report →

Started Nov 07, 2025, 03:21 PM UTC· resolved Nov 07, 2025, 05:43 PM UTC

1. investigating Nov 07, 2025, 03:21 PM UTC

   CXone Mpower Expert Service Degradation: Generative Search Unavailable. The CXone Mpower Expert Engineering team is investigating reports of Generative Search not working.

2. investigating Nov 07, 2025, 03:40 PM UTC

   CXone Mpower Expert Service Degradation: Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of Generative Search being unavailable.

3. identified Nov 07, 2025, 03:42 PM UTC

   CXone Mpower Expert Service Degradation: Generative Search Unavailable. The issue has been identified and a fix is being worked on for deployment.

4. identified Nov 07, 2025, 04:00 PM UTC

   CXone Mpower Expert Service Degradation: Generative Search Unavailable. The issue has been identified and a fix is being worked on for deployment.

5. identified Nov 07, 2025, 04:15 PM UTC

   CXone Mpower Expert Service Degradation: Generative Search Unavailable. The issue has been identified and a fix is being worked on for deployment.

6. identified Nov 07, 2025, 04:30 PM UTC

   CXone Mpower Expert Service Degradation: Generative Search Unavailable. The issue has been identified and a fix is being worked on for deployment.

7. identified Nov 07, 2025, 04:45 PM UTC

   CXone Mpower Expert Service Degradation: Generative Search Unavailable. The issue has been identified and a fix is being worked on for deployment.

8. identified Nov 07, 2025, 05:00 PM UTC

   CXone Mpower Expert Service Degradation: Generative Search Unavailable. The issue has been identified and a fix is being worked on for deployment.

9. identified Nov 07, 2025, 05:15 PM UTC

   CXone Mpower Expert Service Degradation: Generative Search Unavailable. The issue has been identified and a fix is being worked on for deployment.

10. identified Nov 07, 2025, 05:40 PM UTC

    CXone Mpower Expert Service Degradation: Generative Search Unavailable. The issue has been identified and a fix is being worked on for deployment.

11. resolved Nov 07, 2025, 05:43 PM UTC

    This incident has been resolved.

12. postmortem Mar 16, 2026, 11:12 PM UTC

    **Impact Start Time \(UTC\):** 11/07/2025 03:05 PM UTC **Impact End Time \(UTC\):** 11/08/2025 05:43 AM UTC ‌ **Incident Summary:** Updated on 01/23/2026 - On 11/08/2025, a NiCE CXone Mpower customer reported that the Generative Search feature was failing to return any results within the CXone Mpower Expert knowledge portal. The issue stemmed from a period of heavy data ingestion, which led to high Central Processing Unit \(CPU\) utilization and a sharp increase in throttling on our third-party vendor’s database. The issue was resolved by scaling up CPU resources within our platform, allowing it to manage the increased utilization and restore affected services. ‌ **Root Cause:** The issue stemmed from a period of heavy data ingestion, which led to high CPU utilization and a sharp increase in throttling on our third-party vendor’s database. Monitoring metrics confirmed that the cluster was under substantial load due to intensive upsert and indexing operations. This activity temporarily saturated CPU resources, causing degraded performance and intermittent timeouts. Further investigation revealed that a large dataset was likely uploaded without batching, creating a concentrated indexing workload across all nodes. Additionally, some nodes restarted during the scaling process, adding to the temporary instability. These factors collectively resulted in severe latency and timeouts in the production cluster, leading to the customer-reported impact. ‌ **Corrective Actions:** _Detection_: While engineers were investigating latency and timeout issues on the platform, a customer reported that the Generative Search feature was not returning any results within the CXone Mpower Expert knowledge portal. _Remediation_: The issue was resolved by scaling up CPU resources within our platform, allowing it to manage the increased utilization and restore affected services. The vendor later confirmed that once the ingestion and indexing processes were completed, the cluster stabilized, and performance returned to normal. Completed on 11/07/2025. _Prevention_: Our Engineering team, in collaboration with our third-party vendor, will be implementing a new feature that provides the platform with a dedicated indexing cluster. This enhancement will improve data ingestion performance and help prevent similar incidents in the future. An update will be provided by End of Day MT on 03/20/2026. ‌ **Incident Timeline \(UTC\):** 11/07/2025 03:05 PM \(UTC\) - The customer case was opened, and Tech Support \(TS\) engineers initiated validation and troubleshooting. This was later confirmed to be related to the ongoing platform issue. 11/07/2025 03:19 PM \(UTC\) - After identifying potential impact on the production cluster, they notified the Network Operations Center \(NOC\) engineers. service outage notification was published on the Status portal to inform customers of the issue and its expected impact. 11/07/2025 03:22 PM \(UTC\) - A major incident was proposed and confirmed. 11/07/2025 03:42 PM \(UTC\) - Engineers, in collaboration with the third-party vendor, identified a probable root cause and initiated remediation efforts. 11/07/2025 05:43 PM \(UTC\) - The impact was resolved after engineers increased the capacity of CPU resources. Following successful test validations, the major incident was marked as resolved.

Read the full incident report →

Started Oct 20, 2025, 04:26 PM UTC· resolved Oct 20, 2025, 11:34 PM UTC

1. identified Oct 20, 2025, 04:26 PM UTC

   CXone Mpower Expert - Service Degradation detected - There is currently an active service degradation with Amazon Web Services which is causing degradation with GenSearch and some reports in Expert. We are awaiting further updates from Amazon and we will post those as they are received.

2. identified Oct 20, 2025, 05:17 PM UTC

   Amazon Web Services has updated that they are still working on mitigating the current issues and a fix is in validation process. Once we receive any further updates we will post those.

3. identified Oct 20, 2025, 06:35 PM UTC

   Amazon Web Services has updated that they are still working on mitigating the current issues and a fix is in validation process. Once we receive any further updates we will post those.

4. identified Oct 20, 2025, 07:56 PM UTC

   Amazon Web Services has updated that they are still working on mitigating the current issues and a fix is in validation process. Once we receive any further updates we will post those.

5. identified Oct 20, 2025, 08:30 PM UTC

   We are now seeing the issue reports loading as intended. GenSearch is still intermittent but we are awaiting further Amazon updates.

6. monitoring Oct 20, 2025, 09:36 PM UTC

   At this time we are now seeing GenSearch work as intended after further updates from Amazon Web Services. We will be monitoring further to make sure everything remains stable.

7. resolved Oct 20, 2025, 11:34 PM UTC

   CXone Mpower Expert - Service Disruption Resolved - All Services Running Normally.

Read the full incident report →

Started Oct 10, 2025, 03:19 AM UTC· resolved Oct 10, 2025, 03:37 AM UTC

1. monitoring Oct 10, 2025, 03:19 AM UTC

   CXone Mpower Expert Service Degradation: Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

2. resolved Oct 10, 2025, 03:37 AM UTC

   This incident has been resolved.

3. postmortem Oct 20, 2025, 06:00 PM UTC

   Impact Start Time \(UTC\) 10/10/2025 03:03 AM UTC Impact End Time \(UTC\) 10/10/2025 03:37 AM UTC **Incident Summary** Updated on 10/17/2025 - On 10/10/2025, some NiCE CXone Mpower customers reported an inability to access the CXone Mpower Expert knowledge portal, encountering a “504 Gateway Time-out” error on all pages. The issue stemmed from a scheduled, routine change release intended to provision new resources, which resulted in a mismatch between release versions and caused an interruption in traffic routing. The impact was resolved by correcting the release version on the affected application load balancer, which successfully restored the impacted production sites. **Root Cause** The issue stemmed from a scheduled, routine change release intended to provision new resources, which resulted in a mismatch between release versions and caused an interruption in traffic routing. During the deployment, manual intervention was required, which resulted in the removal of the older version while migrating to the new one. As traffic was transitioned, sites experienced downtime due to improper traffic draining, since the removal procedure was not aligned with the recommended automated process. The issue was ultimately traced to a flaw in the deployment process used as the software release strategy for one of the load balancer components from a previous release. Although the problem was quickly identified and promptly corrected during deployment, engineers recognized key opportunities to improve the deployment process further to prevent similar occurrences in the future **Corrective Actions** **Detection:** Internal support teams detected a potentially customer-impacting issue during a change deployment, which was later confirmed by customer reports indicating an inability to access the CXone Mpower Expert knowledge portal, encountering a “504 Gateway Time-out” error on all pages. **Remediation:** The impact was resolved by correcting the release version on the affected application load balancer, which successfully restored the impacted production sites. Completed on 10/10/2025. **Prevention:** The Engineering team will enhance the deployment process by upgrading it to ensure that each version uses its own instance. This improvement will allow versions to operate independently and prevent similar issues in the future. Furthermore, testing procedures will be strengthened to include both old and new release versions during the Staging phase. An update will be provided by End of Day \(EOD\) MT on 10/31/2025. The Engineering team is updating the deployment documentation and automation scripts to prevent residual deployments from interfering with traffic routing and to avoid similar incidents in the future. Furthermore, an additional validation stage will be added to the release process to verify the previous release and ensure configuration consistency before deployment. An update will be provided by EOD MT on 11/14/2025. **Incident Timeline \(UTC\)** 10/10/2025 03:03 AM \(UTC\) - Engineers identified a potentially customer-impacting issue during a change deployment. 10/10/2025 03:08 AM \(UTC\) - Engineers notified the Network Operations Center \(NOC\) engineers about a possible customer impact during a change deployment. 10/10/2025 03:16 AM \(UTC\) - The first customer case was opened, and Tech Support \(TS\) engineers initiated the troubleshooting investigation. This case was later confirmed to be related to the identified problem and subsequently linked to the major incident. 10/10/2025 03:24 AM \(UTC\) - A major incident was raised and confirmed due to a potential impact on customer services. 10/10/2025 03:37 AM \(UTC\) - Engineers rectified the configuration issues that caused the impact. Internal validation confirmed successful restoration of services, and the major incident was then marked as resolved.

Read the full incident report →

Started Oct 03, 2025, 06:23 AM UTC· resolved Oct 03, 2025, 06:35 AM UTC

1. investigating Oct 03, 2025, 06:23 AM UTC

   CXone Mpower Expert Service Degradation: Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

2. investigating Oct 03, 2025, 06:26 AM UTC

   We are continuing to investigate this issue.

3. investigating Oct 03, 2025, 06:34 AM UTC

   We are continuing to investigate this issue.

4. resolved Oct 03, 2025, 06:35 AM UTC

   This incident has been resolved.

5. postmortem Oct 09, 2025, 04:45 PM UTC

   **Incident Summary** Updated on 10/08/2025 - On 10/03/2025, some NiCE CXone Mpower customers reported that articles, directories, and thumbnails were not working, and the search page was failing to load on the CXone Mpower Expert knowledge portal. The issue stemmed from a recent change release implemented to spin up new resources and add new configuration routing rules, which introduced a mismatch in traffic routing between the new and old release versions. The impact was resolved by migrating all sites to the latest release version, which immediately restored all affected services. **Root Cause** The issue stemmed from a recent change release implemented to spin up new resources and add new configuration routing rules, which introduced a mismatch in traffic routing between the new and old release versions. This was caused by a flaw in the deployment process used as a software release strategy on one of the load balancer components. During this process, both versions were configured to share the same load balancer component instead of operating on separate instances. This shared configuration created a mismatch between the release versions, causing the impact. After the new routing rule was added, traffic was directed correctly to the new release version, but it didn’t work on the old version because that version didn’t have the new rule. Although testing was performed in the lower environment during the Staging phase, the problem went undetected until the deployment reached the production environment. Following this, engineers identified the need to enhance the testing process to include both the old and new release versions, ensuring similar issues are prevented in the future. **Corrective Actions** **Detection:** Internal support teams received reports from customers indicating that articles, directories, and thumbnails were not working, and the search page was failing to load on the CXone Mpower Expert knowledge portal. **Remediation:** The impact was resolved by migrating all sites to the latest release version, which immediately restored all affected services. Completed on 10/03/2025. **Prevention:** Although no similar change releases are scheduled in the near future to mitigate the risk of recurrence, the Engineering team will enhance the deployment process by upgrading it to ensure that each version uses its own instance. This improvement will allow versions to operate independently and prevent similar issues in the future. Furthermore, testing procedures will be strengthened to include both old and new release versions during the Staging phase. An update will be provided by End of Day \(EOD\) MT on 10/31/2025. The Engineering team will review the existing alarm mechanisms to evaluate whether adjustments are needed and explore ways to make monitoring more granular. These improvements will enhance proactive detection, enabling faster issue resolution and preventing potential impact on customers. An update will be provided by EOD MT on 10/31/2025. ‌ **Incident Timeline \(UTC\)** 10/03/2025 05:50 AM \(UTC\) - The first customer case was opened, and Tech Support \(TS\) engineers began their initial validations and troubleshooting investigation. 10/03/2025 06:19 AM \(UTC\) - While engineers were already aware of the issue and had identified a suspected cause, they immediately began their remediation efforts. 10/03/2025 06:21 AM \(UTC\) - TS engineers notified the Network Operations Center \(NOC\) engineers about the reported customer impact; a major incident was proposed and confirmed. 10/03/2025 06:35 AM \(UTC\) - The impact was resolved after engineers migrated all sites to the new release version. Following successful validations, the major incident was marked as resolved.

Read the full incident report →

Started Jun 05, 2025, 08:40 PM UTC· resolved Jun 05, 2025, 10:24 PM UTC

1. investigating Jun 05, 2025, 08:40 PM UTC

   CXone Mpower Expert Service Degradation: Slow load times. The CXone Mpower Expert Engineering team is investigating reports of slow load times

2. investigating Jun 05, 2025, 08:55 PM UTC

   We are continuing to investigate this issue.

3. monitoring Jun 05, 2025, 08:59 PM UTC

   A fix has been implemented and we are monitoring the results.

4. monitoring Jun 05, 2025, 09:17 PM UTC

   We are continuing to monitor for any further issues.

5. monitoring Jun 05, 2025, 09:33 PM UTC

   We are continuing to monitor for any further issues.

6. monitoring Jun 05, 2025, 09:48 PM UTC

   We are continuing to monitor for any further issues.

7. monitoring Jun 05, 2025, 10:03 PM UTC

   We are continuing to monitor for any further issues.

8. monitoring Jun 05, 2025, 10:18 PM UTC

   We are continuing to monitor for any further issues.

9. resolved Jun 05, 2025, 10:24 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Apr 10, 2025, 01:59 PM UTC· resolved Apr 10, 2025, 03:03 PM UTC

1. investigating Apr 10, 2025, 01:59 PM UTC

   CXone Mpower Expert Service Degradation: Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability

2. identified Apr 10, 2025, 02:18 PM UTC

   CXone Mpower Expert Service Degradation: Sites unavailable. The issue has been identified and a fix is being worked on for deployment.

3. identified Apr 10, 2025, 02:36 PM UTC

   CXone Mpower Expert Service Degradation: Sites unavailable. The issue has been identified and a fix is being worked on for deployment.

4. monitoring Apr 10, 2025, 02:48 PM UTC

   CXone Mpower Expert - Fix Deployed. Event duration (32 mins). Status = initiating monitoring

5. resolved Apr 10, 2025, 03:03 PM UTC

   CXone Mpower Expert - Service Disruption Resolved - All Services Running Normally. The CXone Mpower Expert Engineering team has deployed a fix and monitored the deployment to make sure sites are stable. The issue is now resolved at this time. Event duration 32 minutes

6. postmortem Apr 16, 2025, 05:55 PM UTC

   **Impact Start Time \(UTC\)** - 04/10/2025 01:50 PM \(UTC\) **Impact End Time \(UTC\)** - 04/10/2025 02:22 PM \(UTC\) **Incident Summary** Updated on 04/16/2025 - On 4/10/2025, some CXone Mpower customers reported inability to access CXone Mpower Expert application sites, encountering '504 Gateway Timeout" error messages and slow response times when attempting to access the affected sites. The issue was caused by the system being overwhelmed due to numerous middleware service pods making requests to the Application Programming Interface \(API\) pod, which could not keep up due to each pod having limited database connections. The impact was resolved by increasing the resources capacity of the API pods, which recovered the system to be able to process the requests that had queued up as well as the normal traffic. **Root Cause** The issue was caused by the system being overwhelmed due to numerous middleware service pods making requests to the API pod, which could not keep up due to each pod having limited database connections. To address previous slow response times, internal teams increased the number of middleware service pods for better throughput. However, each API pod had limited connections, which did not scale with the increased traffic and caused a bottleneck on the system. Since the API pods were unable to scale automatically, the system entered an unhealthy state. The high number of middleware service pods making requests to the API exceeded the connection limit per pod, leading to delays. **Corrective Actions** **Detection:** Internal support teams detected a potentially customer-impacting issue through proactive alarms and monitoring mechanisms, which was later confirmed by customer reports indicating their inability to access CXone Mpower Expert application sites, encountering '504 Gateway Timeout" error messages and slow response times when attempting to access the affected sites. **Remediation:** The impact was resolved by increasing the capacity resources of the API pods, which recovered the system to be able to process the requests that had queued up as well as the normal traffic. Completed on 04/10/2025. **Prevention:** The Engineering team is actively working to have the API pods scale off of other metrics to help with the API scaling issues. **Incident Timeline \(UTC\)** 04/10/2025 01:50 PM \(UTC\) - The Engineering team detected a potential customer issue. 04/10/2025 02:06 PM \(UTC\) - The first customer case was opened, and Tech Support \(TS\) engineers began the troubleshooting investigation. 04/10/2025 02:08 PM \(UTC\) - Tech Support \(TS\) engineers notified the Network Operations Center \(NOC\) engineers about the reported customer impact; a major incident was proposed and confirmed. 04/10/2025 02:18 PM \(UTC\) - Engineers identified a suspected cause and began the remediation steps. 04/10/2025 02:22 PM \(UTC\) - Impact was resolved after engineers identified the issue and implemented a fix and internal tests were successful. Impact and major incident resolved.

Read the full incident report →

Started Apr 07, 2025, 02:23 PM UTC· resolved Apr 07, 2025, 02:57 PM UTC

1. monitoring Apr 07, 2025, 02:23 PM UTC

   CXone Mpower Expert - Service Degradation - Investigating issues related to slow load times on sites.

2. resolved Apr 07, 2025, 02:57 PM UTC

   This incident has been resolved.

Read the full incident report →