MindTouch incident

CXone Mpower Expert - – Monitoring complete. Status = All Services Running Normally

Major Resolved View vendor source →

MindTouch experienced a major incident on August 13, 2025 affecting Application (General Service) and Search and 1 more component, lasting 29m. The incident has been resolved; the full update timeline is below.

Started
Aug 13, 2025, 02:43 PM UTC
Resolved
Aug 13, 2025, 03:12 PM UTC
Duration
29m
Detected by Pingoru
Aug 13, 2025, 02:43 PM UTC

Affected components

Application (General Service)SearchGenerative SearchIn-Product Contextual HelpEmail ServicesMindTouch Success CenterAnalyticsGeoblocking for Russia

Update timeline

  1. investigating Aug 13, 2025, 02:43 PM UTC

    CXone Mpower Expert Service Degradation: Sites unavailable. The CXone Mpower Expert Engineering team is investigating reports of site unavailability.

  2. monitoring Aug 13, 2025, 02:55 PM UTC

    CXone Mpower Expert - Fix Deployed - All Services Running Normally. The CXone Mpower Expert Engineering team has deployed a fix and all services are running normally. We are currently monitoring sites for deployment stability. Event duration (19 mins)

  3. resolved Aug 13, 2025, 03:12 PM UTC

    CXone Mpower Expert - Service Disruption Resolved - All Services Running Normally. The CXone Mpower Expert Engineering team has deployed a fix and monitored the deployment to make sure sites are stable. The issue is now resolved at this time. Event duration (19 mins)

  4. postmortem Aug 20, 2025, 10:40 PM UTC

    Impact Start Time \(UTC\) 08/13/2025 02:32 PM UTC Impact End Time \(UTC\) 08/13/2025 02:40 PM UTC **Incident Summary** Updated on 08/20/2025 - On 08/13/2025, some CXone Mpower customers reported performance issues, including slow response times and encountering errors when accessing the CXone Mpower Expert application sites. The issue stemmed from a common source that generated an unusually high volume of web requests. The issue was resolved when the system recovered on its own by scaling up the number of servers to allow more connections. **Root Cause** The issue stemmed from a common source that generated an unusually high volume of web requests. The excessive load strained the shared database infrastructure connections, resulting in degraded performance and slow response times across all associated sites. ### Corrective Actions **Detection:** Internal support teams detected a potentially customer-impacting issue, which was later confirmed by customer reports indicating performance issues, including slow response times and encountering errors when accessing the CXone Mpower Expert application sites. **Remediation:** The issue was resolved when the system recovered on its own by scaling up the number of servers to allow more connections. Completed on 08/13/2025. **Prevention:** The Engineering team will enhance the Web Application Firewall \(WAF\) by implementing targeted rules designed to detect and block traffic from sources exhibiting similar behavior. An update will be provided by End of Day \(EOD\) MT on 10/01/2025. The Engineering team is working to improve the scaling mechanism, preventing similar issues in the future. An update will be provided by EOD MT on 10/01/2025. **Incident Timeline \(UTC\)** 08/13/2025 02:32 PM \(UTC\) - Engineers detected a potential customer impacting issue received through an alarm and promptly initiated a troubleshooting investigation; a major incident was proposed and confirmed. 08/13/2025 02:39 PM \(UTC\) - The first customer case was opened, and Tech Incident Timeline \(UTC\) Support Engineers \(TSEs\) began the troubleshooting investigation. 08/13/2025 02:40 PM \(UTC\) - Engineers observed restoration of the system following the activation of the auto-scaling mechanism, restoring system stability by automatically adjusting the resources. After successful test validations, the major incident was marked as resolved.