Healthise experienced a minor incident on November 29, 2023 affecting Coach, lasting 8m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Nov 29, 2023, 03:39 PM UTC
Healthwise-hosted solutions are experiencing performance issues. Our Network Administrators and Engineers are working to fix the problem. We will post updates as we learn more.
- resolved Nov 29, 2023, 03:48 PM UTC
This incident is still under investigation and the RCA will be updated when the full investigation is complete. If you are subscribed to the status page unfortunately you will not receive a notice when the RCA is updated to the final version. However, at any time you may come back to this page to see the latest updates.
- postmortem Dec 13, 2023, 10:42 PM UTC
## Introduction The purpose of this Root Cause Analysis \(RCA\) is to determine the causes that contributed to performance degradation for Healthwise® Coach on November 29, 2023. ## Event Description At 8:22 AM MST, on Wednesday, November 29, 2023, Healthwise administrators were alerted to an issue with Healthwise Coach and began an investigation. They found that the server resources were overwhelmed and causing degraded performance. Healthwise administrators rolled back recent changes which restored resource availability and improved application performance. Production monitors reported stable performance at 9:22 AM MST. The total time of the incident was 60 minutes. ## Findings and Root Cause Based on the investigation conducted, the team determined the following finding regarding this event: Recent Healthwise Coach enhancements to improve session performance stressed server resources. When the server resources reached their limit some requests to the application became unresponsive and couldn’t access the application. ## Corrective Action Performance was restored after Healthwise Coach was rolled back to a known stable version. We are implementing ways to make it easier to recreate production issues. Additionally, the enhancements involved in this incident will not be deployed until they are further refined.