Healthise experienced a major incident on January 23, 2024, lasting —. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Jan 23, 2024, 11:51 PM UTC
At 7:44 AM MST, on Wednesday, January 17, 2024, Healthwise administrators were alerted to an issue with Healthwise Coach and began an investigation. They found that the server resources were overwhelmed and causing intermittent degraded performance. Healthwise administrators rolled back recent changes which improved resource availability and restored application performance. Production monitors reported stable performance at 8:33 AM MST. The total time of the incident was 49 minutes.
- postmortem Jan 23, 2024, 11:52 PM UTC
## Introduction The purpose of this Root Cause Analysis \(RCA\) is to determine the causes that contributed to the intermittent performance degradation for the Healthwise® Coach and EMR Module applications on January 17, 2024. ## Event Description At 7:44 AM MST, on Wednesday, January 17, 2024, Healthwise administrators were alerted to an issue with Healthwise Coach and began an investigation. They found that the server resources were overwhelmed and causing intermittent degraded performance. Healthwise administrators rolled back recent changes which improved resource availability and restored application performance. Production monitors reported stable performance at 8:33 AM MST. The total time of the incident was 49 minutes. ## Findings and Root Cause Based on the investigation conducted, the team determined the following finding regarding this event: A recent Healthwise Coach application update to upgrade a 3rd party component stressed server resources that are shared with the EMR Module application. When the server resources reached their limit some requests couldn’t access the applications. ## Corrective Action Performance was restored after Healthwise Coach was rolled back to a known stable version. We have escalated the issue to the 3rd party vendor to assist with troubleshooting. We will attempt another updated to the 3rd party component after implementing changes to address the issue.