Ribbon Health experienced a minor incident on May 12, 2023 affecting Provider Search and Locations Search and 1 more component, lasting 3h 58m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating May 12, 2023, 02:15 PM UTC
We are currently investigating this issue.
- investigating May 12, 2023, 02:16 PM UTC
We are continuing to investigate this issue.
- resolved May 12, 2023, 06:13 PM UTC
This incident has been resolved.
- postmortem May 18, 2023, 08:15 PM UTC
The issue was caused by our API Redis instance running out of memory. As an immediate term fix we increased the size of the Redis instances in order to reduce latency. We have been monitoring the API the past few days and are seeing the service maintain the baseline performance we expect. We are also profiling our API platform to better understand where the spikes in memory are coming from and addressing those at core to ensure that this sort of incident does not happen again. Please don’t hesitate to reach out if you have any further questions or concerns in the meantime. We sincerely apologize for this incident and the impact it had on your team and your end-users. Ribbon Team