Digital Pharmacist incident
Intermittent issues with IVR Dead Air, Busy Signals and Keypad Entry
Digital Pharmacist experienced a notice incident on September 27, 2021 affecting NPE IVR and Digital Pharmacist IVR Services (Legacy Cloud), lasting 110d 8h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Sep 27, 2021, 04:19 PM UTC
Infrastructure is functioning normally. Intermittent issues are being experienced with a select few numbers and widespread downtime is not anticipated for our customers.
- investigating Sep 27, 2021, 07:54 PM UTC
Additional information has come to light regarding the specific nature of the issue: One of the largest telecom providers in North America is experiencing intermittent service interruptions that are widespread and far-reaching. The core of the issue is not native to Digital Pharmacist or our provider's infrastructure or services. However as the interruptions are impacting higher level service providers, the telecom industry is experiencing the overall impact.
- monitoring Sep 28, 2021, 04:52 PM UTC
A fix has been implemented and we are monitoring the results.
- investigating Sep 29, 2021, 02:52 PM UTC
We are currently investigating this issue.
- investigating Sep 30, 2021, 03:04 PM UTC
Ongoing intermittent issues with a select few phone numbers are being investigated.
- monitoring Sep 30, 2021, 09:52 PM UTC
Our carrier has corrected the issue from earlier today and all phone numbers are operational.
- monitoring Nov 09, 2021, 11:09 PM UTC
At approximately 1:30pm EST this afternoon, our voice service providers began experiencing issues with an underlying carrier which resulted in a portion of inbound calls returning a busy signal/tone. The recovery efforts are still in progress and they are monitoring the situation.
- identified Nov 12, 2021, 09:32 PM UTC
Our voice provider is reporting and troubleshooting failed calls and degraded audio issues for 20% of all inbound calls. We are monitoring carefully.
- monitoring Nov 18, 2021, 02:52 PM UTC
We've received alerts of call quality and connection issues from yesterday afternoon. Certain calls were not connecting. Other calls exhibited degraded audio quality. These symptoms are in line with previous DDoS attacks. We are continuing to monitor the situation and work with our carrier on these issues.
- resolved Jan 16, 2022, 01:08 AM UTC
This incident has been resolved.