doxy.me experienced a major incident on June 29, 2021 affecting Doxy.me webpages, lasting 4h 1m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jun 29, 2021, 12:08 PM UTC
We are investigating an issue with intermittent login issues. We will provide an update in the next 30 minutes.
- identified Jun 29, 2021, 12:38 PM UTC
The root cause of the login and access issues this morning has been identified. We are implementing a fix with a vendor who's services went down this morning and are continuing to monitor this problem.
- monitoring Jun 29, 2021, 12:56 PM UTC
A fix has been implemented and we are continuing to monitor the results.
- resolved Jun 29, 2021, 04:09 PM UTC
This incident has been resolved. We will provide a port mortem in the next few days.
- postmortem Jul 07, 2021, 09:19 PM UTC
* **Summary of event:** Doxy.me experienced connectivity issues due to a 3rd party’s servers going down. * **Business Impacts:** Providers were unable to log in to doxy.me. Patients were unable to check in to their provider's queue. * **How was the issue resolved:** We deployed a software update which prevented our system from making the call to the 3rd party provider. * **Root cause:** Our 3rd party provider was refusing connections at that time. The part of our system that was trying to connect to this 3rd party and get an answer was not able to recover from the refused connection. * **Preventative next steps:** We are continuing to test and implement updates to our code that will better handle future scenarios when our vendors have server problems.