Hint Health experienced a major incident on May 22, 2024 affecting HintOS App and HintOS Outbound Email Delivery (Mailgun), lasting 9h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified May 22, 2024, 04:54 AM UTC
At 5:47pm PST our email vendor Mailgun has disabled our account without advance notice due an unspecified security concern. We were not able to confirm any security issue, unauthorized access or unauthorized sending of mail. At 6:41pm PST they requested a full reset of all API keys, webhook keys, SMTP settings and user credentials. This work was completed at 8:42pm PST. We are now waiting on Mailgun to re-activate our account. They have provided us with no timeline. While Mailgun has been a reliable email vendor in the past, email is a critical part of our application and cannot be dependent on a single vendor. Our team has immediately prioritized work to setup a secondary email provider, and we will provide an estimate on when this will be available as soon as we have one. We're very sorry for the inconvenience that this is causing all our customers. - Graham Melcher
- resolved May 22, 2024, 01:55 PM UTC
At 2:38 AM PST (on May 22) our Mailgun account was enabled again. New emails after that point where delivered right away and emails that failed to send while the account was disabled where retried successfully by our team. We continue to work on adding a secondary email provider as backup.