Welkin Health incident

Email sending subsystem degradation, due to DDoS attack on Mailgun service

Minor Resolved View vendor source →

Welkin Health experienced a minor incident on February 12, 2024 affecting Welkin API, lasting 8h 12m. The incident has been resolved; the full update timeline is below.

Started
Feb 12, 2024, 02:53 PM UTC
Resolved
Feb 12, 2024, 11:06 PM UTC
Duration
8h 12m
Detected by Pingoru
Feb 12, 2024, 02:53 PM UTC

Affected components

Welkin API

Update timeline

  1. monitoring Feb 12, 2024, 02:53 PM UTC

    A fix has been implemented and we are monitoring the results.

  2. monitoring Feb 12, 2024, 05:01 PM UTC

    Await confirmation of results from Mailgun, after fix

  3. monitoring Feb 12, 2024, 08:45 PM UTC

    Mailgun cloud provider is working through internal escalation process to help them with the mitigation, meanwhile they are working on scaling up the infrastructure to support the load.

  4. resolved Feb 12, 2024, 11:06 PM UTC

    No degradation in email sending and delivery was observed during the monitoring period.