MageMojo incident

Amazon SES (Simple Email Service) sending impaired

Minor Resolved View vendor source →

MageMojo experienced a minor incident on September 17, 2020 affecting Webscale STRATUS - Northern Virginia and Webscale STRATUS - Frankfurt and 1 more component, lasting 1h 36m. The incident has been resolved; the full update timeline is below.

Started
Sep 17, 2020, 03:06 PM UTC
Resolved
Sep 17, 2020, 04:42 PM UTC
Duration
1h 36m
Detected by Pingoru
Sep 17, 2020, 03:06 PM UTC

Affected components

Webscale STRATUS - Northern VirginiaWebscale STRATUS - FrankfurtWebscale STRATUS - Sydney

Update timeline

  1. investigating Sep 17, 2020, 03:06 PM UTC

    We are investigating this issue.

  2. resolved Sep 17, 2020, 04:42 PM UTC

    This incident is resolved. Email is being delivered again.

  3. postmortem Sep 17, 2020, 09:07 PM UTC

    There was an issue with Amazon SES this morning from 8AM ET to 12:45PM ET. SES was rejecting all emails. The SES service was suspended due to what Amazon thought was a high bounce rate during a manual review. There was no high bounce rate. The SES service was unsuspended and email delivery was flowing again. Unfortunately, SES rejected the email attempts and did not queue, so emails needs to be manually resent. We’re talking with Amazon on ways to prevent this mistake in the future and making email queuing more resilient in the future. Please note that you are not required to use our Amazon SES email delivery service. You may bring your own third party mail delivery provider like Send Grid, Mandrill, Mailgun, etc..