Aula Education incident

Delayed email notifications in Coventry environment

Minor Resolved View vendor source →

Aula Education experienced a minor incident on June 23, 2022 affecting Notifications, lasting 23h 27m. The incident has been resolved; the full update timeline is below.

Started
Jun 23, 2022, 04:15 PM UTC
Resolved
Jun 24, 2022, 03:43 PM UTC
Duration
23h 27m
Detected by Pingoru
Jun 23, 2022, 04:15 PM UTC

Affected components

Notifications

Update timeline

  1. identified Jun 23, 2022, 04:15 PM UTC

    We are currently observing a higher amount of errors in our email delivery pipeline in Coventry. This is likely to be a repeat of the issue we observed earlier in the week that was caused by throttling. We have raised the issue with Coventry IT teams to investigate.

  2. monitoring Jun 23, 2022, 08:12 PM UTC

    The throttling appears to have been removed and all emails are now sending. We will continue to monitor the situation.

  3. monitoring Jun 24, 2022, 08:43 AM UTC

    We have not observed any further throttling overnight or this morning. Continuing to monitor.

  4. monitoring Jun 24, 2022, 10:24 AM UTC

    The SMTP server no longer appears to be responding. We have raised this with the Coventry IT Team.

  5. monitoring Jun 24, 2022, 12:52 PM UTC

    The SMTP server has been restarted and is now sending email again.

  6. monitoring Jun 24, 2022, 02:53 PM UTC

    SMTP server appears to be stable for now. We have developed and tested some mitigations that will be deployed early next week.

  7. resolved Jun 24, 2022, 03:43 PM UTC

    This incident has been resolved.