mailprotector experienced a minor incident on March 20, 2023 affecting CloudFilter and SafeSend Email Security, lasting 21h 34m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Mar 20, 2023, 06:22 PM UTC
The team is investigating delays with outbound email delivery. A surge of activity has resurfaced that is being actively managed to push the queues forward.
- investigating Mar 20, 2023, 07:53 PM UTC
The team continues to manage the queues and seek the issue's root cause. The behavior is outside the bounds of regular activity, and the team is investigating a potential denial of service attack pattern.
- investigating Mar 21, 2023, 12:29 AM UTC
We are continuing to work on the outbound delivery issue. There is no evidence of a denial of service attack, and we are working on other possible root causes.
- monitoring Mar 21, 2023, 01:35 AM UTC
The outbound surge queue is normal; however, we will continue to monitor the services closely.
- monitoring Mar 21, 2023, 01:57 PM UTC
Log data and system observation show outbound mail flow is operating as expected. We are continuing to monitor things closely and will send another update to confirm the resolution later this morning.
- resolved Mar 21, 2023, 03:57 PM UTC
Log data comparisons indicate the certificate error was the root cause of yesterday's performance. The team is closely monitoring the outbound mail flow. Further investigation into the certificate error and the addition of new monitors for this type of situation are ongoing. As of this update, outbound mail flow is operating as expected, and Mailprotector expects it to remain healthy going forward.