Keysight Eggplant incident
Intermittent Delays on Recovery Email notifications
Keysight Eggplant experienced a notice incident on September 5, 2023 affecting Email Alerting, lasting 2d 20h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Sep 05, 2023, 12:05 PM UTC
On very occasional instances, there may be a slight delay in receiving test recovery emails or, in even rarer cases, they might not get sent. If you're expecting a recovery email and it hasn't arrived, we recommend logging into the portal to verify if the monitor has recovered. We are diligently working to investigate the root of this issue. Rest assured, we'll provide timely updates on our findings and progress. Recoveries sent via SMS, Voice and Webhooks are not affected.
- resolved Sep 08, 2023, 08:36 AM UTC
We have upgraded the Message service to use a highly efficient process manager and tested the endpoints to ensure the service is working expected. We have deployed the updated service and recovery emails are dispatched as expected. We are also expanding our internal monitoring capabilities for more detailed monitoring of the service.