Service Issue - Email Notification Issues from Cloud Manager

Update timeline

1. investigating Mar 09, 2026, 04:04 AM UTC

   Our team is investigating an emerging issue where some customers are receiving delayed event notification or notification emails for activities performed on Cloud Manager. We will share additional updates as we have more information.

2. investigating Mar 09, 2026, 05:48 AM UTC

   We are continuing to investigate this issue. The appropriate subject matter experts are engaged. Subsequent updates will be posted as progress is made.

3. investigating Mar 09, 2026, 06:31 AM UTC

   We are continuing to investigate this issue. As a workaround, customers can login to https://cloud.linode.com/ and manually check the notifications until the issue is mitigated. Subsequent updates will be posted as progress is made.

4. resolved Mar 09, 2026, 03:18 PM UTC

   Our team has identified that the issue affecting the Cloud Manager and API is related to a previously communicated incident from March 4, 2026 (https://status.linode.com/incidents/yzlp8ykymmhm). Today's issue was caused by the backlog of notifications that accumulated due to the previous incident, and it has cleared now. We haven't observed any additional issues with the Cloud Manager or API, and will now consider this incident resolved. If you continue to experience issues, please contact us at 855-454-6633 (+1-609-380-7100 Intl.), or send an email to [email protected] for assistance.

5. postmortem Mar 13, 2026, 09:10 PM UTC

   On March 3, 2026, at approximately 20:00 UTC,, a cron job responsible for email notifications from Cloud Manager began failing due to timeouts. As the job exceeded the request timeout interval, it repeatedly failed and restarted, causing the process to resume from the beginning of its queue each time. This resulted in customers receiving multiple duplicate email notifications and some delay in those notifications. This issue only affected event-based notification emails related to host jobs such as VM shutdowns, startups, deletions, or deployments. Other notification emails set up by the customer, such as CPU usage thresholds, and the host jobs themselves remained unaffected. To address the issue, the cron job was temporarily stopped, associated services were restarted, and a processing limit was introduced to control the number of records handled per run. These actions stabilized the system, and the incident was considered mitigated at approximately 19:00 UTC on March 4, 2026. After mitigation of the root cause, a backlog of approximately 2 million pending events had to be processed, which caused further delays in notification delivery over the following days for some customers. The system worked through this backlog and gradually returned to normal processing. As of March 10, 2026, the backlog has been cleared and notification emails are being delivered as expected. Since this incident, we have already tested and implemented additional cron job optimizations to ensure consistent performance. We are committed to making continuous improvements to make our systems better and prevent recurrence. This summary provides an overview of our current understanding of the incident given the information available. Our investigation is ongoing and any information herein is subject to change.