Date Time Trigger Automation Failues

Update timeline

1. resolved Apr 29, 2026, 11:19 PM UTC

   Between April 21st and April 23rd, many rules triggered by date time triggers did not run. New rule triggers ran normally starting April 23rd, and the delayed triggers were processed by April 27th.

2. postmortem Apr 30, 2026, 12:43 AM UTC

   Starting on **April 20th, 2026 at 10:00 AM PDT**, a combination of scaled-up background action jobs in the EU and a recent change to how date time trigger volume is managed overloaded the system that processes resource intensive background action jobs. This caused multi-hour delays in email and inbox notifications in the EU and prevented date time triggers from executing in all regions. Engineers were first alerted on **April 21st** by our monitoring systems, but did not recognize the full scope of the incident until **April 23rd**. On April 23rd, the change to manage date time trigger volume was reverted and the problematic background action jobs were isolated. All delayed notifications were sent by **April 24th, 11:30 AM PDT**, and all delayed date time triggers were executed by **April 27th, 4:00 AM PDT**.