Function run scheduling delays

Update timeline

1. investigating Mar 26, 2026, 10:42 PM UTC

   Status: Investigating We are actively investigating delays with function run scheduling. We will provider further updates as we identify the cause and resolve the issue. Affected components Function execution (Degraded performance)

2. identified Mar 26, 2026, 10:58 PM UTC

   Status: Identified We have identified the cause of the issue affecting a core system queue. We are rolling our a mitigation now and preparing follow up changes. Affected components Function execution (Degraded performance)

3. identified Mar 26, 2026, 11:15 PM UTC

   Status: Identified We have deployed an additional hot fix. The earlier change rolled out have addressed the core issue and the system is now processing the backlog. The backlog is decreasing. We will provide another update as we have an estimate on time to recovery. Affected components Function execution (Degraded performance)

4. monitoring Mar 26, 2026, 11:35 PM UTC

   Status: Monitoring The mitigations have fixed the issue and the event backlog is now caught up. We continue to monitor and evaluate other short and long term mitigations to add. Affected components Function execution (Operational)

5. resolved Mar 26, 2026, 11:45 PM UTC

   Status: Resolved The incident is now resolved and the system is full operational. This was related to an issue caused by the part of the system powering the debounce feature. The internal event backlog is fully caught up and the two mitigations deployed have addressed the issue. The team is preparing a post-mortem to ensure this issue does not reoccur. Affected components Function execution (Operational)