PlayFab experienced a major incident on July 17, 2025, lasting —. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Jul 17, 2025, 01:12 AM UTC
No scheduled tasks were started on 7/15/2025 00:00 UTC likely due to a product deployment. Our engineers are still investigating the root causes of the incident in order to strengthen our processes and increase the resilience of our services to prevent reoccurrence.
- postmortem Aug 05, 2025, 09:04 PM UTC
Some customers experienced a failure of scheduled tasks configured to run on July 15, 2025 at 00:00 UTC. The incident was caused by a deployment overlapping with the scheduled task run time, leading to an interruption in the task processing service. The issue was resolved when tasks executed successfully during the following scheduled run \(July 15, 2025 at 00:05 UTC or later\). ### Impact Customers’ scheduled tasks did not execute at the expected time on July 15, 2025. This resulted in inconvenience and potential disruption for customers and their end users who relied on timely execution of these tasks. All affected tasks ran as expected during the next scheduled occurrence. ### Root Cause Analysis The root cause was a deployment that coincided with the scheduled task run, which caused the service to shut down and cancel the ongoing task processing. Additionally, code defects prevented the new service instance from queuing missed tasks when it started, and existing monitoring did not detect the issue because only a single execution was missed. ### Action Items To prevent similar incidents from happening again, we have implemented a mechanism to queue missed tasks when a new service instance starts, ensuring tasks are not skipped during deployments or restarts.