RCA - Azure Data Factory V2 - West Europe

Update timeline

1. postmortem Dec 13, 2021, 12:30 PM UTC

   Summary of Impact: Between approximately 06:30 UTC and 12:30 UTC on 13 December 2021, you were identified as a customer that may have experienced intermittent errors accessing Azure Data Factory resources (ADF) in West Europe. Root Cause: We determined a backend service, responsible for processing API requests became unhealthy. Retry logic from ADF, coupled with this unhealthy service, resulted in a rare combination of transient conditions and lead to additional errors. This resulted in intermittent API failing calls for Azure Data Factory resources. Mitigation: We restarted the backend service which mitigated the issue. Next Steps: We apologize for the impact to affected customers. We are continuously taking steps to improve the Microsoft Azure Platform and our processes to help ensure such incidents do not occur in the future. In this case, this includes (but is not limited to): • Implement an early alert system to notify failures at this level • Create a repair item and work with the backend service team to create a solution to help prevent future occurrences Provide Feedback: Please help us improve the Azure customer communications experience by taking our survey: https://aka.ms/AzurePIRSurvey