Copado incident

Azure DevOps Global Degradation

Notice Resolved View vendor source →

Copado experienced a notice incident on April 24, 2024 affecting CICD and CICD and 1 more component, lasting 13h 39m. The incident has been resolved; the full update timeline is below.

Started
Apr 24, 2024, 01:13 PM UTC
Resolved
Apr 25, 2024, 02:53 AM UTC
Duration
13h 39m
Detected by Pingoru
Apr 24, 2024, 01:13 PM UTC

Affected components

CICDCICDCICDEssentialsEssentialsEssentials

Update timeline

  1. identified Apr 24, 2024, 01:13 PM UTC

    Copado has identified that Azure DevOps Saas began experiencing service degradation and availability issues starting at approximately 2:30 PM Central Time on April 23, 2024, that could result in failed authentication to Azure DevOps repos via HTTPS. We are currently working with our engineering and infrastructure teams to help identify impacted customers, and are monitoring the ADO status via their site, https://status.dev.azure.com. We have confirmed that this problem does not impact customers utilizing SSH to connect to their repositories. We currently recommend that if possible, customers update their Git Repository record configuration from HTTPS to SSH, to mitigate this outage and return to the full use of Copado and their pipelines with ADO Saas repositories. If you need assistance with this change, please reach out to Copado Support via https://success.copado.com. We will provide updates as more information becomes available.

  2. identified Apr 24, 2024, 02:38 PM UTC

    We have updated this incident to indicate broader impact; we are seeing examples of customers using Azure DevOps that are experiencing issues beyond authentication. ADO has also broadened the impact of their ongoing incident: https://status.dev.azure.com/_event/496920595 We recommend contacting ADO directly if you are utilizing an ADO repository and are experiencing issues, including but not limited to authentication, performance, and availability.

  3. identified Apr 24, 2024, 06:11 PM UTC

    We are actively engaged with ADO support and they have provided an estimate of 4 hours to complete the work required to resolve this issue.

  4. resolved Apr 25, 2024, 02:53 AM UTC

    This incident has been resolved.