Progress Telerik incident

NuGet Service Disruption (deprecated v2 server only)

Major Resolved View vendor source →

Progress Telerik experienced a major incident on November 11, 2021, lasting 2h 28m. The incident has been resolved; the full update timeline is below.

Started
Nov 11, 2021, 04:08 PM UTC
Resolved
Nov 11, 2021, 06:37 PM UTC
Duration
2h 28m
Detected by Pingoru
Nov 11, 2021, 04:08 PM UTC

Update timeline

  1. investigating Nov 11, 2021, 04:08 PM UTC

    We are currently investigating service disruption reports for the deprecated v2 NuGet server (https://nuget.telerik.com/nuget). Note: There are no currently known issues with the new v3 NuGet server (https://nuget.telerik.com/v3/index.json). You can do an in-place swap of the server URLs to regain access to the feed.

  2. investigating Nov 11, 2021, 05:01 PM UTC

    We are continuing to investigate this issue.

  3. investigating Nov 11, 2021, 05:06 PM UTC

    The cause of disruption was identified and repaired. The servers are currently fulfilling queued requests in the backlog. We will update this status again when operations return to 100%. Note: some new request responses may be slow, which might cause a timeout depending on your client. In such a case, rerun the restore command again after waiting several minutes.

  4. resolved Nov 11, 2021, 06:37 PM UTC

    All Telerik NuGet servers are back at full operation.

  5. postmortem Nov 11, 2021, 06:37 PM UTC

    One of the nodes that handle NuGet server traffic was experiencing degraded performance. This caused a backlog of queued requests, which resulted in occasional 503 responses to any dotnet or nuget.exe client requesting use of the search or download API endpoints. The node was investigated by the server team, repaired and operations returned to 100% capacity after the queue was flushed.