Octopus incident

Dynamic Worker Leases failing in WestEU

Major Resolved View vendor source →

Octopus experienced a major incident on February 26, 2024 affecting Octopus Cloud, lasting 3d 6h. The incident has been resolved; the full update timeline is below.

Started
Feb 26, 2024, 07:07 PM UTC
Resolved
Mar 01, 2024, 01:53 AM UTC
Duration
3d 6h
Detected by Pingoru
Feb 26, 2024, 07:07 PM UTC

Affected components

Octopus Cloud

Update timeline

  1. investigating Feb 26, 2024, 07:07 PM UTC

    Our upstream provider, Microsoft Azure, is currently experiencing an issue which is affecting our ability to provision worker virtual machines for a subset of customers in WestEU. We are working with Azure to resolve the issue. If you experience this problem, deployments utilising dynamic workers will fail with an error specifying that Octopus Deploy could not obtain a dynamic worker lease. We will provide an update in an hour, at approximately 20:00 UTC. We will resolve this incident once we have received confirmation the issue has been resolved from our upstream provider.

  2. investigating Feb 26, 2024, 10:09 PM UTC

    We continue to work with Azure to investigate the issue. We are also investigating a potential workaround. Another update will be provided within the next hour.

  3. investigating Feb 26, 2024, 10:36 PM UTC

    Dynamic Workers are now provisioning successfully. We are continuing to monitor for any degradation of service.

  4. monitoring Feb 26, 2024, 10:37 PM UTC

    Dynamic Workers are now provisioning successfully. We are continuing to monitor for any degradation of service.

  5. monitoring Feb 27, 2024, 01:16 AM UTC

    Dynamic Workers are now provisioning successfully. We are continuing to monitor for any degradation of service.

  6. resolved Mar 01, 2024, 01:53 AM UTC

    This incident has been resolved.