Octopus incident

Package Acquisition step sometimes fails with exception

Major Resolved View vendor source →

Octopus experienced a major incident on December 9, 2024 affecting Octopus Cloud, lasting 1d 2h. The incident has been resolved; the full update timeline is below.

Started
Dec 09, 2024, 09:32 PM UTC
Resolved
Dec 11, 2024, 12:08 AM UTC
Duration
1d 2h
Detected by Pingoru
Dec 09, 2024, 09:32 PM UTC

Affected components

Octopus Cloud

Update timeline

  1. investigating Dec 09, 2024, 09:32 PM UTC

    Some Octopus Cloud customers are having intermittent issues with package acquisition steps throwing exceptions. We are currently investigating.

  2. identified Dec 09, 2024, 11:56 PM UTC

    We are continuing to actively work on this issue. We have identified the issue and are testing a potential mitigation/fix. We are also continuing to investigate the underlying causes of the issue to build confidence our mitigation will work as expected.

  3. identified Dec 10, 2024, 01:39 AM UTC

    We are continuing to actively work on resolving this issue. Our first potential mitigation didn't work as expected. We have a second potential mitigation that is performing better in testing. We will update here if/when we ship it.

  4. identified Dec 10, 2024, 02:13 AM UTC

    We've identified the underlying code change that caused this issue to start appearing. We have a fix for it that will be rolling out to Cloud customers as quickly as possible. Self-hosted customers will not be affected.

  5. monitoring Dec 10, 2024, 07:27 AM UTC

    The fix is now rolling out to Cloud. If this issue is affecting you, please reach out to the support team to ask them to upgrade your instance to the latest version.

  6. resolved Dec 11, 2024, 12:08 AM UTC

    The fix has been released to Cloud - if you are still encountering issues, please ask the support team to upgrade your instance to 2025.1.3365