Blue Canvas incident

New deployments are stuck in "Packaging..."

Major Resolved View vendor source →

Blue Canvas experienced a major incident on April 20, 2021 affecting Metadata Deploy (Outbound), lasting 18m. The incident has been resolved; the full update timeline is below.

Started
Apr 20, 2021, 08:18 PM UTC
Resolved
Apr 20, 2021, 08:36 PM UTC
Duration
18m
Detected by Pingoru
Apr 20, 2021, 08:18 PM UTC

Affected components

Metadata Deploy (Outbound)

Update timeline

  1. investigating Apr 20, 2021, 06:12 PM UTC

    We are investigating an incident affecting new deployments and will update this incident with more details as they are available.

  2. investigating Apr 20, 2021, 06:41 PM UTC

    Deployment requests are currently unavailable while we continue to investigate the issue. New deployments requests can be created but will remain in a queue until the incident is resolved.

  3. identified Apr 20, 2021, 06:50 PM UTC

    We have identified the issue and are working on a fix.

  4. identified Apr 20, 2021, 07:17 PM UTC

    Blue Canvas is impacted by a major outage at our upstream provider Auth0 (https://status.auth0.com/). We are working to mitigate the issue and restore full service as soon as possible. At the moment, outbound deployments are unavailable while the remaining application works as expected.

  5. identified Apr 20, 2021, 07:41 PM UTC

    We expect full service to be restored in the next 90 minutes. Engineers are implementing a mitigation for a major outage at our upstream provider Auth0.

  6. monitoring Apr 20, 2021, 08:14 PM UTC

    Full service has been restored. Deployment Requests that were still in "Packaging..." status are now being processed. Deployment Requests in "Internal Error" status have to be restarted manually, please click the "Retry" button on the deployment page.

  7. monitoring Apr 20, 2021, 08:18 PM UTC

    Full service has been restored and we are continuing to monitor for stability.

  8. resolved Apr 20, 2021, 08:36 PM UTC

    We have resolved an issue where new deployment requests were not being processed. Blue Canvas was impacted by a major outage at our authentication provider.