Qubole incident

Tunnel Server issue on wellness.qubole.com

Critical Resolved View vendor source →

Qubole experienced a critical incident on March 27, 2021 affecting Site Availability and Command Processing and 1 more component, lasting 4d 12h. The incident has been resolved; the full update timeline is below.

Started
Mar 27, 2021, 11:08 PM UTC
Resolved
Apr 01, 2021, 11:15 AM UTC
Duration
4d 12h
Detected by Pingoru
Mar 27, 2021, 11:08 PM UTC

Affected components

Site AvailabilityCommand ProcessingQubole SchedulerCluster Operations

Update timeline

  1. investigating Mar 27, 2021, 11:08 PM UTC

    We are currently investigating this issue.

  2. identified Mar 29, 2021, 07:19 PM UTC

    Devops has confirmed the cluster issue with the wellness environment. We're working on gathering internal resources to correct it.

  3. identified Mar 30, 2021, 06:34 PM UTC

    This issue may only impact a small subset of customers on wellness.qubole.com. Devops is confirming a change to get those customers back online.

  4. identified Mar 31, 2021, 08:19 PM UTC

    Devops is currently implementing a server change on the affected wellness environment that should resolve this issue. This incident will be updated again as soon as that is complete.

  5. resolved Apr 01, 2021, 11:15 AM UTC

    Devops has updated the wellness environment to correct this issue.