Qubole incident

Degraded performance issue on api.qubole.com

Minor Resolved View vendor source →

Qubole experienced a minor incident on July 29, 2021 affecting QDS API and Command Processing and 1 more component, lasting 1h 46m. The incident has been resolved; the full update timeline is below.

Started
Jul 29, 2021, 01:43 PM UTC
Resolved
Jul 29, 2021, 03:30 PM UTC
Duration
1h 46m
Detected by Pingoru
Jul 29, 2021, 01:43 PM UTC

Affected components

QDS APICommand ProcessingQubole Scheduler

Update timeline

  1. investigating Jul 29, 2021, 01:43 PM UTC

    Api.qubole.com is currently seeing some degraded performance and is returning errors during access. At this time failures appear to be partial and intermittent, but DevOps is investigating.

  2. investigating Jul 29, 2021, 01:50 PM UTC

    We are continuing to investigate this issue.

  3. identified Jul 29, 2021, 02:14 PM UTC

    We have identified that one of the tunnel server node is having an issue.

  4. monitoring Jul 29, 2021, 02:57 PM UTC

    Replacement of bad node is completed and traffic flowing through it.

  5. resolved Jul 29, 2021, 03:30 PM UTC

    It seems that the tunnel server node issue is resolved as api.qubole.com is now working fine.