Qubole experienced a minor incident on August 5, 2021 affecting Command Processing and Cluster Operations, lasting 4h 26m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Aug 05, 2021, 02:42 PM UTC
us.qubole.com is currently seeing some degraded performance and is returning errors during cluster start and command processing. At this time failures appear to be partial and intermittent. Currently, DevOps is investigating.
- identified Aug 05, 2021, 02:59 PM UTC
DevOps is continuing to work on this issue. We have identified that one of the tunnel server node is having an issue.
- monitoring Aug 05, 2021, 03:07 PM UTC
There seems to be an issue with one of the tunnel servers node in us.qubole.com, we have removed the bad tunnel node. The team continues to monitor the environment's stability.
- monitoring Aug 05, 2021, 04:54 PM UTC
We are continuing to monitor for any further issues.
- resolved Aug 05, 2021, 05:26 PM UTC
DevOps has confirmed that after replacing the bad tunnel node the us.qubole.com seems to be working fine.