Dataclay incident

QUE API — some 504 gateway timeouts

Minor Resolved View vendor source →

Dataclay experienced a minor incident on May 14, 2025 affecting API, lasting 1d 22h. The incident has been resolved; the full update timeline is below.

Started
May 14, 2025, 05:07 PM UTC
Resolved
May 16, 2025, 03:42 PM UTC
Duration
1d 22h
Detected by Pingoru
May 14, 2025, 05:07 PM UTC

Affected components

API

Update timeline

  1. investigating May 14, 2025, 05:07 PM UTC

    We are investigating some QUE API gateway timeouts on the /jobs endpoint. We will update as we discover more information.

  2. monitoring May 15, 2025, 02:47 PM UTC

    After investigating the timeout issue with the QUE API endpoints, we have identified two possible causes. We have issued a fix and are monitoring activity.

  3. resolved May 16, 2025, 03:42 PM UTC

    After investigation, we resolved an issue causing slowdowns and some timeouts in job logging queries. We also added rules to mitigate a series of malformed requests that were not successfully connecting to QUE endpoints, but were also slowing or causing some timeouts for QUE user clients. This appears to have resolved the issues we were seeing. If you continue to experience any connection timeouts, please contact Dataclay support.