Dataclay incident

QUE API experiencing technical issues

Critical Resolved View vendor source →

Dataclay experienced a critical incident on January 19, 2023 affecting API, lasting 9h 35m. The incident has been resolved; the full update timeline is below.

Started
Jan 19, 2023, 12:13 PM UTC
Resolved
Jan 19, 2023, 09:48 PM UTC
Duration
9h 35m
Detected by Pingoru
Jan 19, 2023, 12:13 PM UTC

Affected components

API

Update timeline

  1. investigating Jan 19, 2023, 12:13 PM UTC

    We are aware of the QUE API being unreachable and are investigating the issue now. We apologize for this inconvenience and thank you for your patience.

  2. investigating Jan 19, 2023, 12:23 PM UTC

    While we continue to investigate the issue, we have pointed the API domain to our backup production environment. Users can now work with their QUE data, but we will be investigating the primary production environment failure and updating later. Thank you for your patience.

  3. identified Jan 19, 2023, 05:04 PM UTC

    The identified issue relates to the QUE API server maxing out its host capacity. We are currently upgrading server capacity to ensure stability of the environment as request queries increase as more users integrate QUE with their systems. We will update more as we test out the rebuilt environment.

  4. monitoring Jan 19, 2023, 05:23 PM UTC

    The primary production environment was rebuilt with increased resource capacity. We will continue to monitor the API to verify these new specifications resolve the issue.

  5. resolved Jan 19, 2023, 09:48 PM UTC

    After monitoring the updated primary environment we are now resolving this incident. Future incidents will be reported as they come up. Thank you for your patience, and happy rendering!