Voucherify incident

AS1 - degraded performance

Minor Resolved View vendor source →

Voucherify experienced a minor incident on September 21, 2021 affecting AS1 - API and AS1 - Dashboard, lasting 3h 12m. The incident has been resolved; the full update timeline is below.

Started
Sep 21, 2021, 02:20 AM UTC
Resolved
Sep 21, 2021, 05:33 AM UTC
Duration
3h 12m
Detected by Pingoru
Sep 21, 2021, 02:20 AM UTC

Affected components

AS1 - APIAS1 - Dashboard

Update timeline

  1. investigating Sep 21, 2021, 02:20 AM UTC

    We are currently investigating the issue. We have noticed a too long response time for some API methods. This problem occurs only in the AS1 environment, which is hosted in the Singapore AWS region.

  2. identified Sep 21, 2021, 02:25 AM UTC

    The issue has been identified, and we applied the first fix. Now we will spend more time analyzing the root cause to implement the final solution.

  3. monitoring Sep 21, 2021, 03:21 AM UTC

    There's been an issue with automatic scaling of some of our resources. A fix has been applied and we are monitoring the results.

  4. resolved Sep 21, 2021, 05:33 AM UTC

    This incident has been solved. Ultimately, we introduced improved resource allocation on that cluster, and the cluster condition returned to normal.