Voucherify incident

Search API - degraded performance

Minor Resolved View vendor source →

Voucherify experienced a minor incident on July 6, 2020 affecting EU1 - API and EU1 - Dashboard, lasting 6h 47m. The incident has been resolved; the full update timeline is below.

Started
Jul 06, 2020, 10:40 AM UTC
Resolved
Jul 06, 2020, 05:27 PM UTC
Duration
6h 47m
Detected by Pingoru
Jul 06, 2020, 10:40 AM UTC

Affected components

EU1 - APIEU1 - Dashboard

Update timeline

  1. investigating Jul 06, 2020, 10:40 AM UTC

    Communication with API is currently being impacted by the anomalous behavior of one of our clusters - AS1. W are investigating the issue. The affected API methods are related to filtering customers and validation rules based on customer segmentation. The issue occurs only for tenants whos projects are hosted on AS1.

  2. monitoring Jul 06, 2020, 11:01 AM UTC

    A fix has been applied and Voucherify Team is monitoring the platform.

  3. resolved Jul 06, 2020, 05:27 PM UTC

    This incident has been resolved. We muted segments that use "contains" condition which was affecting our search API. Now we are working on the long-run solution which should replace inefficient search queries using "contains" condition.