HireVue incident

Elastic Search Slowness (Recurring)

Minor Resolved View vendor source →

HireVue experienced a minor incident on September 20, 2022 affecting Admin / Recruiter and API and 1 more component, lasting 3h 39m. The incident has been resolved; the full update timeline is below.

Started
Sep 20, 2022, 03:14 PM UTC
Resolved
Sep 20, 2022, 06:54 PM UTC
Duration
3h 39m
Detected by Pingoru
Sep 20, 2022, 03:14 PM UTC

Affected components

Admin / RecruiterAPIEvaluations

Update timeline

  1. identified Sep 20, 2022, 03:14 PM UTC

    We are experiencing a secondary service disruption with our API, Candidate UI, and Evaluator UI components within our US region that began about 1 hour ago. The issue is with our backend search capability and is causing degraded performance with our public API, Candidate UI, and Evaluator UI. Error rates on search functions are running ~2-3%. Mitigation steps are currently underway to resize the Elastic Search cluster. We are making every effort to reduce the impact and length of the disruption. We apologize for any inconvenience this may cause, and we thank you for your understanding. Individual system components are listed below.

  2. identified Sep 20, 2022, 03:14 PM UTC

    We are continuing to work on a fix for this issue.

  3. identified Sep 20, 2022, 04:09 PM UTC

    We are continuing to work on a fix for this issue.

  4. identified Sep 20, 2022, 04:10 PM UTC

    We are experiencing a service disruption with our API, Candidate UI, and Evaluator UI components within our US region that began approximately 90 minutes ago. You may experience slow load speeds or failure to load when opening the evaluator for OnDemand interviews, loading the list of candidates on a position, and other tasks related to our public API, Candidate UI, and Evaluator UI. We have identified the cause and mitigation steps are currently underway to reduce the impact and length of the disruption. We apologize for any inconvenience this may cause, and we thank you for your understanding.

  5. monitoring Sep 20, 2022, 04:31 PM UTC

    A fix has been implemented and we are monitoring the results.

  6. monitoring Sep 20, 2022, 04:35 PM UTC

    We've resolved performance issues and are monitoring our systems closely to ensure that they remain healthy. We are no longer seeing degraded performance, and don't anticipate any more impacts to our services.

  7. resolved Sep 20, 2022, 06:54 PM UTC

    After closely monitoring, this issue is now resolved