BenchPrep incident

Degraded Performance

Minor Resolved View vendor source →

BenchPrep experienced a minor incident on February 10, 2021 affecting API and Learning Application, lasting 1d 3h. The incident has been resolved; the full update timeline is below.

Started
Feb 10, 2021, 05:59 PM UTC
Resolved
Feb 11, 2021, 09:46 PM UTC
Duration
1d 3h
Detected by Pingoru
Feb 10, 2021, 05:59 PM UTC

Affected components

APILearning Application

Update timeline

  1. investigating Feb 10, 2021, 05:59 PM UTC

    We are experiencing slower than normal load times and reports of pages unable to load. We are currently investigating the issue and will post any relevant updates here.

  2. monitoring Feb 10, 2021, 06:18 PM UTC

    We have identified an issue in our Dallas datacenter and have routed all traffic to San Jose for the time being. We will continue to investigate.

  3. monitoring Feb 10, 2021, 07:06 PM UTC

    We are currently routing all traffic to San Jose and are working with our infrastructure provider to address some network connectivity issues. All services are operational.

  4. investigating Feb 11, 2021, 02:57 PM UTC

    We are currently investigating an issue related to memory contention on one of our databases.

  5. monitoring Feb 11, 2021, 05:14 PM UTC

    We have identified an issue with our database connection pooling software. Restarting it has temporarily resolved the issue, we are monitoring and looking for permanent solutions.

  6. monitoring Feb 11, 2021, 07:54 PM UTC

    We are continuing to monitor for any further issues.

  7. resolved Feb 11, 2021, 09:46 PM UTC

    Connectivity to both data centers has been restored and we are sending traffic to both locations now. We will continue to monitor the situation.