PetDesk incident

Elevated Errors & Latency

Major Resolved View vendor source →

PetDesk experienced a major incident on August 15, 2022 affecting Provider Communications Dashboard, lasting 6h 42m. The incident has been resolved; the full update timeline is below.

Started
Aug 15, 2022, 02:02 PM UTC
Resolved
Aug 15, 2022, 08:44 PM UTC
Duration
6h 42m
Detected by Pingoru
Aug 15, 2022, 02:02 PM UTC

Affected components

Provider Communications Dashboard

Update timeline

  1. investigating Aug 15, 2022, 02:02 PM UTC

    We are investigating reports of elevated errors and slow loading times.

  2. investigating Aug 15, 2022, 04:34 PM UTC

    Some users are experiencing issues loading pages in our Dashboard web application. We are continuing to investigate this issue.

  3. investigating Aug 15, 2022, 06:02 PM UTC

    After investigation and load reductions, we continue to see elevated resource usage on the primary DB.

  4. investigating Aug 15, 2022, 06:03 PM UTC

    Attempting to resolve through failover to secondary

  5. monitoring Aug 15, 2022, 06:24 PM UTC

    A fix has been implemented and we are monitoring the results.

  6. resolved Aug 15, 2022, 08:44 PM UTC

    Failover to DB secondary and reboot of primary resolved issues. DB CPU continues to run slightly higher than in recent history, but this is explicable by normal variations in usage. Incident Resolved.