Percy incident

API Outage

Critical Resolved View vendor source →

Percy experienced a critical incident on July 19, 2022 affecting API, lasting 12m. The incident has been resolved; the full update timeline is below.

Started
Jul 19, 2022, 07:25 PM UTC
Resolved
Jul 19, 2022, 07:38 PM UTC
Duration
12m
Detected by Pingoru
Jul 19, 2022, 07:25 PM UTC

Affected components

API

Update timeline

  1. investigating Jul 19, 2022, 07:25 PM UTC

    We are currently investigating an issue where our API is not responding.

  2. identified Jul 19, 2022, 07:27 PM UTC

    The issue has been identified and a fix is being implemented.

  3. monitoring Jul 19, 2022, 07:27 PM UTC

    A fix has been implemented and we are monitoring the results.

  4. resolved Jul 19, 2022, 07:38 PM UTC

    This issue was caused by a bad database query that was shipped to our API. This part of our system was turned off to mitigate the issue. Many API requests would have timed out or 500 errored between the incident timeframe of: July 19 7:12 PM to 7:34 PM UTC. We're sorry for this issue.