Percy incident

Degraded performance in percy - builds taking longer to finish

Major Resolved View vendor source →

Percy experienced a major incident on September 23, 2024 affecting Rendering infrastructure, lasting 30m. The incident has been resolved; the full update timeline is below.

Started
Sep 23, 2024, 02:01 PM UTC
Resolved
Sep 23, 2024, 02:32 PM UTC
Duration
30m
Detected by Pingoru
Sep 23, 2024, 02:01 PM UTC

Affected components

Rendering infrastructure

Update timeline

  1. investigating Sep 23, 2024, 05:02 PM UTC

    We are currently investigating this issue.

  2. identified Sep 23, 2024, 05:02 PM UTC

    The issue has been identified and a fix is being implemented.

  3. monitoring Sep 23, 2024, 05:02 PM UTC

    A fix has been implemented and we are monitoring the results. We are waiting for backed up queues to get processed.

  4. resolved Sep 23, 2024, 05:03 PM UTC

    This incident has been resolved. No jobs failed, the system was slower than usual and you might have seen longer build processing time.