Aviator incident

Elevated error rates on GitHub API

Major Resolved View vendor source →

Aviator experienced a major incident on November 3, 2023 affecting Background queues, lasting 1h 15m. The incident has been resolved; the full update timeline is below.

Started
Nov 03, 2023, 06:51 PM UTC
Resolved
Nov 03, 2023, 08:07 PM UTC
Duration
1h 15m
Detected by Pingoru
Nov 03, 2023, 06:51 PM UTC

Affected components

Background queues

Update timeline

  1. monitoring Nov 03, 2023, 06:51 PM UTC

    We are seeing degraded performance of GitHub APIs failing due to authentication issues, and are monitoring the situation closely. The queues will have degraded performance due to this incident.

  2. monitoring Nov 03, 2023, 06:52 PM UTC

    We are continuing to monitor for any further issues.

  3. identified Nov 03, 2023, 06:52 PM UTC

    We are seeing degraded performance of GitHub APIs failing due to authentication issues, and are monitoring the situation closely. The queues will have degraded performance due to this incident.

  4. monitoring Nov 03, 2023, 07:37 PM UTC

    The API calls have recovered, we are just chugging along the backlog in queues, should be caught up in next 10-15 mins.

  5. resolved Nov 03, 2023, 08:07 PM UTC

    This incident has been resolved.