Threekit incident

AR jobs are stuck in pending status

Minor Resolved View vendor source →

Threekit experienced a minor incident on September 23, 2021 affecting AR Service and AR Service, lasting 4h 15m. The incident has been resolved; the full update timeline is below.

Started
Sep 23, 2021, 03:23 PM UTC
Resolved
Sep 23, 2021, 07:38 PM UTC
Duration
4h 15m
Detected by Pingoru
Sep 23, 2021, 03:23 PM UTC

Affected components

AR ServiceAR Service

Update timeline

  1. investigating Sep 23, 2021, 03:23 PM UTC

    Our team has identified an issue regarding AR jobs which is causing them to remain in a pending state. Due to this AR jobs may not complete at this time.

  2. identified Sep 23, 2021, 04:35 PM UTC

    Our engineering team has identified the portion of code causing the issue, and have begun the process of implementing a fix for it

  3. monitoring Sep 23, 2021, 07:17 PM UTC

    We have implemented the fix onto our preview environment, and AR jobs are now completing successfully. We are now pushing this fix into production.

  4. resolved Sep 23, 2021, 07:38 PM UTC

    The issue has been resolved with a code change on our production environment.