Census incident

Delays starting scheduled syncs

Minor Resolved View vendor source →

Census experienced a minor incident on June 27, 2024 affecting Sync Engine, lasting 20h 57m. The incident has been resolved; the full update timeline is below.

Started
Jun 27, 2024, 08:46 PM UTC
Resolved
Jun 28, 2024, 05:44 PM UTC
Duration
20h 57m
Detected by Pingoru
Jun 27, 2024, 08:46 PM UTC

Affected components

Sync Engine

Update timeline

  1. investigating Jun 27, 2024, 08:46 PM UTC

    We are currently experiencing delays of up to 20 minutes starting scheduled syncs due to a downstream vendor dependency. This is especially pronounced for syncs that run at common times (e.g., around hour boundaries). We are working with the vendor to identify and fix the issue.

  2. monitoring Jun 28, 2024, 03:36 AM UTC

    We have identified the issue, which appears to have predominantly impacted syncs scheduled to run around 9am Eastern/6am Pacific. We have put in place a mitigation and will monitor sync runs running that time tomorrow to make sure the issue does not recur.

  3. resolved Jun 28, 2024, 05:44 PM UTC

    We have confirmed that syncs are now running as scheduled.