Census incident

GCP Outage May Affect Syncs to and from Google Resources

Minor Resolved View vendor source →

Census experienced a minor incident on June 12, 2025 affecting Sync Engine, lasting 2h 35m. The incident has been resolved; the full update timeline is below.

Started
Jun 12, 2025, 06:30 PM UTC
Resolved
Jun 12, 2025, 09:05 PM UTC
Duration
2h 35m
Detected by Pingoru
Jun 12, 2025, 06:30 PM UTC

Affected components

Sync Engine

Update timeline

  1. monitoring Jun 12, 2025, 06:30 PM UTC

    We are aware of the current GCP outage and we are monitoring its impact on Census syncs to and from Google and Google Cloud Resources. Census does not run any critical components on GCP but we do use GCP resources, including Cloud Storage, to facilitate interaction with BigQuery and other Google Cloud resources.

  2. monitoring Jun 12, 2025, 06:37 PM UTC

    Internal monitoring indicates that syncs to and from BigQuery started experiencing increased failure rates at around 18:05 UTC. The rate of failures is now decreasing but has not returned to usual levels.

  3. monitoring Jun 12, 2025, 08:19 PM UTC

    Our monitoring systems continue to see a sustained error rate when communicating with Google BigQuery. We will monitor and report when the issue is resolved from Census' perspective. Google Cloud has provided additional information on their status page, located here: https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW

  4. resolved Jun 12, 2025, 09:05 PM UTC

    Census systems remain unaffected as GCP comes back online. We are now seeing the majority of our GCP hosted customers have successful syncs.