Hevo incident
Possible replication interruptions from GCP outage
Hevo experienced a major incident on June 12, 2025 affecting Sources and Sources and 1 more component, lasting 3h 43m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jun 12, 2025, 07:27 PM UTC
We’re seeing some pipeline failures over the past 40 minutes caused by an ongoing Google outage, which can be tracked here: https://status.cloud.google.com/ Our engineering team is actively investigating the impact. We will keep you posted on this.
- investigating Jun 12, 2025, 07:30 PM UTC
We are continuing to investigate this issue.
- monitoring Jun 12, 2025, 08:56 PM UTC
Update: All pipelines except the ones connected from the us-central1 GCS environment have now started to recover. We expect to see some pipelines fail for a short period of time before getting rescheduled and succeeding. If there were any ingestion triggers to run in the past hour, they would have failed. GCS Status page: https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW
- monitoring Jun 12, 2025, 09:34 PM UTC
All sources and Destinations should recover automatically. In case you see any entities that are still failed, please re-authorize them, and it should then recover the entity.
- monitoring Jun 12, 2025, 09:45 PM UTC
We are continuing to monitor for any further issues.
- resolved Jun 12, 2025, 11:11 PM UTC
This incident has been resolved.