Census experienced a major incident on September 20, 2024 affecting Sync Engine, lasting 2h 37m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Sep 20, 2024, 08:18 PM UTC
Sync Alerts were deleted from our database in the last 2 days. The cause has been identified, and we are working on restoring the deleted alerts.
- identified Sep 20, 2024, 09:36 PM UTC
As part of this incident, any sync runs that should have triggered an alert in the last 2 days would not have resulted in an email alert. Once we restore the alert data, any sync runs that fail will trigger an alert, even though the sync may have been failing for the last 2 days. In very rare cases, we may email you at most twice about a failure as we perform the recovery process.
- resolved Sep 20, 2024, 10:55 PM UTC
Sync Alert data has been restored. Sync alerting should work again for any syncs that trigger alerts moving forward.