Is Permutive down?

We are currently investigating an issue affecting Audience and Campaign Insights, where some customers may see errors or missing data when loading Insights. This appears to be caused by heavy load on our reporting infrastructure, which is causing some queries to time out. This affects the Insights reporting layer only — data collection, cohort building, and campaign activation and delivery are not impacted, and no data is being lost. Our engineering team is actively working on it. We will provide a further update as soon as possible.

We have applied fixes to our reporting infrastructure, and Audience and Campaign Insights have recovered — queries are now loading normally in our testing. Earlier today, some customers experienced errors and missing data in Insights. This was caused by a temporary surge in processing: as data backfilling was carried out, the sudden influx of data generated unusually heavy queries, which triggered the reporting system's built-in safeguards and caused some Insights queries to time out. This affected the Insights reporting layer only — data collection, cohort building, and campaign activation and delivery were not impacted, and no data was lost. We are monitoring closely to confirm stability and will carry out a full review to prevent recurrence.

This incident has been resolved. Audience and Campaign Insights are fully operational and performing normally. Summary: On the afternoon of 4 June, some customers experienced errors and missing data when loading Audience and Campaign Insights over a period of roughly a few hours. The issue was caused by a surge of unusually heavy reporting queries — combined with a backfill of data — which triggered the reporting system's built-in load safeguards and caused some Insights queries to time out. This affected the Insights reporting layer only. Data collection, cohort building, and campaign activation and delivery were not impacted, and no data was lost. Resolution: We applied fixes to our reporting infrastructure on the evening of 4 June, which restored normal performance, and we have confirmed Insights is loading correctly across customers. Next steps: Our investigation identified inefficiencies in how some reporting data is indexed, which made certain queries more expensive than expected. We will be making improvements to the reporting system's data structure and storage over the coming weeks to make it more resilient and prevent recurrence. We apologise for the disruption and thank you for your patience.

Between 26 March 2026 and 17 April 2026, segment metadata was not being delivered to streaming routing BigQuery destinations. This was identified on 15th April. All other event routing continued to operate normally throughout this period and was unaffected. We have identified the root cause, and a fix has already been rolled out to the affected destinations. A tested backfill solution for segment metadata is available. Please contact [email protected] for more details. We sincerely apologise for the disruption and for the delay in communicating it. As a follow-up, we are investing in improved monitoring so that per-destination write failures are detected proactively in future.

The fix has been fully rolled out, and segment metadata delivery to streaming Routing BigQuery destinations has been confirmed as restored. All affected destinations are operating normally. A tested backfill solution for segment metadata remains available — please contact [email protected] if you would like us to restore segment metadata for the affected period. Thank you for your patience throughout this incident.

We are aware of an issue affecting audience volume data displayed in Insights dashboards. Some customers may notice reduced audience sizes for cohorts since 4 April. Underlying data collection is unaffected and no data has been lost. The issue is limited to how volumes are reflected in the dashboard. Our engineering team is actively working on a fix and we expect to recover the affected data. We will provide a further update as soon as we have more information.

We have identified a contributing cause and recovery is underway. Data in Audience Insights and Campaign Insights has been restored. Plans data repair is still in progress.

This incident has been resolved.