Permutive experienced a major incident on November 1, 2019 affecting Permutive API, lasting 5h 35m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Nov 01, 2019, 12:26 PM UTC
The Permutive API is currently experiencing a partial outage, as a result of a global networking issue with Google Cloud. For some customers, a high proportion of the events sent by the Permutive SDK will be lost, although targeting for returning users / cached segments will remain unaffected. We are communicating to those customers who are impacted, and we will continue to provide updates via this status page. Details about the Google Cloud outage can be found here: https://status.cloud.google.com/incident/compute/19008#19008004.
- investigating Nov 01, 2019, 03:43 PM UTC
The Permutive API is still experiencing a partial outage, as a result of a global issue with Google Cloud. For all customers, we have created a workaround to allow the Permutive SDK to provide segmentation and client-side activation as normal; new events generated will be used in situ on respective user devices. While server-side integrations (TheTradeDesk, DV360, SalesForce Marketing Cloud, Adestra, SailThru) are not populating segments in partner platforms, we are collecting a backlog and expect to transfer all pending data when the Google cloud issue is resolved. For some customers, a high proportion of events are not being collected server-side by the API, and will therefore be unavailable for future analytics. Cross device/cross-domain users will also not have their state synced across devices/domains in these instances. We will communicate specific details to those customers affected. Resolution of the issue is dependent on Google's resolution of the Google Cloud outage, details of which can be found here: https://status.cloud.google.com/incident/compute/19008#19008004. We will continue to monitor the situation, and provide updates via this status page.
- resolved Nov 01, 2019, 07:08 PM UTC
This incident has now been resolved and full service restored for all customers. Our Engineering team are continuing to monitor.