Header Bidding Reporting Data Loss

Update timeline

1. investigating Apr 07, 2021, 12:16 AM UTC

   An issue leading to data loss in the header bidding analytics collection endpoints occurred for a short period on April 5th and again on April 6th. This issue does not effect revenue generation or analytics reporting.

2. identified Apr 07, 2021, 12:19 AM UTC

   During the roll out of a new data collection endpoint an issue leading to data loss was discovered on April 5th, 2021. This led to a small amount of data being lost on April 5th. A fix was applied at that time. On April 5th, the issue recurred. We determined that the underlying reason for the April 5th issue was more fundamental than initially identified and that the April 5th fix did not fully address the core issue. The underlying issue then resulted in more data loss on April 6th. We believe that we've identified the root issue is related to changes in variable marshalling in the new analytics endpoints. We are in the process of applying additional mitigation. While we monitor the fixes and work to mitigate the issue we believe any additional loss will be small and we expect full resolution shortly.

3. monitoring Apr 07, 2021, 01:14 PM UTC

   A fix has been implemented and we are monitoring it. When the fix is fully confirmed we will post a full post mortem.

4. resolved Apr 13, 2021, 08:44 PM UTC

   This incident has been resolved.