Kevel experienced a minor incident on May 1, 2024 affecting Ad Serving, lasting 2h 55m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating May 01, 2024, 09:51 PM UTC
We are seeing issues with our Real Time Reporting API. This system powers the data within our UI, as well as ad pacing and delivery. We are investigating at this time.
- identified May 01, 2024, 10:29 PM UTC
We have identified a misconfiguration in our Real Time Reporting system that resulted in messages not being processed. Ads that reached their caps after 04:00 UTC were not properly removed from the ad servers, which may have led to over delivery. Ad pacing is also impacted. We have determined the cause of the misconfiguration and are working on a fix.
- monitoring May 01, 2024, 11:09 PM UTC
We are continuing to work on a fix for the identified issue.
- identified May 01, 2024, 11:21 PM UTC
We are continuing to work on a fix for the identified issue.
- monitoring May 01, 2024, 11:32 PM UTC
A fix has been implemented and messages are now processing. We expect the system to take some time to process all message and remove all ads that have reached their caps. We will continue to monitor and update when complete.
- resolved May 02, 2024, 12:47 AM UTC
Our queue has caught up and this issue has been resolved. Ads were not properly removed after hitting their caps or paced properly from about 04:00 UTC to 23:30 UTC on May 1st. All ads should now be serving as expected.