Adnuntius experienced a notice incident on March 5, 2024 affecting Ad Serving, lasting 10h 47m. The incident has been resolved; the full update timeline is below.
Affected components
Ad Serving
Update timeline
- investigating Mar 05, 2024, 05:24 PM UTC
We are investigating over delivery in several campaigns
- identified Mar 05, 2024, 06:04 PM UTC
The issue was caused by a backlog of requests in one of our data centres.
- monitoring Mar 05, 2024, 06:05 PM UTC
The backlog has been cleared, and we are monitoring the results
- resolved Mar 06, 2024, 04:12 AM UTC
A feedback loop between increased traffic and rate control has been identified, and we introduced a circuit breaker mechanism as an immediate fix. We will make further adjustments to this mechanism and to the system architecture for handling load after testing these sufficiently to ensure there are no side effects.