Servebolt incident
Elinor-osl sent old emails after a stopped queue had gone unnoticed
Servebolt experienced a minor incident on January 6, 2023, lasting —. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Jan 06, 2023, 08:52 PM UTC
At 20:34 a stale email queue with cronlog output was restarted when routine maintenance was conducted. The queue wasn't correctly configured in our monitoring systems and had therefore gone unnoticed. Upon restarting it a batch of emails was sent erroneously before the queue was stopped. The problem has been identified and fixed, we apologize for the confusion caused by this. We're implementing further checks and monitoring rules to prevent anything like this from occurring again.