Doppler experienced a critical incident on February 6, 2020, lasting —. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Feb 07, 2020, 12:05 AM UTC
We had a migration script that failed to run before the rest of our services deployed. This caused the newly deployed services to crash because the database was missing a column in a critical table. Our deployment orchestrator quickly detected the issue and reverted to the previous build. Total downtime was approximately 2 minutes. We plan to mitigate future failures by requiring our migrations to complete before rolling out our remaining services.