Bitly experienced a critical incident on August 14, 2024 affecting Website (Bitly.com) and API and 1 more component, lasting 38m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Aug 14, 2024, 03:10 PM UTC
Hi community. We’re currently investigating issues impacting our website and API. We apologize if this is causing trouble for you. We’ll be sure to update you as soon as we know more!
- monitoring Aug 14, 2024, 03:34 PM UTC
We have implemented a fix for the issue that was affecting our Website, API, and Pages and we are seeing full recovery. We will continue monitoring and report back with any updates.
- resolved Aug 14, 2024, 03:49 PM UTC
This incident has been resolved.
- postmortem Aug 26, 2024, 05:48 PM UTC
On `2024-08-14` We experienced an outage affecting the Bitly API from `15:03Z` to `15:10Z` and affecting the Bitly website from `15:03Z` to `15:31Z`. The root cause was a failure in our deployment pipeline which caused the incorrect release to be deployed. This change was immediately detected and rolled back but the failure triggered an unknown dependency loop which required a manual rollback to restore some systems. We have already taken steps to resolve some root causes and are evaluating ways to improve our ability to rollback more quickly in similar situations.