Metabase experienced a major incident on August 10, 2021 affecting Metabase Cloud Platform, lasting 4h 1m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Aug 10, 2021, 04:37 PM UTC
We identified an issue in our Business tier customers that run on isolated environments. The issue appears to be an issue with our load balancers not being able to flag instances as healthy which results in application servers being recycled continously.
- identified Aug 10, 2021, 05:46 PM UTC
We've identified a fix for the outage and we're in the process of issuing an update to resolve the situation
- monitoring Aug 10, 2021, 06:46 PM UTC
We've deployed a fix and we're monitoring the health of the instances.
- resolved Aug 10, 2021, 08:38 PM UTC
This incident has been resolved.
- postmortem Aug 10, 2021, 11:25 PM UTC
A failure in the upgrade in our Elastic Beanstalk deployment for improving the timeout on reverse proxies led to instances not being able to revert back to previous version leaving them in an inconsistent state.