Uberflip experienced a critical incident on July 27, 2021 affecting Hubs and App and 1 more component, lasting 2h 41m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jul 27, 2021, 06:33 PM UTC
We are currently experiencing an issue which is impacting availability of the platform. The team is investigating.
- identified Jul 27, 2021, 07:59 PM UTC
Our Engineering team has identified the issue and are currently preparing a fix. We will provide additional updates as soon as the fix has been deployed and validated.
- monitoring Jul 27, 2021, 08:25 PM UTC
A fix has now been implemented and operations are returning to normal. The team will continue to monitor and further updates will be posted here.
- resolved Jul 27, 2021, 09:14 PM UTC
This incident has been resolved.
- postmortem Aug 04, 2021, 04:20 PM UTC
# **Technical Summary** At 14:14 ET on July 27, 2021 our systems reported an increase in latency to our primary database cluster due to an abnormal and significant increase in traffic. This increase outpaced our scaling capabilities, and resulted in a cascading failure that brought down the database cluster responsible for our Hub Platform. Recovery efforts were initiated quickly once the cause was identified, however a complication during recovery extended the outage. By 16:19 ET the Hub Platform was back online, and all systems were brought back to normal operation shortly thereafter. Remediation efforts on our infrastructure and product performance are underway to prevent a recurrence of this issue.