Firmhouse experienced a major incident on October 8, 2024 affecting Portal and Main Platform and Checkout & Self Service Center and 1 more component, lasting 13h 11m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 08, 2024, 05:08 PM UTC
We are currently investigating instability in connecting with Firmhouse pages and APIs.
- investigating Oct 08, 2024, 05:10 PM UTC
We are continuing to investigate this issue.
- identified Oct 08, 2024, 05:39 PM UTC
We have identified the issues relating to our underlying hosting platform. We're implementing mitigating actions.
- identified Oct 08, 2024, 06:18 PM UTC
Our hosting platform has posted a status update: https://status.heroku.com/incidents/2684. We are still implementing mitigating measures.
- identified Oct 08, 2024, 09:32 PM UTC
We have put several mitigating solutions in place and set up a secondary infrastructure and rerouted traffic for Portal and API. Some hosted merchant Checkout flows may still see degraded performance. Please get in touch with us with any questions or comments.
- monitoring Oct 08, 2024, 11:05 PM UTC
We have put mitigating measures in place and the outage is resolved at our infrastructure provider. We will keep monitoring throughout the night if everything is back up as it should be.
- resolved Oct 09, 2024, 06:19 AM UTC
Our engineers have monitored the situation during the night. Everything seems to be resolved at our infrastructure provider Heroku/SalesForce so we are closing this incident as resolved.