Signable incident
Investigating - High Severity - Signable services unavailable
Signable experienced a critical incident on March 21, 2024 affecting Web App and API and 1 more component, lasting 6m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Mar 21, 2024, 05:21 PM UTC
We are investigating an issue impacting Signable Services. We will share an update as soon as we have more information. Thank you for your patience.
- resolved Mar 21, 2024, 05:28 PM UTC
We can confirm that our monitoring is showing that the issues impacting Signable services have been resolved. If you continue to experience problems please contact our support team [email protected]. Thank you for your patience.
- postmortem Mar 21, 2024, 05:47 PM UTC
**What happened?** At 16:36 21/03/24 abnormally large load caused our download service to go down. Due to this service being relied upon by a number of other key services, this caused a cascade of errors which eventually overloaded our database taking down all services at around 16:56. Upon investigating the issue we immediately triggered a failover of the database which brought back all services at 17:16. **What are we doing about this?** We are currently in the process of migrating our download service to a new more scalable, fault tolerant and optimised service. In the case of large download loads, this new service will be able to scale up to meet requirements and in the case of error won’t have the same knock on affect. This change is scheduled within the next month.