Pronto experienced a critical incident on January 24, 2022 affecting Pronto Platform and Pronto Web App, lasting 2h 57m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jan 24, 2022, 11:17 PM UTC
We have been alerted to an issue with our backend database that is causing a system outage. We are currently investigating.
- investigating Jan 24, 2022, 11:35 PM UTC
Our backend database provider has acknowledged an issue on their platform and are working to diagnose. We will continue to update here as soon as we know more.
- investigating Jan 25, 2022, 12:31 AM UTC
Our database vendor has identified a hardware issue in the database cluster and is working to mediate.
- identified Jan 25, 2022, 12:50 AM UTC
As mentioned in the previous update, the issue is due to some underlying hardware failures. Our database vendor is working to move the affected components to new hardware.
- monitoring Jan 25, 2022, 01:15 AM UTC
Some of the affected components have been restored and we are starting to see some improvement. The Pronto service is back up, but may have degraded performance for a little while as the other components are restored.
- resolved Jan 25, 2022, 02:14 AM UTC
The Pronto platform is back to normal. We will be analyzing data from this incident over the coming days to understand what went wrong and how to better prevent a similar event in the future.