Runn experienced a major incident on March 5, 2025, lasting 2h. The incident has been resolved; the full update timeline is below.
Update timeline
- investigating Mar 05, 2025, 09:02 PM UTC
Runn started experiencing intermittent downtime of our app at approximately 19:30 UTC. Users trying to load the app would see an message "There has been an error".
- identified Mar 05, 2025, 09:04 PM UTC
At approximately 19:35 UTC we started reseting the servers which brought the back up temporarily before going down again. We contined to investigate and regularly reset the servers.
- identified Mar 05, 2025, 09:05 PM UTC
At approximately 20:15 UTC we identified a issue with our database that was causing the database not to respond.
- identified Mar 05, 2025, 09:06 PM UTC
At approximately 20:20 UTC we resolved the issue with the database, and app.runn.io appears to now be running normally.
- monitoring Mar 05, 2025, 09:09 PM UTC
We have a high degree of confident this has resolved the ongoing issue. We will continue to monitor to see if the issue re-occurs and take action. The team is working on a perment fix that would avoid this happening again. We do not expect any additional downtime before the fix is deployed, as this issue has been within the app for several month, and only caused downtime due to a unlucky combination of factors. We expect the perment fix to be released in the next several hours.
- monitoring Mar 05, 2025, 09:11 PM UTC
Note that his incident was unrelated to the incident several days ago which was caused by our hosting provider.
- resolved Mar 05, 2025, 11:03 PM UTC
We have created a permanent fix for this issue and are in the process of deploying. We consider this incident now resolved. Runn is sorry for any inconvinced caused to our customers by this incident. We will be reviewing this incident to see how we can improve our processes and avoid a similar incident in the future.