Fiix Software experienced a notice incident on July 31, 2024 affecting Fiix Desktop and Fiix Mobile, lasting 5d 2h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- monitoring Jul 31, 2024, 09:52 AM UTC
Intermittent short outages in our desktop application have been mitigated . We will continue to monitor this. We do currently expect some delays in the generation of work orders from scheduled maintenances . If you have any issues accessing the application, please contact [email protected] for further assistance
- monitoring Jul 31, 2024, 02:20 PM UTC
We are continuing to monitor for any further issues.
- monitoring Aug 01, 2024, 11:59 PM UTC
We’ve deployed a fix that improves the performance of scheduled maintenances , users should now see fewer delays in work order generation . We will continue to monitor , please contact [email protected] for assistance
- resolved Aug 05, 2024, 12:20 PM UTC
This incident has been resolved.
- postmortem Aug 15, 2024, 10:13 PM UTC
### What Happened? On July 27th, we upgraded our database to the latest version. Despite thorough testing, starting July 29th, some users experienced slow performance and issues with the mobile app. ### Root Cause The new database version handled data differently than expected. Our tests didn’t fully capture the real-world usage, leading to problems when many users logged in on Monday morning. ### Actions Taken When we noticed the issues, we quickly reduced the load on our main database by redirecting some requests to backup databases. This helped but caused some mobile app sync problems. We worked on improving the new database, restoring normal performance by August 6th, and fixing the mobile sync issue by August 7th. ### Preventive Measures To prevent this from happening again, we will: * Improve our testing to better simulate real-world usage. * Roll out future updates in stages. * Optimize how our mobile app handles data. * Work with users to identify and fix any problematic queries.