Tilroy Outage History

## Incident Report: PDF generating Issues On 19/03/2026 , our platform experienced a service interruption , which prevented the creation of PDF invoices. We understand the significant impact this has on your daily operations, and we sincerely apologize for the inconvenience this caused. ## What happened? To generate invoices within our software, we use a background service that controls a digital browser \(similar to Chrome\) to format and prepare your documents as PDFs. In the past, we had to manually configure and update this browser to ensure it stayed compatible with our specific systems. ## What went wrong? Yesterday, an automatic update occurred within the browser software’s external environment. This new version unexpectedly became incompatible with our PDF module. The issue was particularly persistent because our systems are designed to automatically fetch the latest browser version upon startup. As a result, even rolling back to a previous version of our PDF software did not resolve the issue, as the system continued to install the new, incompatible browser version. This explains why it took longer than usual to fully restore the service. ## How was this resolved? Our engineering team has changed how the PDF module is structured to ensure long-term stability: * **Integrated Linking:** We have rebuilt the system so that the PDF module now directly manages its own specific browser version. They are now treated as a single, unified package. * **Guaranteed Compatibility:** This change ensures that the module and the browser can never "fall out of sync." If one is updated, the other is automatically and safely updated alongside it. * **Stabler Infrastructure:** We have migrated to a new technical foundation specifically optimized for these tasks, increasing the overall reliability of our document generation. ## Conclusion The outage was caused by an external update that no longer aligned with our manual configuration. By changing the underlying technology and giving the system full control over its own dependencies, we have eliminated this risk. This ensures that a similar compatibility issue cannot occur in the future. Thank you for your patience and for your continued trust in our platform.

First we sincerely apologize for the disruption to your operations on 05/02 morning. We know that in retail, every second at the checkout counts, and we regret the frustration this caused your teams and customers. **What happened?** We recently introduced an update designed to make the system even more resilient. It included a "fallback" feature, essentially a safety net, to catch and fix rare errors that occasionally caused a sale to fail. While this update performed perfectly in our testing environment for several weeks, a very specific and unusual combination of data occurred in the live environment yesterday morning. This specific data "confused" the safety net, causing the service to shut down unexpectedly. **Why did it spread?** Our system is designed to be "self-healing." If one part goes down, another takes over immediately, and the first part restarts itself within a minute or two. However, because the "bad data" was still being processed, it triggered the same crash across all our backup systems one after another. This created a loop where the services were restarting, but being knocked offline again before they could fully recover. **How are we preventing this?** * **Immediate Fix:** We have identified the specific data combination that caused the crash and updated our code to handle it safely. * **Expanded Testing:** This specific scenario has been added to our permanent testing suite to ensure it can never happen again. We are committed to providing a stable platform for your business and are treating this as a priority learning moment for our engineering team.