Resolved after 33 minutes, affecting Point Of Sale and E-commerce and 1 more, on February 5, 2026.
First we sincerely apologize for the disruption to your operations on 05/02 morning. We know that in retail, every second at the checkout counts, and we regret the frustration this caused your teams and customers. **What happened?** We recently introduced an update designed to make the system even more resilient. It included a "fallback" feature, essentially a safety net, to catch and fix rare errors that occasionally caused a sale to fail. While this update performed perfectly in our testing environment for several weeks, a very specific and unusual combination of data occurred in the live environment yesterday morning. This specific data "confused" the safety net, causing the service to shut down unexpectedly. **Why did it spread?** Our system is designed to be "self-healing." If one part goes down, another takes over immediately, and the first part restarts itself within a minute or two. However, because the "bad data" was still being processed, it triggered the same crash across all our backup systems one after another. This created a loop where the services were restarting, but being knocked offline again before they could fully recover. **How are we preventing this?** * **Immediate Fix:** We have identified the specific data combination that caused the crash and updated our code to handle it safely. * **Expanded Testing:** This specific scenario has been added to our permanent testing suite to ensure it can never happen again. We are committed to providing a stable platform for your business and are treating this as a priority learning moment for our engineering team.
Affected components
Update timeline
- identified Feb 05, 2026, 08:53 AM UTC
At the moment Tilroy is not available. Use offline mode to continue. We have found the issue and are implementing a fix.
- monitoring Feb 05, 2026, 08:59 AM UTC
A fix has been implemented and we are monitoring the results.
- resolved Feb 05, 2026, 09:26 AM UTC
This incident has been resolved.
- postmortem Feb 09, 2026, 01:37 PM UTC
First we sincerely apologize for the disruption to your operations on 05/02 morning. We know that in retail, every second at the checkout counts, and we regret the frustration this caused your teams and customers. **What happened?** We recently introduced an update designed to make the system even more resilient. It included a "fallback" feature, essentially a safety net, to catch and fix rare errors that occasionally caused a sale to fail. While this update performed perfectly in our testing environment for several weeks, a very specific and unusual combination of data occurred in the live environment yesterday morning. This specific data "confused" the safety net, causing the service to shut down unexpectedly. **Why did it spread?** Our system is designed to be "self-healing." If one part goes down, another takes over immediately, and the first part restarts itself within a minute or two. However, because the "bad data" was still being processed, it triggered the same crash across all our backup systems one after another. This created a loop where the services were restarting, but being knocked offline again before they could fully recover. **How are we preventing this?** * **Immediate Fix:** We have identified the specific data combination that caused the crash and updated our code to handle it safely. * **Expanded Testing:** This specific scenario has been added to our permanent testing suite to ensure it can never happen again. We are committed to providing a stable platform for your business and are treating this as a priority learning moment for our engineering team.
Your product runs on more than Tilroy
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services