Voyado experienced a minor incident on October 14, 2025 affecting API, lasting 37m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 14, 2025, 09:08 AM UTC
We are currently investigating this issue.
- identified Oct 14, 2025, 09:19 AM UTC
The issue has been identified and a fix is being implemented.
- resolved Oct 14, 2025, 09:45 AM UTC
Incident is resolved. No more errors should be seen.
- postmortem Oct 14, 2025, 02:32 PM UTC
**Description** During an A/B test of personalization functionality, an update introduced data that was not compatible with older system versions. This caused search-page errors for some shoppers when their requests were routed between nodes running different versions. The A/B test was disabled promptly, preventing new incompatible data from being written. This action significantly reduced the error rate. Errors continued to occur for a limited number of shoppers who had already been affected, until all older nodes were fully upgraded. * The majority of customers running the test experienced no impact. * Most customers faced none or a few errors. All systems were upgraded to the new version, eliminating the incompatibility and restoring full stability. **Business impact** * Some shoppers attempting searches received error messages, which could disrupt their browsing and purchase flow. * Only customers actively running the personalization A/B test were affected. For most, the number of shoppers impacted was very limited. **Review** * Review of rollout and A/B testing procedures to reduce cross-version risk. * Improve error handling in personalization so that issues cannot cause shopper-facing failures. * Add faster “kill switch” capabilities to disable personalization in case of future incidents.