Redox incident
Filters are unavailable in the dashboard UI. Processing is not impacted.
Redox experienced a minor incident on February 6, 2025 affecting Dashboard Tools, lasting 47m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- identified Feb 06, 2025, 04:48 PM UTC
Viewing and editing filters is unavailable in the dashboard. Message processing with filters is not impacted. We have identified the issue and a fix is being implemented.
- resolved Feb 06, 2025, 05:36 PM UTC
This incident has been resolved.
- postmortem Feb 12, 2025, 02:18 PM UTC
# Summary * On February 6, 2025 at 1025CT we became aware of an issue that affected the ability to view or modify some filters in the Redox dashboard for a subset of customers; at 1136CT we deployed a fix that enabled users to once again interact with filters in the dashboard. * That subset included only customers who had filters that were previously linked to a deleted subscription and there was no impact to filter logic or message processing during this time. ## What Happened * On February 5 we implemented a code change to support an upcoming enhancement to the customer filters experience. * After implementation, we uncovered that, in cases where a subscription had a filter and was later deleted, the user would receive an error; this prevented filters from loading for other subscriptions in that same environment, which meant users could not interact with them for a period of time. * Approximately an hour after we learned of the issue, we deployed a fix and users were again able to interact with filters in the dashboard as expected. ## What we are doing about this: * We are adding additional alerting that would support proactively identifying this type of issue in the future. * We are expanding our understanding of edge cases and incorporating additional testing to support those scenarios.