Is Redox down?

At approximately 11:43am CT, Redox became aware of an issue with DocumentGet requests sent outbound to other organizations on the Carequality network. What this means: DocumentGet requests to retrieve documents from other organizations on the Carequality network might not return the document to the initiator. What we're doing: We are currently investigating and will provide updates as they become available. Please contact us at [email protected] if you have questions.

A fix has been implemented for and document requests are now working as expected. We're actively monitoring the situation to ensure continued successes. If you had failed DocumentGet queries for Carequality you can re-query and you will now get responses. If you have any additional questions, please contact [email protected]

The incident has been resolved and the Redox Engine has resumed normal operations. If you have any additional questions please reach out to [email protected]

## Summary Between Wednesday, July 8, 2026 at approximately 1:00 PM CT and Thursday, July 9, 2026 at 12:47 PM CT, customers using Carequality DocumentGet received empty document data in responses. Requests appeared to complete successfully on both ends, but document content was missing from what was returned. PatientSearch and PatientQuery were not affected. Service has been fully restored, and no data was lost. All documents were successfully retrieved during this window but not correctly included in outbound responses. ## What Happened The incident was triggered by a routine software update deployed on Wednesday afternoon. * The update introduced an incompatibility in how we format and map document data into outbound responses. Rather than producing an error, the affected step silently returned empty content. * Because the system reported no errors and appeared healthy, our automated monitoring did not alert. The issue was identified through customer reports on Thursday morning. * Once our engineering team identified the root cause, we rolled back the update, which restored correct behavior. Service was confirmed recovered at 12:47 PM CT on July 9. ## What We Are Doing About This To prevent this from happening again and to catch similar issues faster, we are taking the following actions: * **Fixing the Underlying Incompatibility:** We are updating the specific configuration affected by this change so the original update can be safely re-applied without reintroducing the issue. * **Broader Configuration Audit:** We are reviewing other configurations that could be affected by the same underlying change to ensure no similar failures exist elsewhere in the platform. * **Improved Monitoring:** We are adding monitoring that validates response content — not just whether a request completed — so that data-omission failures trigger alerts rather than going undetected. * **Expanded Test Coverage:** We are adding tests that verify the actual content of mapped responses, so this class of failure is caught during development before it reaches production.

The RedoxEngine message/transmission logs have fallen behind starting at 10:15 AM Central. Our team is currently investigating the cause and working on a solution. Overall message traffic is unaffected. If you have any additional questions, please notify us at [email protected].

We are continuing to investigate this issue.

We have identified the cause for the log visibility issues and are actively working on a fix. We are seeing the number of messages with delays decrease. We will update the status of this incident when said fix has been deployed.

A fix has been implemented and we are monitoring the results.

This incident has been resolved.

At approximately 4:45pm CT, Redox became aware of an issue impacting VPN tunnels. Impacted tunnels will see errors in attempts to deliver messages to destinations. What this means: you may experience a delay before receiving traffic from Redox. We are actively looking into this issue and will have a fix available as soon as possible. If you are concerned that your logs have been impacted or have any additional questions, please reach out to [email protected].

The issue has been identified and a fix is being implemented.

A fix has been implemented and we are monitoring the results. VPN message traffic is resuming as normal.

Messages are now processing normally. While we were unable to send VPN traffic we queued messages and now that VPNs are up and stable we are sending out all those messages. If you have any additional questions or are not seeing expected logs, please reach out to [email protected]

### Summary Between **4:20 PM CT and 5:38 PM CT** on May 6, 2026, some customers experienced an interruption in outbound traffic for VPN-based connections. During this window, data egressing via the network proxy service failed to authorize, resulting in unsuccessful transmissions for MLLP traffic. Service was fully restored by **6:10 PM CT** after configuration settings were corrected and gateway services were restarted. ### What Happened The service interruption was triggered during a planned migration of our network configuration management system. * **Summary of Causes:** During the transition to a new configuration framework, an authentication credential used by the network proxy service was incorrectly identified as unused and subsequently removed. While pre-deployment testing appeared successful, the removal of this credential caused authorization failures once applied across the broader production environment. * **Summary of Impact:** VPN-based connections for outbound data experienced a loss of egress connectivity. They would get an error on the first outbound message on a subscription to a VPN destination. Transmissions were queued during the issue and sent after the problem was resolved; no transmissions were dropped. ### What we are doing about this: * **Enhanced Testing Protocols:** We are developing new end-to-end tests specifically for outbound MLLP traffic to ensure that connectivity is verified across all gateways during configuration changes. * **Improved Service Observability:** We are updating our alerting logic for our proxy services to trigger more immediate notifications if egress traffic volume drops below expected thresholds, ensuring faster detection.