Service Degradation: Extraction Failures and Delays

Update timeline

1. investigating Apr 20, 2025, 10:44 AM UTC

   We have identified a couple of issues affecting the processing of data extraction requests on the server. You will see extractions failing randomly with error code EE0005 error code. And this will affect data collection from last 2 days and any new onboarding initiated in this period. The second problem is related to an internal request management service and message queue failure that has caused a backlog in the extraction processing pipeline. Team is working on resolution of these issues. Once resolved all errors will be retried by the system and no action will be required from users The issue impacts only extraction processing. The UI remains fully functional and is not affected. Our team is actively working on resolving these issues and clearing the backlog as quickly as possible. We appreciate your patience and understanding while we work towards a resolution. Please follow this status page for regular updates on the issue

2. monitoring Apr 21, 2025, 06:08 PM UTC

   A fix has been implemented and we are monitoring the results.

3. investigating Apr 22, 2025, 09:27 AM UTC

   We have been monitoring the progress for the last 8 hours and the system is should be able to process the backlog in the next 12 hours. The extractions that failed when the issue first happened last week have also been queued for retry and will be processed soon

4. resolved May 22, 2025, 04:54 AM UTC

   the issue has not re-surfaced and all failures from the time of the original incident have been retried.