Dashboard transaction processing delays

Update timeline

1. identified Feb 03, 2026, 12:41 PM UTC

   We have identified a lag in transaction processing for the Sardine dashboard and are actively working on resolution.

2. resolved Feb 03, 2026, 04:29 PM UTC

   This incident has been resolved.

3. postmortem Feb 09, 2026, 05:34 AM UTC

   ## Overview Between January 31 and February 3, 2026, customers using Sardine's issuing transaction API in the EU region experienced delay in data visibility. New transaction data was not appearing in the Dashboard during this period. ## Impact **Duration:** January 31 - February 3, 2026 **Customer Impact:** * Users on EU instance saw missing issuing transaction during Jan 31th - Feb 4th and errors on Alert details page associated to the missing transaction data. **Data Integrity:** All transaction data was successfully retained and processed. No data loss occurred. ## What Happened Our transaction processing infrastructure in the EU region experienced degraded performance due to database connectivity issues. This caused a backlog in our data pipeline, preventing transaction information from being displayed in customer dashboards in real-time. The issue was triggered by increased load on our storage infrastructure combined with inefficiencies in our data access patterns. As the system attempted to retry failed operations, the backlog continued to grow, eventually requiring manual intervention to resolve. ## Resolution Our engineering team implemented several fixes to restore service: 1. Added improved error handling to prevent cascading failures 2. Optimized database access patterns to reduce load 3. Scaled infrastructure resources to handle the backlog 4. Successfully processed all delayed transactions with no data loss The backlog was fully cleared on February 3, 2026, and all transaction data became available in customer dashboards. ## What We're Doing to Prevent This We've identified several improvements to prevent similar incidents: * **Enhanced Monitoring:** Implementing additional alerting on data pipeline health and database performance metrics * **Improved Architecture:** Optimizing our data access patterns to reduce infrastructure load * **Better Resilience:** Adding improved error handling with appropriate deadletter handling * **Infrastructure Parity:** Ensuring consistent configuration and capacity across all deployment environments We sincerely apologize for the disruption this caused. If you have any questions or concerns about this incident, please contact your account team or reach out to [[email protected]](mailto:[email protected]).