R2Capital Outage History

This incident has been resolved. Root cause: a sporadic load spike on our database temporarily degraded API response times between 13:00 and 13:09 UTC. Total impact during this window: out of approximately 7,200 API requests, 121 (1.7%) exceeded 7 seconds or timed out, while the remaining 7,090 stayed within the expected response time range. No server errors occurred and no data was lost. Response times returned to normal at 13:09 UTC and have remained stable since. We have identified the internal process that caused the load spike and are taking measures to prevent it from recurring.

This incident has been resolved. A brief disruption in our API gateway at 23:48 UTC caused a small number of payment requests to fail for about one second. No payments were processed incorrectly, and any affected request can be safely retried. We have seen no further errors since.

The issue disrupting service availability has been resolved. A scheduled infrastructure maintenance process caused both DNS resolution pods to restart simultaneously, resulting in a brief period where applications were unable to reach external services. The DNS service was fully restored within approximately 4 minutes, and all affected services came back to normal operation automatically. We appreciate your patience.

The issue affecting service availability for http://embedded-views.r2capital.co/ has been resolved. A configuration change in the S3 bucket policy caused requests to return "Access Denied" XML errors, preventing the interface from loading. The policy has been corrected to include all necessary network ranges, and the service is now operating normally. We appreciate your patience.

The issue affecting content delivery from http://assets.r2capital.co/ has been resolved. A configuration change in the S3 origin caused requests to return 404 Not Found responses, preventing CloudFront from serving static assets. The configuration has been corrected, and asset delivery is now operating normally. We appreciate your patience.

The issue affecting access to content served from http://assets.r2capital.co/ has been resolved. The misconfigured policy on the public bucket has been corrected, and asset delivery is operating normally. We are continuing to review the incident and will publish a full timeline and root cause analysis (RCA) shortly. We appreciate your patience.

We are currently investigating an issue affecting access to content served from http://assets.r2capital.co/. At this time, assets were not served during a window that is still being determined.

Internal APIs experienced intermittent availability between 16:47 and 16:50 UTC due to infrastructure node rotation. The platform self-healed automatically. No manual intervention was required.

The service has been fully migrated to AWS CloudFront. Most of the requests were stabilized by Cloudflare at 12:21 UTC, and at 13:00 UTC we completed the full migration to CloudFront. The issue is now resolved.

An unexpected failure in the main database replica caused a service disruption of approximately 3 minutes. Logs indicate the issue originated from an internal PostgreSQL process error within the AWS RDS Aurora managed environment. The incident was on 2025-11-08: from 16:41 UTC to 16:46 UTC

Starting on May 3rd, 2025 at 12:58 UTC, our systems detected a significant drop in the number of approved applications. After investigation, we identified an issue with our contract signing provider, which was preventing contract generation. The issue was escalated to the provider and resolved on May 5th, 2025 at 12:00 UTC. We are currently monitoring to ensure the service remains fully restored. Thank you for your patience.

Between 06:00 UTC and 06:40 UTC, we performed a scheduled upgrade of our relational databases as part of our ongoing maintenance and patching plan. During this window, while our fallback mechanisms successfully routed read and write operations to multiple replicas, some HTTP requests may have experienced intermittent failures. The upgrade has been completed successfully, and all systems are operating normally. Thank you for your understanding.