Datacake incident

Performance degradation

Minor Resolved View vendor source →

Datacake experienced a minor incident on December 8, 2025, lasting —. The incident has been resolved; the full update timeline is below.

Started
Dec 08, 2025, 11:43 AM UTC
Resolved
Dec 08, 2025, 11:43 AM UTC
Duration
Detected by Pingoru
Dec 08, 2025, 11:43 AM UTC

Update timeline

  1. resolved Dec 08, 2025, 11:43 AM UTC

    Type: Incident Duration: 51 minutes Affected Components: GraphQL API Dec 8, 11:43:42 GMT+0 - Investigating - We are investigating increased latencies with the API. Dec 8, 11:58:14 GMT+0 - Monitoring - We’ve identified the root cause as an issue with one of our database providers. The affected database has recovered, and all services are fully back online. While we continue to monitor the situation, we’re working closely with the vendor to understand what happened and to prevent future occurrences. We excuse any inconvenience caused and appreciate your patience. Dec 8, 12:34:21 GMT+0 - Resolved - All systems continue to be stable, so we are marking this incident as resolved. Our database vendor has acknowledged the issue as originating on their end and is continuing to investigate the root cause. Dec 9, 12:51:39 GMT+0 - Postmortem - On December 8, several of our services experienced a brief period of instability due to an issue originating from our database provider. The incident began at 11:42 UTC and services recovered by 11:51 UTC. **What happened** Our vendor identified the root cause as a failure in their backend node replacement service. This failure caused delays in deploying new nodes, which led to performance issues and affected tasks related to DNS updates, including node replacements. The impact was limited to databases whose nodes had recently been replaced or newly created, for example after forking or scheduled maintenance. **Resolution** The vendor has identified and fixed the underlying issue in their node replacement service. Once the affected database recovered, all Datacake services returned to normal operation and have remained stable since. **Next steps** We are continuing to work with the vendor to ensure the issue is fully understood and that safeguards are in place to prevent a recurrence.