Cockroach Labs incident

Degraded connectivity between clusters and external services.

Minor Resolved View vendor source →

Cockroach Labs experienced a minor incident on January 24, 2024 affecting CockroachDB Basic/Standard and CockroachDB Advanced, lasting 11h 13m. The incident has been resolved; the full update timeline is below.

Started
Jan 24, 2024, 04:51 AM UTC
Resolved
Jan 24, 2024, 04:05 PM UTC
Duration
11h 13m
Detected by Pingoru
Jan 24, 2024, 04:51 AM UTC

Affected components

CockroachDB Basic/StandardCockroachDB Advanced

Update timeline

  1. identified Jan 24, 2024, 04:51 AM UTC

    A subset of clusters are unable to reach external services, affecting features like CMEK, log export, and metrics export on impacted clusters. The clusters themselves remain available. We're actively working on a mitigation and will post updates until the issue is resolved.

  2. identified Jan 24, 2024, 07:00 AM UTC

    We are continuing to work on a fix for this issue.

  3. identified Jan 24, 2024, 10:56 AM UTC

    Fix has been identified and deployment to impacted clusters in progress. All CockroachDB Serverless environments are mitigated now.

  4. resolved Jan 24, 2024, 04:05 PM UTC

    Fix deployed on all affected clusters. CockroachDB Dedicated is mitigated as well. All services back to normal and incident closed.