Cognite incident

Users experiencing HTTP 500/503 errors when searching or aggregating Data Models

Resolved after 9 hours 40 minutes, affecting Cognite Data Fusion API and Cognite Data Fusion API and 1 more, on September 4, 2026.

resolved Sep 04, 2026, 05:58 PM UTC

• Problem: The production propertygraph-search Elasticsearch cluster in westeurope-1 went red for roughly 10 minutes. • Root cause: Propertygraph syncer activity and cluster topology created excessive master-coordination overhead. PVC mount failures were also observed on Elasticsearch master-data pods, but were not confirmed as the original cause. • Resolution: Engineering Team scaled down the propertygraph syncers to recover cluster health, then restored every propertygraph-sync-v3 deployment to one replica in stages. They also reduced node/master-coordination overhead and reconfigured all 15 master-data nodes with 19 GB heaps; the cluster reached zero unassigned shards and was considered stable.

Started
Sep 04, 2026, 08:17 AM UTC
Resolved
Sep 04, 2026, 05:58 PM UTC
Duration
9h 40m
Detected by Pingoru
Sep 04, 2026, 08:17 AM UTC

Affected components

Cognite Data Fusion APICognite Data Fusion APICognite Data Fusion APICognite Data Fusion APICognite Data Fusion APICognite Data Fusion APICognite Data Fusion APICognite Data Fusion APICognite Data Fusion APICognite Data Fusion API

Update timeline

  1. monitoring Sep 04, 2026, 08:17 AM UTC

    The engineering team is investigating and fixing issues with the Data Modelling search and aggregate APIs in the westeurope-1 cluster. Customer impact includes increased error rates on DM APIs and in the data exploration app in Fusion, and delays before new data is available in search results.

  2. monitoring Sep 04, 2026, 08:18 AM UTC

    The engineering team is investigating and fixing issues with the Data Modelling search and aggregate APIs in the westeurope-1 cluster. Customer impact includes increased error rates on DM APIs and in the data exploration app in Fusion, and delays before new data is available in search results.

  3. monitoring Sep 04, 2026, 10:29 AM UTC

    The engineering team is investigating and fixing issues with the Data Modelling search and aggregate APIs in the westeurope-1 cluster. Customer impact includes increased error rates on DM APIs and in the data exploration app in Fusion, and delays before new data is available in search results.

  4. resolved Sep 04, 2026, 05:58 PM UTC

    • Problem: The production propertygraph-search Elasticsearch cluster in westeurope-1 went red for roughly 10 minutes. • Root cause: Propertygraph syncer activity and cluster topology created excessive master-coordination overhead. PVC mount failures were also observed on Elasticsearch master-data pods, but were not confirmed as the original cause. • Resolution: Engineering Team scaled down the propertygraph syncers to recover cluster health, then restored every propertygraph-sync-v3 deployment to one replica in stages. They also reduced node/master-coordination overhead and reconfigured all 15 master-data nodes with 19 GB heaps; the cluster reached zero unassigned shards and was considered stable.

Your product runs on more than Cognite

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →