Cognite Outage History

Started Oct 16, 2026, 11:50 AM UTC· resolved Oct 16, 2026, 02:06 PM UTC

1. investigating Oct 16, 2026, 11:50 AM UTC

   We are investigating reports of service degradation and disruption affecting the 3D service in the az-tyo-002 cluster. 3D models and related API endpoints may be unavailable. Our engineering team is currently investigating the issue

2. identified Oct 16, 2026, 12:05 PM UTC

   The 3D service in the az-tyo-002 cluster is experiencing service disruptions due to repeated service restarts during startup health checks. The engineering team has identified the mitigation and is currently deploying an adjustment to the startup probe delays to stabilize the pods and restore service availability

3. monitoring Oct 16, 2026, 12:34 PM UTC

   The fix increased the health probe startup delay to 60 seconds. The 3D service in az-tyo-002 is back online, and pods appear to have stopped restarting. The engineering team is now Monitoring and checking the stability while awaiting customer confirmation.

4. resolved Oct 16, 2026, 02:06 PM UTC

   The issue affecting 3D services in az-tyo-002 has been fixed by increasing the health probe startup delay. Service has been restored, and is now resolved.

Read the full incident report →

Started Oct 14, 2026, 03:56 PM UTC· resolved Oct 14, 2026, 11:39 PM UTC

1. investigating Oct 14, 2026, 03:56 PM UTC

   We are investigating elevated errors affecting Atlas AI agent chat across multiple regions. Users may experience failed chat requests. Initial findings suggest upstream timeouts from Azure OpenAI in the East US 2 region may be contributing to the issue, including requests originating from other regions. We are continuing to investigate the cause and impact.

2. resolved Oct 14, 2026, 11:39 PM UTC

   Upstream provider Microsoft has confirmed that the underlying service-side timeout issues in Azure OpenAI (East US 2) are fully resolved. Customer-facing mitigation remains in place and all Atlas AI agent chat metrics, traffic and availability have returned to normal operating baselines across all clusters.

Read the full incident report →

Started Oct 08, 2026, 11:49 AM UTC· resolved Oct 08, 2026, 12:06 PM UTC

1. identified Oct 08, 2026, 11:49 AM UTC

   We experienced an issue with the Time Series API that caused temporary service disruptions across select clusters: • az-arn-001: Time series datapoint ingestion (ingesting and deleting data points) experienced HTTP 503 errors and unavailability between 13:49 and 14:02 CEST. • az-sin-sp-001: Time series metadata endpoints (listing, searching, and creating classic time series) experienced a brief period of errors between 14:02 and 14:05 CEST. Datapoint ingestion remained operational.

2. monitoring Oct 08, 2026, 12:04 PM UTC

   Our engineering team identified the regression and completed a rollback to restore service. All services have recovered and are operating normally. We are actively monitoring the systems to ensure ongoing stability

3. resolved Oct 08, 2026, 12:06 PM UTC

   All services have recovered and are operating normally.

Read the full incident report →

Started Oct 02, 2026, 02:15 PM UTC· resolved Oct 06, 2026, 05:33 PM UTC

1. investigating Oct 02, 2026, 02:15 PM UTC

   We are currently investigating reports of increased latency and degraded performance affecting services in the az-phx-001 cluster (Azure West US 3). Our engineering teams are actively investigating the underlying storage performance and working on mitigating the issue. Next update will be provided within as soon as more information becomes available.

2. monitoring Oct 06, 2026, 02:48 PM UTC

   A fix has been deployed by migrating the affected database storage pods to a new node pool in the az-phx-001 cluster. Cluster latency and performance have returned to normal levels, and our team is actively monitoring the service to ensure stability.

3. resolved Oct 06, 2026, 05:33 PM UTC

   The issue causing increased latency and degraded performance in the az-phx-001 cluster has been resolved. All affected services and API response times are back to normal operation

Read the full incident report →

Started Oct 02, 2026, 01:54 PM UTC· resolved Oct 07, 2026, 09:53 AM UTC

1. investigating Oct 02, 2026, 01:54 PM UTC

   We are currently investigating an issue causing degraded performance for Transformations and Workflows in the westeurope-1 cluster. Customer Impact: • Delayed start times for scheduled transformation jobs. • Extended overall runtime for workflows and chained data pipelines. • Jobs are continuing to process and complete, but latency is significantly higher than baseline. Current Status & Actions: Engineering teams are actively investigating resource contention and applying mitigations across the worker pools to restore normal throughput. We will provide the next status update soon as further details are available.

2. monitoring Oct 06, 2026, 03:40 PM UTC

   The issue is fixed. We are continuing to monitor this.

3. resolved Oct 07, 2026, 09:53 AM UTC

   The workflow/transformation performance issue has been resolved. Increasing the project’s task-pool priority and unscheduling bootcamp transformations significantly improved performance, reducing average transformation start delays.

Read the full incident report →

Started Sep 29, 2026, 03:52 PM UTC· resolved Sep 30, 2026, 04:16 PM UTC

1. investigating Sep 29, 2026, 03:52 PM UTC

   https://azure.status.microsoft/en-us/status Azure has reported degraded performance for certain services. See the above link for more specific details.

2. monitoring Sep 29, 2026, 07:37 PM UTC

   Microsoft Azure has reported that the underlying platform issue in the swedencentral region has now been mitigated. Our internal metrics indicate that service health and model response times are recovering. We are actively observing system performance and endpoint latencies across all affected clusters to verify sustained stability. Further updates will be shared as soon as full recovery is confirmed.

3. resolved Sep 30, 2026, 04:16 PM UTC

   The issue causing slow response times for Azure OpenAI models in the EU Datazone has been resolved.

Read the full incident report →

Started Sep 28, 2026, 11:18 AM UTC· resolved Sep 30, 2026, 11:15 AM UTC

1. investigating Sep 28, 2026, 11:18 AM UTC

   We are currently investigating an issue where scheduled calculations created in Charts are delayed or failing to write new data points. Our engineering team is actively investigating the root cause. Further updates will be provided as soon as they become available.

2. investigating Sep 28, 2026, 12:28 PM UTC

   Current Status Update — INC-3602 (Scheduled Calculations in Charts) Impact: Scheduled calculation executions in Charts are failing or significantly delayed, preventing new data points from being written. The issue currently appears isolated to cluster az-tyo-gp-001 (verified working normally on aw-tyo-001). Current Findings: • High CPU utilization (100–1000%) observed on backend workers, causing jobs to fall hours behind schedule. • Extensive timeout errors and ~236 failed jobs recorded across the cluster over the weekend. • Newly created and existing scheduled calculations are affected. Current Actions & Next Steps: • Engineering is investigating the root cause of the CPU spike and timeout failures. • Evaluating remediation options to relieve backend load and process the backlog of calculations.

3. monitoring Sep 28, 2026, 03:57 PM UTC

   Monitoring - Fixes to scale worker capacity and improve service autoscaling have been deployed. We are observing scheduled calculations resuming execution. Due to the backlog accumulated during the degradation, jobs are currently catching up and may experience processing delays over the next few hours. We are actively monitoring the service until all scheduled calculations are fully up to date.

4. monitoring Sep 30, 2026, 09:47 AM UTC

   Monitoring – Scheduled calculations have resumed and data points are updating as expected. Our team is actively monitoring the service and verifying data flow stability with affected users before marking the incident as fully resolved.

5. resolved Sep 30, 2026, 11:15 AM UTC

   All scheduled calculations have resumed and data point processing is fully operational. We have verified full recovery and confirmed normal service performance.

Read the full incident report →

Started Sep 23, 2026, 05:45 PM UTC· resolved Sep 24, 2026, 12:08 PM UTC

1. investigating Sep 23, 2026, 05:45 PM UTC

   We are currently experiencing performance degradation affecting search and data modeling indexing services in the westeurope-1 region. Impact: Search and aggregate queries may experience elevated latency, and newly created instances or index updates may take longer to appear in search results. Core data storage is unaffected. Our engineering team is actively investigating the issue and working on mitigation steps to stabilize the service. We will provide updates as more information becomes available

2. monitoring Sep 23, 2026, 07:10 PM UTC

   • Problem: The Propertygraph Elasticsearch cluster in westeurope-1 experienced degradation, temporarily entering yellow and red states. • Impact: Customers with projects in westeurope-1 may experience delays in newly created instances appearing in search results, along with elevated latency for search and aggregate queries affecting data modeling indexing (datamodelstorage) Core data storage is unaffected. • Root Cause: A syncer update triggered concurrent restarts across syncer deployments simultaneously, overwhelming the search master nodes with high-volume reads. In addition, an search pod encountered a PVC mount failure during rolling updates. • Status & Mitigation: Mitigation steps have been applied. Our engineering team scaled down the syncers to allow the cluster to recover to a healthy green state, subsequently re-enabled the synchronizers in stages, and resolved the PVC volume mount issue. The cluster is stabilizing, and our team is actively monitoring system performance and recovery.

3. resolved Sep 24, 2026, 12:08 PM UTC

   The performance degradation affecting Data Modeling search and indexing in westeurope-1 has been fully resolved. Syncer capacity has been restored, cluster health metrics have returned to normal, and all indexing backlogs have cleared. Search, aggregation, and data modeling queries are functioning as expected. We are continuing to monitor the services to ensure ongoing stability.

Read the full incident report →

Started Sep 22, 2026, 07:57 PM UTC· resolved Sep 23, 2026, 09:05 AM UTC

1. investigating Sep 22, 2026, 07:57 PM UTC

   We are currently investigating performance degradation and high latency affecting search and data modeling services in the westeurope-1 region. Impact: Search and aggregate requests may experience high latency or temporary failures. Newly created instances or index updates may take longer than expected to appear. Status: Our engineering team is currently scaling down background syncing processes to reduce load and restore cluster stability. Further updates will be posted as soon as they are available.

2. monitoring Sep 22, 2026, 10:43 PM UTC

   The Elasticsearch cluster in westeurope-1 became overloaded and entered a red state. This degraded search and data-modeling indexing. search and aggregate requests could be slow or fail, while new instances and index changes could take longer to appear. Root cause: A rollout restart of pg3 overloaded the Elasticsearch cluster. Fixing status: The incident is now in Monitoring. Engineering team paused some background synchronizers to reduce load, allowed Elasticsearch to stabilize, and then started re-enabling the synchronizers. The fix was marked as applied.

3. resolved Sep 23, 2026, 09:05 AM UTC

   Service has been restored. The Elasticsearch cluster in westeurope-1 has stabilized, and search, aggregation, and data-modeling indexing are operating normally.

Read the full incident report →

Started Sep 22, 2026, 10:08 AM UTC· resolved Sep 24, 2026, 12:16 PM UTC

1. investigating Sep 22, 2026, 10:08 AM UTC

   We have identified an issue affecting search and data modeling indexing in the westeurope-1 region. Users and automated pipelines in this cluster may experience delays in newly ingested data or schema updates appearing in search and data modeling views. Querying existing indexed data and other CDF services remain operational. Our engineering team has implemented traffic-shedding and resource adjustments to stabilize the underlying cluster, and indexing throughput is beginning to recover. We are closely monitoring cluster health and will provide our next update soon,

2. monitoring Sep 22, 2026, 10:31 AM UTC

   The underlying cluster in westeurope-1 has stabilized, and search and aggregate queries are functioning normally. Indexing pipelines are currently processing the remaining backlog, so some users may still see slight delays in newly ingested data appearing. We are monitoring the recovery closely.

3. resolved Sep 24, 2026, 12:16 PM UTC

   The performance degradation affecting Data Modeling search and indexing in westeurope-1 has been fully resolved. Syncer capacity has been restored, cluster health metrics have returned to normal, and all indexing backlogs have cleared. Search, aggregation, and data modeling queries are functioning as expected. We are continuing to monitor the services to ensure ongoing stability.

Read the full incident report →

Started Sep 11, 2026, 07:20 AM UTC· resolved Sep 18, 2026, 07:18 PM UTC

1. investigating Sep 11, 2026, 07:20 AM UTC

   We are currently investigating an issue impacting the Data Modeling service (datamodelstorage) in the az-arn-001 cluster. A large spike in traffic against Data Modeling endpoints has led to an elevated rate of 503 errors (affecting roughly 3% of requests) in this cluster. Users and services interacting with Data Modeling endpoints may experience intermittent request failures. Our engineering team is actively investigating the cause of the errors and working on mitigation steps. Further updates will be provided as more information becomes available

2. resolved Sep 18, 2026, 07:18 PM UTC

   Increased error rates were found to be confined to a particular client configuration with excessive retries, and were mitigated. Overall cluster availability remained nominal.

Read the full incident report →

Started Sep 11, 2026, 02:44 AM UTC· resolved Oct 01, 2026, 07:11 AM UTC

1. investigating Sep 11, 2026, 02:44 AM UTC

   We are currently investigating an issue causing elevated 503 error rates and service degradation for Data Modeling services within the az-tyo-gp-001 cluster. Users and applications in this cluster may experience intermittent errors or failures when accessing data modeling endpoints and related features. Our engineering team is actively investigating the root cause to restore normal operations. Further updates will be shared as more information becomes available.

2. monitoring Sep 11, 2026, 03:09 AM UTC

   The engineering team has allocated additional memory resources to the affected services, and normal performance has been restored with error rates dropping to zero. We are continuing to closely monitor the system to ensure ongoing stability. Further updates will be provided once the incident is fully resolved.

3. resolved Oct 01, 2026, 07:11 AM UTC

   This incident has been resolved. Data modeling services in az-tyo-gp-001 have fully recovered and error rates have returned to normal. All systems are operating as expected.

Read the full incident report →

Started Sep 06, 2026, 11:34 AM UTC· resolved Sep 06, 2026, 02:51 PM UTC

1. investigating Sep 06, 2026, 11:34 AM UTC

   The engineering team is investigating and fixing issues with the Data Modelling search and aggregate APIs in the westeurope-1 cluster. Customer impact includes increased error rates on DM APIs and in the data exploration app in Fusion, and delays before new data is available in search results.

2. investigating Sep 06, 2026, 02:51 PM UTC

   This incident has been resolved. The issue causing HTTP 500/503 errors when searching or aggregating Data Models in westeurope-1 has been mitigated, and service performance has fully recovered. All systems are operating normally.

3. resolved Sep 06, 2026, 02:51 PM UTC

   This incident has been resolved. The issue causing HTTP 500/503 errors when searching or aggregating Data Models in westeurope-1 has been mitigated, and service performance has fully recovered. All systems are operating normally.

Read the full incident report →

Started Sep 06, 2026, 05:18 AM UTC· resolved Sep 06, 2026, 02:54 PM UTC

1. investigating Sep 06, 2026, 05:18 AM UTC

   We experienced elevated 5XX error rates affecting the Timeseries and Sequences APIs on cluster az-pnq-gp-001 between approximately 07:18 and 07:45 CEST. Services have recovered and all deployments are healthy, with no indication of data loss. The engineering team is actively monitoring the service to ensure continued stability while completing the investigation into the underlying cause.

2. resolved Sep 06, 2026, 02:54 PM UTC

   Services are fully recovered and operating normally. The elevated 5XX error rates affecting the Timeseries and Sequences APIs on cluster az-pnq-gp-001 between approximately 07:18 and 07:45 CEST on September 6, 2026 were resolved following auto-healing of the FoundationDB operator pod instability. Monitoring confirms all deployments remain healthy and stable, with no data loss incurred.

Read the full incident report →

Started Sep 03, 2026, 02:51 PM UTC· resolved Sep 03, 2026, 03:05 PM UTC

1. monitoring Sep 03, 2026, 02:51 PM UTC

   Identified - We are investigating an issue affecting user logins across Cognite Data Fusion (Fusion), Infield, and Maintain. Some users may experience login failures displaying the error: "An HTTP line is larger than 4096 bytes". A recent identity service update has been identified as the likely cause, and our engineering team is actively deploying a rollback to restore full login functionality. We will provide another update soon as the rollback is completed.

2. resolved Sep 03, 2026, 03:05 PM UTC

   The issue causing login failures across Cognite Data Fusion, InField, and Maintain has been resolved. The underlying deployment change was rolled back, and normal service has been fully restored across all clusters.

Read the full incident report →

Started Sep 01, 2026, 12:05 PM UTC· resolved Sep 01, 2026, 06:32 PM UTC

1. investigating Sep 01, 2026, 12:05 PM UTC

   Summary: We are currently investigating an issue impacting Data Model (DM) Search and aggregation queries in the westeurope-1 region. Customer Impact: Requests to search or aggregate data models in westeurope-1 may fail with HTTP 503 errors or return incomplete/delayed results. Core data storage and other regions are operating normally. Current Actions: Our engineering team is actively working on cluster recovery and load mitigation, including throttling background sync operations to restore search performance. Next Update: We are monitoring recovery progress and will provide another update within 60 minutes or as soon as significant updates are available.

2. monitoring Sep 01, 2026, 06:30 PM UTC

   We have implemented mitigation steps for the issue impacting Data Model (DM) Search and aggregation queries in the westeurope-1 region. The service has stabilized, and we are currently monitoring the system to ensure full recovery. Search and aggregation requests for data models in westeurope-1 should now complete successfully without HTTP 503 errors. Customers may still see slight delays in data freshness as background sync operations catch up, but normal functionality has been largely restored. Core data storage and other regions continue to operate normally. Our engineering team has successfully stabilized the underlying Elasticsearch cluster and carefully re-enabled background sync operations in controlled stages. We will continue to closely monitor cluster health and performance metrics to ensure the service remains stable.

3. resolved Sep 01, 2026, 06:32 PM UTC

   Summary: We experienced an issue affecting the Property Graph search service in the West Europe region. The Elasticsearch cluster became unstable after receiving unusually large synchronization requests, which caused circuit-breaker errors. Steps taken: We stabilized the Elasticsearch cluster, then re-enabled background synchronization in controlled stages while monitoring cluster health, CPU usage, and indexing rates. Resolved status: The service has stabilized and normal functionality has been restored. Background synchronization has resumed, and we will continue monitoring to ensure it remains healthy.

Read the full incident report →

Started Aug 24, 2026, 01:52 PM UTC· resolved Aug 31, 2026, 12:07 PM UTC

1. investigating Aug 24, 2026, 01:52 PM UTC

   We are investigating issues with CDF-hosted Streamlit apps, where apps get the error ModuleNotFoundError: No module named 'ssl' We are investigating the issue. The issue can affect all users of Streamlit apps regardless of which CDF clusters they are hosted in. In the meantime, explicitly adding the "ssl" module to your app's Python requirements can fix the immediate problem and is worth trying as a workaround. Note: CDF-hosted Streamlit apps are in public preview: https://docs.cognite.com/cdf/streamlit/index#streamlit-apps and are not yet in production: https://docs.cognite.com/cdf/product_feature_status#public-preview.

2. identified Aug 25, 2026, 07:44 AM UTC

   The issue leading to the ModuleNotFoundError: No module named 'ssl' error in Streamlit apps has been identified as a change in a third party library, specifically the Microsoft Authentication Library for Python (MSAL). MSAL is commonly used in conjuction with the Cognite SDK to handle authentication. MSAL version 1.38, released on 2026-08-24, references the ssl module without declaring a dependency on it. This works on ordinary CPython installations because that environment includes the ssl module by default. However, Streamlit in the browser uses the Pyodide runtime environment. Pyodide does not include the ssl module by default. This leads to failures when the latest MSAL library is used in Streamlit apps. The engineering team is currently adding a workaround to the Cognite Streamlit runtime environment setup to ensure that the ssl module is always present.

3. monitoring Aug 27, 2026, 11:06 AM UTC

   We have successfully resolved the underlying deployment issue. Our engineering team corrected the production deployment configuration, ensuring it now points to the correct Firebase project and hosting site. The fix has been verified across multiple production clusters and confirmed to be working as expected. If you had previously applied the temporary workaround of adding ssl to your application's INSTALLED PACKAGES, no further action is needed this entry can remain in place and will not interfere with the platform fix.

4. resolved Aug 31, 2026, 12:07 PM UTC

   The issue got fixed on 27th August. All Apps are now up and running.

Read the full incident report →

Started Aug 19, 2026, 04:52 PM UTC· resolved Aug 20, 2026, 09:06 AM UTC

1. investigating Aug 19, 2026, 04:52 PM UTC

   We are aware of a service outage affecting the CDF cluster westeurope-1. The issue has severe impact on some but not all of the projects in cluster and are caused by a database server outage after a routine maintenance operation. Cognite's engineering team is actively working with our cloud provider to resolve the issue.

2. monitoring Aug 19, 2026, 05:46 PM UTC

   The database issue has been resolved and services are recovering. We are monitoring the situation.

3. resolved Aug 20, 2026, 09:06 AM UTC

   Issue is fixed, and no further degradations were observed.

Read the full incident report →

Started Aug 13, 2026, 05:25 PM UTC· resolved Aug 14, 2026, 12:44 PM UTC

1. investigating Aug 13, 2026, 05:25 PM UTC

   We are currently investigating an elevated rate of 5xx errors affecting the Functions API. The issue appears to be caused by an upstream dependency disruption (Azure Key Vault RBAC loading timeout) in the Central US region, rather than a direct cluster-level outage. The connection rates are already stabilizing, and we are monitoring the recovery.

2. monitoring Aug 13, 2026, 05:40 PM UTC

   Functions requests on az-dsm-psaas-001 returned 500s for a temporary period between 19:25 and 19:40 CEST. The immediate impact was resolved at 19:40 CET / 17:40 UTC, and we are now monitoring the clusters to ensure continued stability.

3. resolved Aug 14, 2026, 12:44 PM UTC

   This incident has been resolved. It is operating normally. We apologize for any inconvenience and thank you for your patience.

Read the full incident report →

Started Aug 11, 2026, 10:14 AM UTC· resolved Aug 11, 2026, 01:41 PM UTC

1. investigating Aug 11, 2026, 10:14 AM UTC

   We are currently investigating an issue causing the Cogshop service to be unavailable. Our engineering team has identified the root cause and is actively working on a fix. We will provide further updates as soon as more information is available.

2. monitoring Aug 11, 2026, 01:19 PM UTC

   We identified the issue as a missing TLS certificate in the wildcard-cognite-ai-tls-patch secret. Access to the powerops-certmgr-vault key vault was updated to allow regeneration/import of the certificate, and we can now see the certificate has been recreated. The Cogshop endpoint is responding again: shop-api.az-inso-powerops.cognite.ai/docs: https://shop-api.az-inso-powerops.cognite.ai/docs. We are continuing to monitor the service closely to confirm it remains stable before marking this incident as resolved.

3. resolved Aug 11, 2026, 01:41 PM UTC

   This incident has been resolved. Cogshop service is up and operating normally. We apologize for any inconvenience and thank you for your patience.

Read the full incident report →

Started Jul 22, 2026, 02:42 PM UTC· resolved Jul 22, 2026, 03:17 PM UTC

1. investigating Jul 22, 2026, 02:42 PM UTC

   We are currently investigating an issue affecting the InField application where a UI license watermark ("MUI X Missing License Key") is displayed. Impact: This is a minor visual UI issue and does not impact functionality, data integrity, or core user workflows. Our engineering teams are actively investigating the root cause and working on a resolution.

2. resolved Jul 22, 2026, 03:17 PM UTC

   The license key configuration for Cognite InField has been updated, and the missing license key watermark is now fully resolved across all affected environments. All services are operating normally. Thank you for your patience!

Read the full incident report →

Started Jul 13, 2026, 11:47 AM UTC· resolved Jul 20, 2026, 02:24 PM UTC

1. investigating Jul 13, 2026, 11:47 AM UTC

   We are currently investigating an issue affecting Cognite Maintain for OMV Romania Asset Oil in Production. Users with edit rights are unable to update an activity date, create duplicates, or run deferment calculations. Browsing and read actions continue to work normally. All other customers and clusters are unaffected. Our engineering team is actively investigating the root cause and will provide further updates as soon as possible

2. resolved Jul 20, 2026, 02:24 PM UTC

   We identified and fixed an issue affecting Cognite Maintain for OMV Petrom Romania (Asset Oil) where users with edit access could not update activity dates, duplicate activities, or run deferment calculations in production. Read-only access was not affected. The fix has been applied and the service is operating normally. Root cause: Expired application client secrets on the backend authentication path caused write operations to return 401 Unauthorized in production only. Resolution: New client secrets were generated and applied. The service is operating normally as of Jul 15, 2026, 1:10pm UTC.

Read the full incident report →

Started Jul 08, 2026, 03:53 PM UTC· resolved Jul 09, 2026, 03:43 PM UTC

1. investigating Jul 08, 2026, 03:53 PM UTC

   We have identified a UI interuption affecting a subset of projects in the cluster that is related to a MUI X Missing license key overlay in the Search UI. The team is currently investigating and a fix is on the way.

2. identified Jul 08, 2026, 04:15 PM UTC

   The team has identified the issue and a fix is currently on the way to contain the issue.

3. monitoring Jul 08, 2026, 05:31 PM UTC

   The team has deployed the revert for the change linked to the MUI X Missing license key overlay, with that the incident is now contained where we will be monitoring the applied changes to see the stability of the fix.

4. resolved Jul 09, 2026, 03:43 PM UTC

   The MUI X Missing license key warning is no longer showing in production, and Search/Data Exploration is behaving normally.

Read the full incident report →

Started Jun 25, 2026, 10:51 AM UTC· resolved Jun 25, 2026, 04:25 PM UTC

1. investigating Jun 25, 2026, 10:51 AM UTC

   An issue has been identified affecting Custom Apps for some users, where the page may display "No apps available" despite apps being present. The root cause is currently under investigation, and updates will be provided as more information becomes available.

2. monitoring Jun 25, 2026, 11:14 AM UTC

   The engineering team has identified the root cause and deployed a fix. The Custom Apps view has been confirmed to be functioning normally. The service is being monitored to ensure continued stability.

3. resolved Jun 25, 2026, 04:25 PM UTC

   The engineering team has confirmed that the Custom Apps page is fully operational for all users following the fix. The incident has been resolved and is now closed.

Read the full incident report →

Started Jun 09, 2026, 12:30 PM UTC· resolved Jun 11, 2026, 06:59 AM UTC

1. identified Jun 09, 2026, 12:30 PM UTC

   We have identified a significant delay in the propagation of the isUploaded flag for files uploaded in the az-eastus-1 cluster. Files may take over 2 hours to be marked as uploaded after a successful upload. Root cause identified: At least one project in the az-eastus-1 cluster is being rate limited by the DMS (Data Modeling Service), which is causing the file storage subscriber to back up and delay processing of upload notifications. Our engineering team has applied a fix to increase the DMS write concurrency limit for the affected project, and the change has been deployed. We are actively monitoring the situation to confirm recovery. We will provide further updates as the situation progresses.

2. resolved Jun 11, 2026, 06:59 AM UTC

   File upload propagation delays (isUploaded flag) in the az-eastus-1 cluster have been fully resolved. The root cause was an Azure Event Grid outage in the East US region (05:00 UTC June 9 – 06:30 UTC June 10, 2026), which prevented blob storage notifications from reaching our internal file processing pipeline. Propagation lag has now stabilized at normal, near-zero levels. All affected workflows, including CDF Function deployments and file-dependent processes, should be fully operational. We will continue to monitor system metrics to ensure stability.

Read the full incident report →