Is Acceldata Data Observability Cloud Down Right Now? Live Acceldata Data Observability Cloud Status & Outages | IsDown Outage History

Started Jun 10, 2026, 08:34 AM UTC· resolved Jun 10, 2026, 08:56 AM UTC

1. investigating Jun 10, 2026, 08:34 AM UTC

   There will be a scheduled system maintenance for routine infrastructure activities. There might be a slight disruption in certain services during this period.

Read the full incident report →

Started May 27, 2026, 06:30 PM UTC· resolved May 28, 2026, 02:35 AM UTC

1. resolved May 29, 2026, 10:42 AM UTC

   We identified an issue that caused delays in the processing of platform events. As a result, some customers may have experienced delayed updates or notifications within the platform. Our team investigated the issue and implemented corrective actions to restore normal processing. Services are operating normally, and we continue to monitor the environment to ensure stability.

Read the full incident report →

Started May 25, 2026, 02:36 PM UTC· resolved May 25, 2026, 11:25 PM UTC

1. investigating May 25, 2026, 02:36 PM UTC

   We are currently observing a degraded performance in the pipeline services. The team is investigating the issue.

2. identified May 25, 2026, 05:53 PM UTC

   The issue has been identified and a fix is being implemented.

3. resolved May 25, 2026, 11:25 PM UTC

   This incident has been resolved.

Read the full incident report →

Started May 15, 2026, 04:20 PM UTC· resolved May 15, 2026, 05:30 PM UTC

1. identified May 15, 2026, 04:20 PM UTC

   We are currently observing increased traffic impacting pipeline services. Mitigation actions, including scaling measures, are being implemented to improve system stability and observability. Our team continues to monitor the situation closely.

2. resolved May 15, 2026, 05:30 PM UTC

   This incident has been resolved.

Read the full incident report →

Started May 11, 2026, 05:30 AM UTC· resolved May 11, 2026, 05:30 AM UTC

1. resolved May 12, 2026, 10:02 AM UTC

   On May 11, 2026, a particular Kafka consumer group in US Production experienced degraded processing performance caused by CPU saturation and worker thread exhaustion within the pipeline application. This resulted in increased Kafka consumer lag and delayed pipeline processing, but there was no outage, data loss, or user-facing errors. Mitigation actions included provisioning a dedicated node group with increased resources, scaling the application, and applying deployment optimisations. Service performance recovered fully following the changes and continued monitoring confirmed system stability.

Read the full incident report →

Started May 10, 2026, 08:30 AM UTC· resolved May 10, 2026, 08:45 AM UTC

1. resolved May 12, 2026, 09:45 AM UTC

   Following a production release after scheduled maintenance on May 10, 2025, the Catalog Service experienced degraded performance and delayed startup due to a stalled PostgreSQL migration holding locks on a critical database table. This impacted catalog-related operations for approximately 15 minutes. The issue was mitigated by terminating the blocking queries and restoring normal service operation. Preventive measures are being implemented to improve migration controls and reduce the risk of similar issues during future deployments.

Read the full incident report →

Started May 05, 2026, 04:58 AM UTC· resolved May 05, 2026, 10:50 AM UTC

1. resolved May 10, 2026, 03:25 AM UTC

   The admin-central service experienced temporary performance degradation due to increased load on a dependent authentication component. Mitigation steps have been applied, and the service performance has returned to normal.

Read the full incident report →

Started Dec 11, 2025, 01:41 PM UTC· resolved Dec 11, 2025, 03:25 PM UTC

1. investigating Dec 11, 2025, 01:41 PM UTC

   We are currently experiencing performance degradation affecting parts of our system. Our team has identified the issue and is actively working on a fix. Estimated time to resolution is approximately 40 minutes. We will provide further updates soon. Thank you for your patience.

2. investigating Dec 11, 2025, 03:24 PM UTC

   The database load unexpectedly spiked to 100%, causing noticeable application performance degradation. A rollout restart was performed, which successfully restored the database load to normal operating levels. We are currently investigating the underlying root cause and will continue to monitor the system closely. Operations have returned to normal, and the application is functioning as expected.

3. resolved Dec 11, 2025, 03:25 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Nov 25, 2025, 11:57 AM UTC· resolved Nov 25, 2025, 01:01 PM UTC

1. investigating Nov 25, 2025, 11:57 AM UTC

   We are currently investigating this issue.

2. resolved Nov 25, 2025, 01:01 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Nov 24, 2025, 05:33 PM UTC· resolved Nov 24, 2025, 10:47 PM UTC

1. identified Nov 24, 2025, 05:33 PM UTC

   The issue has been identified and a fix is being implemented.

2. monitoring Nov 24, 2025, 10:46 PM UTC

   A fix has been implemented and we are monitoring the results.

3. resolved Nov 24, 2025, 10:47 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Oct 01, 2025, 01:00 AM UTC· resolved Oct 01, 2025, 01:00 AM UTC

1. resolved Oct 02, 2025, 05:46 PM UTC

   Authentication and authorization APIs were down due to an AdminCentral service outage. Fix has been applied.

Read the full incident report →

Started Sep 12, 2025, 08:36 AM UTC· resolved Sep 12, 2025, 12:53 PM UTC

1. identified Sep 12, 2025, 08:36 AM UTC

   We identified that the Pendo API was causing slowness in UI page loading. We are applying fix

2. resolved Sep 12, 2025, 12:53 PM UTC

   This incident has been resolved.

Read the full incident report →

Started Sep 04, 2025, 04:26 PM UTC· resolved Sep 04, 2025, 04:26 PM UTC

1. resolved Sep 04, 2025, 04:26 PM UTC

   The Snowflake service experienced downtime due to high storage load. The issue was mitigated through remediation steps, and the service is now stable.

Read the full incident report →

Started Sep 02, 2025, 06:16 PM UTC· resolved Sep 04, 2025, 11:12 AM UTC

1. investigating Sep 02, 2025, 06:16 PM UTC

   We are currently investigating this issue.

2. resolved Sep 04, 2025, 11:12 AM UTC

   This incident has been resolved.

Read the full incident report →

Started Sep 01, 2025, 07:00 AM UTC· resolved Sep 01, 2025, 07:00 AM UTC

1. resolved Sep 01, 2025, 09:07 AM UTC

   The Catalog service experienced downtime due to high database load. We mitigated the issue by optimising the database performance and managing parallelism on long running queries Service is now stable.

Read the full incident report →

Started Jul 29, 2025, 05:00 AM UTC· resolved Jul 29, 2025, 11:00 AM UTC

1. resolved Jul 30, 2025, 08:08 AM UTC

   The Pipeline Service is currently down. as the 4.6.0 Upgrade, including the migration, is took longer than expected.

Read the full incident report →

Started Jul 15, 2025, 06:00 PM UTC· resolved Jul 15, 2025, 06:00 PM UTC

1. resolved Jul 17, 2025, 06:20 AM UTC

   Cadence/profile anomaly detection affected.

Read the full incident report →

Started Mar 27, 2025, 09:29 AM UTC· resolved Mar 27, 2025, 10:27 AM UTC

1. investigating Mar 27, 2025, 09:29 AM UTC

   We are currently investigating this issue.

2. investigating Mar 27, 2025, 09:31 AM UTC

   We are continuing to investigate this issue.

3. identified Mar 27, 2025, 09:32 AM UTC

   The fix is being implemented. ETA: 3:45 PM IST

4. monitoring Mar 27, 2025, 09:37 AM UTC

   A fix has been implemented and we are monitoring the results.

5. monitoring Mar 27, 2025, 09:47 AM UTC

   We are continuing to monitor for any further issues.

6. resolved Mar 27, 2025, 10:27 AM UTC

   This incident has been resolved.

7. postmortem Mar 28, 2025, 10:22 AM UTC

   # Root Cause Analysis ## Catalog Service Migration Failure During 4.2.0 Release **Date of Incident:** March 27, 2025 09:29 UTC **Resolved**: March 27, 2025 10:27 UTC **Duration**: 58 minutes **Severity**: P2 ## Affected Services/Functionality * Catalog Service * Data Reliability Features: * Policy Listing Page * Asset Discovery Page ‌ # Incident Summary During the 4.2.0 release deployment, a database migration failure in the Catalog Service caused partial system degradation. The issue stemmed from a schema validation mismatch between the database layer and application ORM, specically around tenant\_id eld length validation. ## Impact: * Intermittent failures in Data Reliability features * Inconsistent UI rendering * Mixed-version service communication issues ‌ # Root Cause Analysis ## Primary Cause: Schema Validation Mismatch **Database vs. ORM Inconsistency:** | **Layer** | **Field** | **Constraint** | | --- | --- | --- | | Database | tenant\_id | VARCHAR\(64\) | | Exposed ORM | tenant\_id | @Column\(length=34\) | \*The migration failed when processing a tenant\_id with 37 characters ## Technical Breakdown: 1. Database Schema: Allows 64-character tenant\_ids 2. ORM Entity: ‌ @Table\(name = "tenants"\) ‌ class Tenant : IntIdTable\(\) \{ ‌ val tenant\_id = varchar\("tenant\_id", 34\) // Constraint ‌ \} 1. Migration attempted to process existing 35-character ID 2. ORM validation rejected the value before DB interaction The migration failed when encountering a tenant\_id exceeding 34 characters. ## Failure Chain 1. Migration script aborted due to ORM validation 2. Catalog Service pods failed initialization 3. Kubernetes maintained previous version pods 4. Version mismatch caused API inconsistencies ‌ # Detailed Incident Timeline ## Investigation Phase | **Time \(UTC\)** | **Duration** | **Action** | | --- | --- | --- | | 09:29 | | Deployment failure detected | | 09:32 | 3 mins | Team mobilisation | | 09:37 | 5 mins | Root cause identified | ## Hotx Development | **Time \(UTC\)** | **Duration** | **Action** | | --- | --- | --- | | 09:40 | | Hotfix development started | | 09:45 | 5 mins | Core x implemented | | 09:50 | 5 mins | Validation logic added | | 09:52 | 2 mins | 09:52 2m Unit tests completed | ## Build & Deployment | **Time \(UTC\)** | **Duration** | **Action** | | --- | --- | --- | | 09:55 | | CI pipeline triggered | | 09:58 | 3 mins | Build artifacts ready | | 10:00 | 2 mins | Staging deployment | | 10:10 | 10 mins | Staging verication | | 10:15 | 5 mins | Production rollout | | 10:25 | 10 mins | Full deployment | | 10:27 | 2 mins | Systems normal | ‌ # Resolution ## Corrective Actions: 1. Deployed hotfix with: 1. Updated ORM constraints \(64 chars\) 2. Additional migration validation 2. Verified all tenant\_id values in production 3. Validated cross-service compatibility ‌ # Preventive Measures | **Action Item** | **Owner** | | --- | --- | | Schema validation pre-checks | Data Eng | ‌ # Key Learnings **Testing Gap**: Need real-production-data migration testing

Read the full incident report →

Started Dec 14, 2024, 08:30 PM UTC· resolved Dec 15, 2024, 09:30 AM UTC

1. resolved Dec 17, 2024, 04:18 AM UTC

   Secure Relay Servers Unresponsive

2. postmortem Jan 07, 2025, 02:25 PM UTC

   **Postmortem Report on Production Outage of Secure Relay Server** 1. **Incident Summary** * **Date & Time:** 15th December 2024 2 PM to 16th December 2024 4 PM * **Duration:** 26 hours * **Impact:** Impacted all communications from Control Plane to all Dataplanes for Data Reliability feature. * **Severity Level:** High * **Affected Systems/Services:** Data Reliability \(Crawl, Profile, Policy Execution\), Cadence **2. Incident Timeline** | **Time \(IST\)** | **Event Description** | | --- | --- | | 16th December 2024 3:45 PM | Detection of the issue. | | 16th December 2024 3:50 PM | Investigation started by the \[Team\]. | | 16th December 2024 3:55 PM | Identified root cause or signs of root cause. | | 16th December 2024 4:00 PM | Fix or mitigation steps were applied. | | 16th December 2024 4:05 PM | Services fully restored. | | 16th December 2024 4:55 PM | Monitoring and validation of fixes. | **3. Root Cause Analysis \(RCA\)** * **Technical Root Cause:** Secure Relay servers, responsible for securely relaying data and communications between systems, ran out of disk space. This caused the system to exhaust all available storage, preventing critical operations like writing logs and handling temporary data. As a result, the service became unresponsive and failed to relay data effectively. * **Systems or Processes Involved:** Secure Relay Servers were the only systems involved in the outage. * **Monitoring Failure:** While monitoring was set up for the Secure Relay Load Balancer for unhealthy hosts, it was not integrated with Opsgenie. This oversight led to delays in detecting and responding to the disk space issue. **4. Resolution** * **Steps Taken:** Increased the disk space of Secure Relay servers. * **Long-term Fixes:** * Added comprehensive monitoring for Secure Relay servers, including service health and disk usage monitoring. * Integrated alerts for Load Balancer with Opsgenie to ensure proactive detection and escalation. * Review All Alerts to have action to integrate with OpsGenie **5. Lessons Learned** * **What Went Well:** Once issue was detected, issue was fixed immediately. All the Secure Relay Clients Connected Successfult * **What Didn’t Go Well:** It took long hours to detect the issue. * **Improvement Areas:** Review the alerts and make sure all alerts are integrated with Opsgenie **6. Action Items** | **Action** | **Owner** | **Deadline** | **Status** | | --- | --- | --- | --- | | Increase the Storage on Servers | Uday Shanbhag | 16th Decemeber 2024 | Closed | | Add Opsgenie integration for Load Balancer Health alert | Uday Shanbhag | 16th Decemeber 2024 | Closed | | Implement Secure Relay monitoring | Ritesh Mahajan | 16th Decemeber 2024 | Closed | | Implement Alerts for Secure Relay Systems | Ritesh Mahajan | 18th Decemeber 2024 | Open | **7. Conclusion** The outage on the Secure Relay Servers was caused due to an exhausted disk space, which resulted in unresponsive services and disrupted communication between critical systems. While the immediate fix of increasing disk space resolved the issue, the lack of proactive monitoring and integrated alerting systems contributed to the delayed response. Moving forward, the implementation of comprehensive monitoring, proactive alerting, and improved escalation workflows will significantly reduce the likelihood of recurrence. By learning from this incident, we are taking the necessary steps to strengthen our systems' resilience and improve overall operational efficiency. ‌ **Prepared by**:Uday Shanbhag **Date:** 17th Decmeber 2024

Read the full incident report →