Coalesce Outage History
Coalesce is up right nowPingoru has recorded 32 incidents for Coalesce: 19 in the last 12 months and 7 in the last 90 days.
Show as a table
| Month | Incidents | Hours with an open incident |
|---|---|---|
| April 2026 | 3 | 10 of 720 (1%) |
| May 2026 | 0 | 14 of 744 (2%) |
| June 2026 | 0 | 0 of 720 (0%) |
| July 2026 | 6 | 30 of 744 (4%) |
| August 2026 | 1 | 1 of 744 (0%) |
| September 2026 | 3 | 30 of 720 (4%) |
| October 2026 | 1 | 1 of 526 (0%) |
Incidents, newest first
Minor October 9, 2026 Transform Job Creation Blocked 1h 5m Scheduler
Started Oct 09, 2026, 06:21 PM UTC· resolved Oct 09, 2026, 07:27 PM UTC
- identified Oct 09, 2026, 06:21 PM UTC
Transform users are currently unable to create scheduled jobs. We have identified the issue and working on a resolution. This only effects the creation of new scheduled jobs, previously created jobs are continuing to execute.
- resolved Oct 09, 2026, 07:27 PM UTC
This issue has been resolved. Please reach out to our Support team if you are experiencing any issues.
Minor September 9, 2026 Transform Snowflake OAuth Errors 14h 43m API, Scheduler, Web App
Started Sep 09, 2026, 01:33 AM UTC· resolved Sep 09, 2026, 04:17 PM UTC
- identified Sep 09, 2026, 01:33 AM UTC
Customers are experiencing OAuth authentication issues with Snowflake. We have identified the issue and are pushing out a fix.
- monitoring Sep 09, 2026, 03:35 AM UTC
We have deployed a fix for anyone experiencing the following error when using OAuth with Snowflake: "error":"Unable to process OAuth callback! error=RPError: iss mismatch, expected snowflake, got: https://example.snowflakecomputing.com/oauth","errorDetail":"" This change introduces case sensitivity for the configured Snowflake URL. If you continue to receive the above error and have a mixed-case URL, a workaround is to change the URL to all lowercase.
- resolved Sep 09, 2026, 04:17 PM UTC
This issue has been resolved.
Minor September 2, 2026 APAC Job Execution Failure 12h 55m australia-southeast1
Started Sep 02, 2026, 02:56 AM UTC· resolved Sep 02, 2026, 03:51 PM UTC
- identified Sep 02, 2026, 02:56 AM UTC
Some of our customers in our APAC region are seeing jobs not completing. We have identified the root cause and will be releasing a fix shortly.
- monitoring Sep 02, 2026, 05:29 AM UTC
A fix has been deployed and we will monitor for any issues. Further details when fully resolved.
- resolved Sep 02, 2026, 03:51 PM UTC
This incident is resolved. This incident was caused by a recent update that lacked backwards compatibility with prior versions of the coa CLI. The resolution was to follow up with another update restoring the backwards compatibility.
Notice September 1, 2026 Job Execution Failure 2h us-central1
Started Sep 01, 2026, 10:00 AM UTC· resolved Sep 01, 2026, 12:00 PM UTC
- resolved Sep 01, 2026, 06:28 PM UTC
Between approximately 10:00 and 12:00 UTC today, some customers experienced delayed or stuck job runs for our Transform customers in North America. During a routine, provider-managed Kubernetes upgrade in one of our US regions, our cloud provider was unable to provision replacement compute capacity. We experienced a similar issue on August 17th as described here: https://status.coalesce.io/incidents/43zcl3trbjxx We took steps after that prior incident to prevent provider upgrades from effecting our customers but those changes were not sufficient. We are currently making the following changes: - Expanded Capacity Buffers: We are adding supplementary node pools (using C3/C3D machine types) alongside our primary pools to act as an immediate capacity buffer if our primary hardware type is unavailable. - Automated Job Cleanup: We deployed an improvement to our job lifecycle management where delayed or stuck jobs now cleanly self-terminate, instantly freeing up resources for healthy jobs. - Resilient Compute Architecture: We are actively implementing GKE Compute Classes, which will allow our infrastructure to automatically draw from up to six different machine types, eliminating our reliance on a single hardware type during cloud provider shortages.
Minor July 29, 2026 Transform North America: Performance Degradation 6h 7m API, Scheduler, Web App
Started Jul 29, 2026, 04:57 PM UTC· resolved Jul 29, 2026, 11:05 PM UTC
- identified Jul 29, 2026, 04:57 PM UTC
We have identified the likely cause of a performance issue with Transform that is effecting overall operations. We will be pushing out a change here shortly to address this and will update this incident when complete.
- monitoring Jul 29, 2026, 07:53 PM UTC
We have pushed out the change that should resolve this issue. Will monitor and close this out with more details when complete.
- resolved Jul 29, 2026, 11:05 PM UTC
We increased resources to our database cluster to better handle customer workloads. This change was rolled out today at approximately 12:15PM Pacific and all operations have returned to normal. We are adding additional monitoring & alerting to improve our response times if this situation occurs in the future.
Minor July 25, 2026 Transform North America: Job Execution Service Interruption 37m us-central1
Started Jul 25, 2026, 04:24 PM UTC· resolved Jul 25, 2026, 05:02 PM UTC
- identified Jul 25, 2026, 04:24 PM UTC
We have identified a downstream issue from our scheduled maintenance that is effecting job execution. We will update this issue when resolved.
- resolved Jul 25, 2026, 05:02 PM UTC
North American Transform Customers, We successfully completed our scheduled outage this morning but some of our customers are experiencing job execution failures due to changes we made during the outage. During the scheduled maintenance we took a series of steps to increase the scale of our cloud infrastructure, which is now fully utilizing all of our documented outbound network range for our Google Cloud us-central1 region. If you have a Data Platform network policy defined, please confirm and update your IP address allowed list to include the full set documented here under the “US | Google | us-central1 | Iowa” section: https://docs.coalesce.io/docs/setup-your-project/setup-requirements/network-requirements Ensure you have the full list configured, not only the addresses that are new to you. Coalesce may connect from any address in this range, and which one it uses on a given connection is not fixed. Allow-listing a subset can appear to work and then fail later. We apologize for the disruption. Please reach out to our Support Team with any questions.
Major July 24, 2026 GitHub Service Degradation Impacting Coalesce Job Scheduler 1h 9m Scheduler, Web App
Started Jul 24, 2026, 04:34 PM UTC· resolved Jul 24, 2026, 05:43 PM UTC
- investigating Jul 24, 2026, 04:34 PM UTC
We are currently experiencing issues establishing connections to GitHub repositories due to a widespread GitHub service disruption. This impacts features within Coalesce that rely on Git repository connectivity, including the Job Scheduler. Our systems are operational, but actions that require communication with GitHub may fail until GitHub restores full service. We are monitoring GitHub’s status page for updates and will post more information as it becomes available. GitHub Status Page: https://www.githubstatus.com/
- resolved Jul 24, 2026, 05:43 PM UTC
Github is reporting this as resolved, we are seeing our Github related services working as expected.
Major April 23, 2026 Job Scheduler 4h 2m Scheduler
Started Apr 23, 2026, 04:37 PM UTC· resolved Apr 23, 2026, 08:39 PM UTC
- investigating Apr 23, 2026, 04:37 PM UTC
We are investigating the impact of a current Github outage that is effecting our Job Scheduler's ability to execute scheduled jobs.
- monitoring Apr 23, 2026, 05:31 PM UTC
The Github outage is recovering and Coalesce scheduled jobs have resumed processing.
- resolved Apr 23, 2026, 08:39 PM UTC
This incident is resolved
Minor April 2, 2026 Notification service outage 3h 32m Scheduler
Started Apr 02, 2026, 01:16 AM UTC· resolved Apr 02, 2026, 04:49 AM UTC
- identified Apr 02, 2026, 01:16 AM UTC
We are resolving an issue with our Notifications service that sends job update email notifications. The service was inadvertently taken offline due to an infrastructure configuration change. Jobs are continuing to be processed and should not be impacted by this outage. We estimate the service should be restored in 30 minutes or less and will update this incident with more details.
- monitoring Apr 02, 2026, 01:26 AM UTC
The service is online and we are monitoring for any issues.
- resolved Apr 02, 2026, 04:49 AM UTC
Incident Summary: Notifications Service Outage Date: April 1, 2026 Duration: approximately 8 hours Severity: Email notifications only - no impact to job processing What happened On April 1st, 2026, our Notifications service that is responsible for sending job update email notifications went offline. The service was unable to start due to a configuration change that removed the service's image from our image registry. Jobs continued to run and complete normally throughout the incident. No data was lost. The impact was the loss of email notifications for job status updates during the outage. Root cause As part of a planned infrastructure cost optimization effort, we activated a storage cleanup policy on our container image registry to remove old, unused images. The Notifications service uses an independent release cycle from our core platform and had not been rebuilt recently. Its deployed image version fell outside the retention window and was removed by the cleanup policy. When the service attempted to restart, it could not pull the required image. Resolution We identified the issue, deployed the latest version of the Notifications service, and confirmed full functionality was restored. Steps taken to prevent recurrence 1. Added additional image version retention policy, ensuring that infrequently built services are never pruned while in service. 2. Upgraded monitoring and alerting - we are adding additional monitors and alerts that will page our On Call Engineering team to ensure faster response times. 3. Audit of all deployed services - we have verified that all currently deployed image versions across all environments are present in the registry and covered by the updated retention policy. We apologize for the inconvenience and are committed to ensuring this does not happen again. If you have any questions, please reach out to our support team.
Critical March 9, 2026 Catalog features unavailable 8h 45m Web App
Started Mar 09, 2026, 12:15 AM UTC· resolved Mar 09, 2026, 09:00 AM UTC
- investigating Mar 09, 2026, 12:15 AM UTC
We are currently investigating an incident with Catalog functionality.
- identified Mar 09, 2026, 02:59 AM UTC
We have identified the cause of this outage and are working towards a resolution.
- resolved Mar 09, 2026, 09:00 AM UTC
This incident has been resolved. A report will be published shortly.
Major February 25, 2026 Intermittent Job Timeouts 15d 11h Scheduler
Started Feb 25, 2026, 11:21 AM UTC· resolved Mar 12, 2026, 10:58 PM UTC
- investigating Feb 25, 2026, 11:21 AM UTC
We are investigating reports of jobs intermittently timing out. Our team is actively working to identify the root cause and will provide updates as soon as more information becomes available.
- identified Feb 25, 2026, 01:13 PM UTC
We have identified a bottleneck in our job scheduling queue in the US region. We are currently recovering our infrastructure and monitoring the backlog of stale jobs. Customers may continue to see some older jobs fail, but new jobs are starting to process. We will provide further updates as the queue returns to normal levels.
- monitoring Feb 25, 2026, 02:07 PM UTC
A fix has been implemented and we are monitoring the results.
- identified Feb 25, 2026, 04:10 PM UTC
Job scheduling issues in the US region have recurred. We have identified that certain jobs are becoming unresponsive and blocking the rest of the queue. We are intervening to clear the backlog and are investigating the underlying cause of these stalled processes. Customers should expect intermittent delays or timeouts in the interim.
- identified Feb 25, 2026, 05:15 PM UTC
We are continuing to investigate. We will post updates at the top of each hour, or sooner if new information becomes available.
- monitoring Feb 25, 2026, 05:37 PM UTC
We are no longer observing job timeouts at this time. Our team continues to monitor the system closely to ensure stability. We will provide further updates if anything changes.
- monitoring Feb 26, 2026, 12:06 AM UTC
Jobs are no longer timing out and the Scheduler is back to operational. We are going to continue to monitor over the next 12-24 hours and provide an update tomorrow with more details on the cause of the service interruption.
- monitoring Feb 26, 2026, 04:43 PM UTC
Between 2-5 AM Pacific Time US we discovered jobs were again backing up and not processing as expected. Engineering has developed a patch to resolve this issue we are targeting to be released by 12 PM Pacific. We will continue to monitor and resolve any issues with job processing and update this incident when the patch has been shipped.
- monitoring Feb 26, 2026, 08:53 PM UTC
We have released a new version 7.29.4 that mitigates the performance issues our customers are experiencing. We expect it to reduce customer impact from this issue. Any customers using the coa CLI are recommended to upgrade. We will continue to update this issue as we make progress towards the root cause.
- monitoring Feb 26, 2026, 11:39 PM UTC
Clients can expect to continue to experience issues with delays and timeouts. We have an additional reliability optimization we have developed that is targeted for release within the next 2 hours or less. Expect our next update here when available.
- monitoring Feb 27, 2026, 01:06 AM UTC
Clients may still see issues with Deploy and Refresh operations which we are continuing to monitor. We have released an additional update, version 7.29.5, that includes follow-on improvements to the reliability of our Refresh and Deploy operations. This update is included in version 7.29.5 of our coa CLI and is a recommended upgrade to all customers. We are continuing to monitor and provide updates as this incident progresses.
- monitoring Feb 27, 2026, 05:48 PM UTC
The update we released yesterday has resolved the ongoing Deploy and Refresh failures from our clients. We are continuing to monitor and will provide details on the root cause of the issue as soon as available. A small number of our clients are seeing delays running Deploy operations that are unrelated to the ongoing issue this week. We have identified a likely root cause and if confirmed and resolved will transition this issue to Operational.
- monitoring Feb 28, 2026, 12:09 AM UTC
We are no longer seeing general or widespread issues with our operations and will continue to monitor this incident over the weekend as we work towards a final solution and root cause analysis.
- monitoring Mar 03, 2026, 11:54 AM UTC
Clients are reporting errors running refresh and deploy operations. We are investigating and will provide updates.
- monitoring Mar 03, 2026, 03:07 PM UTC
We identified a resource that needed to be restarted and the DEADLINE_EXCEEDED errors clients were seeing should cease. This was unrelated to the overall incident we are tracking here and work will continue to resolving fully.
- monitoring Mar 04, 2026, 09:47 PM UTC
Starting at approximately 10PM Pacific time Tuesday March 3rd a single resource in our Scheduler API resource pool degraded in performance and did not recover. This caused a performance impact across the product due to the reliance on the Scheduler as our core API service. At 9AM Pacific time Wednesday March 4th, this resource was restarted and operations returned to normal. We are continuing to root cause the reason we experienced a similar scenario on the AM of March 3rd. We have added additional monitoring and alerting for this specific behavior to reduce any impact if it re-occurs until we deliver a full resolution. Updates to follow with a full timeline and root cause of this entire incident.
- monitoring Mar 06, 2026, 03:28 AM UTC
Incident Update — March 5, 2026 On the afternoon of March 4th, we confirmed the core root cause of this incident. We have been monitoring our infrastructure continuously since then and are not seeing ongoing impact to our clients. Root Cause: This incident was caused by a series of compounding issues, but the primary root cause is a performance flaw in a third-party SQL parsing library used throughout our platform. We identified the problematic behavior early in our investigation, but because SQL parsing is a centralized function critical to Deploy, Refresh, and API operations, we could not simply disable it. A fix required coordination with the third-party vendor, who delivered us a resolution this morning. What We Did While Awaiting the Vendor Fix: Rather than wait, we invested heavily in hardening the platform against the impact of this flaw: - Reliability improvements: Added timeout and retry mechanisms to prevent jobs from hanging indefinitely on unresponsive connections - Resource isolation: Separated heavy SQL parsing workloads onto dedicated infrastructure so they no longer block critical scheduler operations - Performance fixes: Identified and resolved a code-level regression that was amplifying the parsing cost per request - Monitoring & alerting: Deployed new observability tooling and health checks to detect and automatically remediate the application when a degraded state is detected Today's Release: Today we released version 7.30.6, which includes additional reliability and performance improvements that further reduce the impact of the underlying parsing flaw. We will continue to monitor closely and will ship the vendor's fix as soon as it is available. Next Steps: - We are preparing a detailed Incident Report that will be available upon request, including a full timeline and additional details. - We are testing an updated fix from our vendor and expect to release an update early next week, pending our quality checks. We appreciate your patience and collaboration during this ongoing impact to your operations.
- resolved Mar 12, 2026, 10:58 PM UTC
We have released version 7.30.7 that includes the performance improvement from the third-party library we outlined in the prior update. All systems are operating as expected, please reach out to support if you need any assistance.
Notice January 20, 2026 Transform [US Regions Only] - Issues Running Scheduled Jobs 2h Scheduler
Started Jan 20, 2026, 09:49 PM UTC· resolved Jan 20, 2026, 11:50 PM UTC
- identified Jan 20, 2026, 09:49 PM UTC
We are aware of an issue impacting all US regions whereby scheduled jobs are not running. We have identified and we are working to triage the issue. Updates will be posted here shortly on status.
- monitoring Jan 20, 2026, 10:18 PM UTC
We've implemented a fix and are seeing systems return to normal. We are closely monitoring and will provide additional updates shortly.
- resolved Jan 20, 2026, 11:50 PM UTC
All issues have been resolved and we are marking this incident as resolved. Please reach out to Coalesce support if you continue to be impacted in any way.
Major November 26, 2025 Service Degradation with the Coalesce Job Scheduler 1h 25m Scheduler
Started Nov 26, 2025, 04:05 PM UTC· resolved Nov 26, 2025, 05:30 PM UTC
- investigating Nov 26, 2025, 04:05 PM UTC
We are currently experiencing service degradation with the Coalesce Job Scheduler. Some scheduled jobs may be delayed or may fail to start. There is no impact on jobs triggered manually, via the API, or via the CLI—those continue to run as expected. Our engineering team is actively investigating the issue. We will provide further updates as more information becomes available.
- monitoring Nov 26, 2025, 05:02 PM UTC
We have implemented a fix to the Transform Job Scheduler. Scheduled jobs from this point forward should execute successfully as scheduled. If you had any time-sensitive jobs scheduled between 01:00 and 16:30 UTC on 11/26, please review their executions in the Coalesce Deploy window to confirm they ran as expected or to identify any that may require manual intervention. We will continue to monitor the system to ensure continued execution on schedule.
- resolved Nov 26, 2025, 05:30 PM UTC
This incident has been resolved.
Notice October 20, 2025 [Coalesce Transform] - Intermittent Issues 17h 56m API, Scheduler, Web App
Started Oct 20, 2025, 08:18 PM UTC· resolved Oct 21, 2025, 02:15 PM UTC
- identified Oct 20, 2025, 08:18 PM UTC
Please note that we are being impacted by the large AWS us-east-1 issues as detailed on their status page, linked below. We are continuing to monitor the situation and will provide updates as they become available. https://health.aws.amazon.com/health/status
- monitoring Oct 20, 2025, 08:45 PM UTC
We are seeing additional impacts across all Coalesce Transform regions, due to issues with some of our upstream vendors that we depend here at Coalesce, whom are also directly impacted by the AWS outage. We will continue to monitor and update when there is news to share.
- monitoring Oct 21, 2025, 02:14 PM UTC
Our upstream vendors and AWS have resolved their issues and all operations have returned to normal for Coalesce Transform. We are therefore closing this incident. Thank you for your understanding and patience during this time.
- resolved Oct 21, 2025, 02:15 PM UTC
This incident has been resolved.
Minor October 7, 2025 AI Capabilities down due to OpenAI infra set misconfig –
Started Oct 07, 2025, 09:00 AM UTC· resolved Oct 07, 2025, 09:00 AM UTC
- resolved Oct 07, 2025, 04:06 PM UTC
This incident has been resolved.
Minor September 4, 2025 Investigating Intermittent Issues 18h 37m API, Scheduler
Started Sep 04, 2025, 10:22 PM UTC· resolved Sep 05, 2025, 04:59 PM UTC
- investigating Sep 04, 2025, 10:22 PM UTC
We are aware of intermittent issues and are investigating. We will provide updates as we progress in our investigation.
- investigating Sep 04, 2025, 10:22 PM UTC
We are continuing to investigate this issue.
- monitoring Sep 04, 2025, 11:03 PM UTC
A fix has been implemented and we are monitoring the results.
- monitoring Sep 04, 2025, 11:50 PM UTC
All systems continue to be operating normally. We are currently working on identifying the root cause of the issue and will update this incident when determined.
- monitoring Sep 05, 2025, 03:42 PM UTC
We are continuing to monitor this incident and will provide updates throughout the day.
- resolved Sep 05, 2025, 04:59 PM UTC
This incident is resolved and no longer affecting our users and operations. Please reach out to Coalesce Support if you continue to be impacted.
Major August 12, 2025 OAuth Authentication Issue Affecting Private Link Users 9h 37m API, Scheduler, Web App
Started Aug 12, 2025, 03:33 AM UTC· resolved Aug 12, 2025, 01:11 PM UTC
- identified Aug 12, 2025, 03:33 AM UTC
We are currently investigating and resolving an authentication issue affecting customers using Private Link with Snowflake OAuth. The issue is preventing successful authentication callbacks after users are redirected to Snowflake for single sign-on. Following a recent update to improve OAuth redirect handling in Test Connections (release 7.22), we've identified an issue with how the system resolves Private Link DNS endpoints. This is resulting in authentication failures with the error: "Unable to process OAuth callback - getaddrinfo ENOTFOUND." This issue is affecting: Private Link customers using Snowflake OAuth for workspace authentication Users attempting to log into Snowflake via the UI with user credentials Refresh job executions initiated through Private Link URLs Current Status Our engineering team has identified the issue and testing a fix. We are working to release an update to resolve the issue in the next 2-4 hours or less. If this time window changes, we will provide updates.
- monitoring Aug 12, 2025, 05:26 AM UTC
A fix has been implemented and available to all users. We will monitor and update when we are confident this is fully resolved.
- monitoring Aug 12, 2025, 01:11 PM UTC
We are continuing to monitor for any further issues.
- resolved Aug 12, 2025, 01:11 PM UTC
This incident has been resolved.
Major July 1, 2025 Not properly fetching saved credentials from secure storage 2h 15m API, Scheduler
Started Jul 01, 2025, 12:01 AM UTC· resolved Jul 01, 2025, 02:16 AM UTC
- investigating Jul 01, 2025, 12:01 AM UTC
We are investigating an issue where the Coalesce product is not properly fetching saved credentials from secure storage, resulting in failed operations. We will provide the next update by 9:00 PM eastern.
- identified Jul 01, 2025, 12:26 AM UTC
The issue has been identified and a fix is being implemented. We will provide the next update by 9:30 PM eastern.
- identified Jul 01, 2025, 01:27 AM UTC
We are continuing to work on a fix for this issue. We will provide the next update by 10:30 PM eastern.
- monitoring Jul 01, 2025, 01:49 AM UTC
A fix has been implemented and we are monitoring the results. We will provide the next update by 10:45 PM eastern.
- resolved Jul 01, 2025, 02:16 AM UTC
This incident has been resolved.
Notice June 12, 2025 Login Failures via Google Firebase 19h 32m API, Scheduler, Web App
Started Jun 12, 2025, 06:18 PM UTC· resolved Jun 13, 2025, 01:50 PM UTC
- investigating Jun 12, 2025, 06:18 PM UTC
There appears to be an issue with the Google Firebase service which we use for authentication, causing errors during user login. We are investigating and will continue to provide updates.
- investigating Jun 12, 2025, 06:28 PM UTC
We are also observing authentication errors for our Scheduler and API.
- investigating Jun 12, 2025, 06:39 PM UTC
There is now an ongoing outage listed on Google Firebase’s authentication service status page: https://status.firebase.google.com/ We expect normal operation once Google resolves their incident.
- investigating Jun 12, 2025, 06:45 PM UTC
Updating this incident to Major outage
- monitoring Jun 12, 2025, 07:31 PM UTC
Starting to see services recover. We still expect to see intermittent errors as services come back online.
- resolved Jun 13, 2025, 01:50 PM UTC
Google has resolved their issues and service has returned to normal, we are therefore closing this incident. Please contact Coalesce support directly if there are any concerns.
Minor May 29, 2025 New Package Installations Impacted 7h 12m Web App
Started May 29, 2025, 09:45 PM UTC· resolved May 30, 2025, 04:58 AM UTC
- identified May 29, 2025, 09:45 PM UTC
We have identified an issue impacting Coalesce’s package services, specifically preventing the installation of new packages into workspaces. Our team is actively working on a fix to restore full functionality. We will continue to provide updates as progress is made.
- identified May 30, 2025, 12:10 AM UTC
A fix for the issue impacting the Coalesce package services has been identified and our team is working to thoroughly test the software patch. Once all tests have been passed the updated version of the application will be deployed to our production platform and an update will be posted here.
- resolved May 30, 2025, 04:58 AM UTC
We have deployed an update that corrects the issue impacting the Coalesce package service. Customers are now able to install new packages into workspaces. Please reach out to Support for any assistance.
Major May 27, 2025 Errors Commiting to GitHub 6h 46m API, Scheduler, Web App
Started May 27, 2025, 12:41 PM UTC· resolved May 27, 2025, 07:27 PM UTC
- investigating May 27, 2025, 12:41 PM UTC
We are investigating errors received when attempting to commit to git. We will update here as the investigation progresses with any additional information that may be uncovered.
- identified May 27, 2025, 02:07 PM UTC
We've identified the issue as only impacting Coalesce clients that utilize GitHub as their git provider. A fix has been created and is currently being tested internally. Once testing is successful a new version of the Coalesce application will be deployed to all production clusters and release notes will be publicly available.
- identified May 27, 2025, 03:10 PM UTC
Testing of the software fix is proceeding as expected. Additional updates will be posted here when we deploy the software fix to our production infrastructure.
- monitoring May 27, 2025, 06:10 PM UTC
A fix for this issue has passed testing successfully and was deployed to all Coalesce production clusters. Please refresh any browser sessions to realize the fix that is included in the latest version Coalesce v7.18.2. We continue to monitor this situation for approximately one (1) hour. If our logging remains clear we will mark this incident as resolved.
- resolved May 27, 2025, 07:27 PM UTC
We're confirming this issue is resolved and has not reoccured after we released the latest version of the Coalesce application.
Minor April 29, 2025 Snowflake OAuth Workspace Authentication Errors: Request failed with status code 404 2h 15m API, Scheduler, Web App
Started Apr 29, 2025, 04:46 PM UTC· resolved Apr 29, 2025, 07:01 PM UTC
- monitoring Apr 29, 2025, 04:46 PM UTC
We're currently monitoring an issue where some users are experiencing 404 error messages when authenticating to Snowflake via OAuth in the Coalesce user interface. Your data and configurations are safe and not affected by this authentication issue. To quickly resolve this on your end: 1. Clear your browser cache: - Chrome: Settings > Privacy and Security > Clear browsing data - Firefox: Options > Privacy & Security > Cookies and Site Data > Clear Data - Edge: Settings > Privacy, search, and services > Clear browsing data - Safari: Preferences > Privacy > Manage Website Data > Remove All 2. Close and restart your browser completely 3. Log back into Coalesce and attempt to reconnect to Snowflake This workaround has resolved the issue for our affected users. If you continue experiencing problems, please contact our Customer Support Team.
- resolved Apr 29, 2025, 07:01 PM UTC
The 404 errors are no longer occurring. Please reach out to Support if you experience any issues.
Notice April 10, 2025 Scheduler Issue –
Started Apr 10, 2025, 02:30 AM UTC· resolved Apr 10, 2025, 02:30 AM UTC
- resolved Apr 10, 2025, 03:07 AM UTC
From 3:39 PM to 10:20 PM Eastern, we experienced an issue with our job scheduler. During this period, scheduled jobs failed to start, and notification emails for these failures were not sent. We recommend reviewing the status of any jobs scheduled during this timeframe and manually running any missed jobs if necessary. As of 10:20 PM Eastern, the scheduler issue has been resolved, and jobs will now run as scheduled. We apologize for any inconvenience this may have caused and appreciate your patience and understanding.
Minor February 11, 2025 Issue Identified 23m API, Scheduler, Web App
Started Feb 11, 2025, 10:29 PM UTC· resolved Feb 11, 2025, 10:53 PM UTC
- identified Feb 11, 2025, 10:29 PM UTC
We have identified an issue related to our recent release where some static content is being served from a testing domain. This may impact clients with security policies that restrict traffic to specific domain names. Our team is actively working on deploying a fix to resolve this as quickly as possible. We sincerely regret any inconvenience this may cause and appreciate your patience. Temporary Workaround: Allowing content to be served from the domain coalescestage.io may help mitigate the issue while we are working towards a fix. We will provide further updates as we progress toward a resolution. If you have any questions or concerns, please reach out to our support team. Thank you for your understanding. — Coalesce Team
- resolved Feb 11, 2025, 10:53 PM UTC
A fix was developed, tested successfully, and deployed to production. The incident is now resolved.
Minor January 31, 2025 Authentication Issue that is impacting API requests 1h 30m API
Started Jan 31, 2025, 05:07 PM UTC· resolved Jan 31, 2025, 06:37 PM UTC
- identified Jan 31, 2025, 05:07 PM UTC
Coalesce is currently experiencing an authentication issue that is impacting API requests. Our team is actively investigating and working to resolve this degradation. We will provide updates as more information becomes available.
- resolved Jan 31, 2025, 06:37 PM UTC
The authentication issues impacting API requests have been resolved. We identified the root cause, deployed a fix, and are actively monitoring the system to ensure stability. All services have returned to normal operation. Thank you for your patience during this disruption.