Is Coalesce down?

Coalesce Transform for customers in our North America region experienced a service degradation starting on Wednesday April 29th with the release of 7.33 that is affecting job execution. - Root cause: A change in 7.33 made our node selector matching case-insensitive, which in a rare circumstance, made it possible for newly created jobs enter a recursive job-launch loop. - Current state: We have applied a concurrency limit that prevents runaway job launches and are working on a hotfix to resolve the underlying issue. - Fix inbound ASAP: Hotfix reverting selector matching to case-sensitive is approved and in our release pipeline now. We will update this incident once resolved.

The hotfix has been delivered in version 7.33.2

The incident has been resolved.

We are investigating the impact of a current Github outage that is effecting our Job Scheduler's ability to execute scheduled jobs.

The Github outage is recovering and Coalesce scheduled jobs have resumed processing.

This incident is resolved

We are resolving an issue with our Notifications service that sends job update email notifications. The service was inadvertently taken offline due to an infrastructure configuration change. Jobs are continuing to be processed and should not be impacted by this outage. We estimate the service should be restored in 30 minutes or less and will update this incident with more details.

The service is online and we are monitoring for any issues.

Incident Summary: Notifications Service Outage Date: April 1, 2026 Duration: approximately 8 hours Severity: Email notifications only - no impact to job processing What happened On April 1st, 2026, our Notifications service that is responsible for sending job update email notifications went offline. The service was unable to start due to a configuration change that removed the service's image from our image registry. Jobs continued to run and complete normally throughout the incident. No data was lost. The impact was the loss of email notifications for job status updates during the outage. Root cause As part of a planned infrastructure cost optimization effort, we activated a storage cleanup policy on our container image registry to remove old, unused images. The Notifications service uses an independent release cycle from our core platform and had not been rebuilt recently. Its deployed image version fell outside the retention window and was removed by the cleanup policy. When the service attempted to restart, it could not pull the required image. Resolution We identified the issue, deployed the latest version of the Notifications service, and confirmed full functionality was restored. Steps taken to prevent recurrence 1. Added additional image version retention policy, ensuring that infrequently built services are never pruned while in service. 2. Upgraded monitoring and alerting - we are adding additional monitors and alerts that will page our On Call Engineering team to ensure faster response times. 3. Audit of all deployed services - we have verified that all currently deployed image versions across all environments are present in the registry and covered by the updated retention policy. We apologize for the inconvenience and are committed to ensuring this does not happen again. If you have any questions, please reach out to our support team.