Is Astronomer down?

Astronomer has released Runtime updates to address a publicly disclosed security vulnerability concerning private keys associated with eight eMudhra/emSign intermediate certificate authority (CA) certificates. For additional background, visit . **Scope and available updates** This advisory stems from an issue with a third-party certificate authority. Action required for customer-managed components depends on their specific certificate trust stores and verification mechanisms. The following Runtime releases incorporate certificate trust updates to mitigate this vulnerability: • **Runtime 13:** 13.12.1 • **Runtime 3.0:** 3.0-20 • **Runtime 3.1:** 3.1-23 • **Runtime 3.2:** 3.2-12 • **Runtime 3.3:** 3.3-10 For full details, please refer to the [Runtime release feed](https://www.astronomer.io/docs/runtime/runtime-release-feed). **Recommended customer actions** • Upgrade to the appropriate updated Runtime release for your environment. • Rebuild and redeploy custom images using the updated base image. Refer to our [mitigation guide](https://www.astronomer.io/docs/runtime/mitigate-emudhra-ca-compromise) for step-by-step instructions. • Audit customer-managed build pipelines and tooling, applying relevant vendor updates to certificate trust stores and signature verification components. • Ensure TLS and code-signature verification remain enabled. If updates cause validation failures, contact the endpoint owner or software publisher for replacement certificate or artifact guidance. Please note that Runtime updates address the trust components included within those specific releases. Customer-added dependencies, custom certificate bundles, and external tooling may require separate updates. **Support and further guidance** If you are unable to upgrade or require assistance determining the applicable next steps, please contact [**[email protected]**](mailto:[email protected]). Include your deployment model, Runtime version, and whether you maintain custom images or build pipelines. Additional mitigation guidance is currently being finalized. We will update this advisory as further details and Remote Execution releases are confirmed.

We have now released updated agent client patches for use in Remote Execution deployments that incorporate certificate trust updates to mitigate the vulnerability. These are 1.8.11, 1.7.7, and 1.6.5. See more details in the [Agent Release Notes](https://www.astronomer.io/docs/runtime/runtime-release-feed). Additionally, we have released another update of the Runtime images to handle the following two changes related to this vulnerability. • New virtual environments created with `python -m venv` now get pip 26.2.2 • Upgrade `grpcio` to 1.84.1, whose bundled CA list no longer includes the roots affected by the eMudhra compromise The following Runtime image versions include these changes. For more details see [Runtime Release Notes](https://www.astronomer.io/docs/runtime/runtime-release-feed). • **Runtime 13:** 13.12.2 • **Runtime 3.0:** 3.0-21 • **Runtime 3.1:** 3.1-24 • **Runtime 3.2:** 3.2-13 • **Runtime 3.3:** 3.3-11

We are investigating an issue where Astro Alerts notifications may be delayed for some customers. Our team is actively working to resolve the issue, and we will provide an update as soon as we have more information.

We have identified the cause and are rolling out a fix to the affected deployments. Alerts held during the incident have started to be delivered, and some alerts may continue to be delayed until the rollout is complete. We will provide another update once the fix is fully rolled out.

The fix has been applied to the majority of affected deployments, and delayed alerts continue to be delivered as we process the backlog. We are addressing a small number of remaining deployments and will provide another update once this is complete.

The backlog of delayed Astro Alerts has been fully processed, and we are monitoring recovery following the fix. A small number of alerts may still be briefly delayed as the fix completes on remaining systems.

Astro Alerts delivery has returned to normal, and this incident is resolved. We have applied the fix and fully processed the backlog of delayed alerts. Thank you for your patience while we worked to restore service.

We are currently investigating this issue. We will keep this page updated as we have more information.

We have identified the source of the issue and are actively working on a fix. We’ll share further updates as we make progress.

The issue has been resolved, and the Organization Dashboards are now working as expected.

The Astro Core API is showing an elevated number of errors. We are investigating actively and will update as soon as we know more. Next update will be within the next 30 minutes.

Our metrics indicate no impact on existing tasks and Dag Runs. They are continuing to run successfully. We are continuing to investigate what is causing the API and UI errors and are attempting a workaround.

We have restarted one of our services that was experiencing higher than expected load. This has returned the Astro UI and API to functionality. We are continuing to investigate root cause.

We have identified the proximate cause of the incident and are confident that the situation is stable. We will follow up with more detail in a postmortem at a later date.

We are currently seeing a capacity shortage in the Google Cloud (GCP) `us-central1` region. As a result, some Deployments running in this region may experience delays when scaling up worker nodes and pods. Some tasks may remain in a queued state for longer than usual while waiting for compute capacity. In some cases, tasks that stay queued beyond their configured timeout may fail. **Recommended workaround:** Configuring automated task retries (for example, setting `retries` and a `retry_delay` on your tasks or in your DAG's `default_args`) will help tasks recover automatically once capacity becomes available. We are working to address the issue and keep this page posted as we have more updates.

We have seen improved capacity in the GCP us-central1 region across additional machine types, and error rates for worker node and pod scale-up have dropped. If any tasks failed during this incident, you can clear and rerun them. To recover automatically from future capacity constraints, we recommend setting `retries` and `retry_delay` on your tasks or in `default_args`. If you’re still seeing issues, please contact Astronomer Support.