Elastic Cloud incident

Issues Enabling APM During Deployment Creation in 8.19.0 and 9.1.0

Major Resolved View vendor source →

Elastic Cloud experienced a major incident on August 4, 2025, lasting 3d 9h. The incident has been resolved; the full update timeline is below.

Started
Aug 04, 2025, 10:53 AM UTC
Resolved
Aug 07, 2025, 08:27 PM UTC
Duration
3d 9h
Detected by Pingoru
Aug 04, 2025, 10:53 AM UTC

Update timeline

  1. investigating Aug 04, 2025, 10:53 AM UTC

    We're currently investigating an issue where the APM endpoint is not available (greyed out) when creating deployments on versions 8.19.0 and 9.1.0. Initial findings indicate that APM Server is throwing errors during the provisioning process, which may impact user ability to enable APM in affected versions. Our engineering teams are actively investigating the root cause and implementing mitigation steps. We'll provide an update as soon as more information becomes available.

  2. identified Aug 04, 2025, 11:03 AM UTC

    We have identified the root cause of the issue affecting APM availability when creating new deployments in versions 8.19.0 and 9.1.0. APM Server encounters an error during provisioning, which results in the APM endpoint being unavailable. A mitigation is available for users who have already upgraded. For details, please refer to our Knowledge Base article: Mitigation steps(Elastic Cloud account required): https://support.elastic.co/knowledge/40e6b9d4 We are working on a broader fix and will continue to provide updates as progress is made.

  3. identified Aug 04, 2025, 05:08 PM UTC

    We have delisted versions 8.19.0 and 9.1.0, preventing new deployments and/or upgrades from using these versions. Existing deployments already using these versions can use the mitigation available. For details, please refer to our Knowledge Base article: Mitigation steps(Elastic Cloud account required): https://support.elastic.co/knowledge/40e6b9d4 Work continues on a broader fix. New updates will be provided as progress is made.

  4. identified Aug 06, 2025, 02:03 PM UTC

    A fix has been identified and is being validated. Once verified, a new release will be made available. An update will be provided when the release is available.

  5. resolved Aug 07, 2025, 08:27 PM UTC

    We have released new versions (8.19.1 and 9.1.1) to address the cause. New deployments may use these versions, or existing deployments may upgrade to them. Existing deployments already using these versions can use the mitigation available. For details, please refer to our Knowledge Base article: Mitigation steps (Elastic Cloud account required): https://support.elastic.co/knowledge/40e6b9d4