ThousandEyes incident
Cloud and Enterprise Agents showing DNSSEC failures.
Resolved after 2 days 2 hours, affecting Cloud and Enterprise Agents: Data ingress and Agent Configuration and Data Collection, on October 11, 2026.
This incident is now resolved. Please follow the instructions using one of the options below if any of your Enterprise Agents cannot automatically update themselves. Option 1: Add the New DNSSEC Key Adding the new DNSSEC key can restore the Agent’s connection to the ThousandEyes platform. After the Enterprise Agent reconnects, the Test Data collected since the Enterprise Agent’s last check-in should be retroactively pushed to the Test timeline as long as the Test Data remains in the Enterprise Agent’s local cache. The local cache retains Test Data for up to 36 hours after connectivity is lost; data older than that may be permanently lost. Cisco Application-Hosting (CAF) Agent Access the CLI of the Cisco device hosting the CAF Agent. Open a terminal in the app container: app-hosting connect session bash Append the new DNSSEC key to the key file (note – this is all one line): echo ". 172800 IN DNSKEY 257 3 8 AwEAAa96jeuknZlaeSrvyAJj6ZHv28hhOKkx3rLGXVaC6rXTsDc449/cidltpkyGwCJNnOAlFNKF2jBosZBU5eeHspaQWOmOElZsjICMQMC3aeHbGiShvZsx4wMYSjH8e7Vrhbu6irwCzVBApESjbUdpWWmEnhathWu1jo+siFUiRAAxm9qyJNg/wOZqqzL/dL/q8PkcRU5oUKEpUge71M3ej2/7CPqpdVwuMoTvoB+ZOT4YeGyxMvHmbrxlFzGOHOijtzN+u1TQNatX2XBuzZNQ1K+s2CXkPIZo7s6JgZyvaBevYtxPvYLw4z9mR7K2vaF18UYH9Z9GNUUeayffKC73PYc=" >> /usr/lib/te-agent/dns-rootserver.key Docker Agent Run the following command (note – this is all one line) on the Docker host, replacing with the Agent’s container name: docker exec sh -c 'echo ". 172800 IN DNSKEY 257 3 8 AwEAAa96jeuknZlaeSrvyAJj6ZHv28hhOKkx3rLGXVaC6rXTsDc449/cidltpkyGwCJNnOAlFNKF2jBosZBU5eeHspaQWOmOElZsjICMQMC3aeHbGiShvZsx4wMYSjH8e7Vrhbu6irwCzVBApESjbUdpWWmEnhathWu1jo+siFUiRAAxm9qyJNg/wOZqqzL/dL/q8PkcRU5oUKEpUge71M3ej2/7CPqpdVwuMoTvoB+ZOT4YeGyxMvHmbrxlFzGOHOijtzN+u1TQNatX2XBuzZNQ1K+s2CXkPIZo7s6JgZyvaBevYtxPvYLw4z9mR7K2vaF18UYH9Z9GNUUeayffKC73PYc=" >> /usr/lib/te-agent/dns-rootserver.key' Linux Package Agent Append the new DNSSEC key to the key file (note – this is all one line): echo ". 172800 IN DNSKEY 257 3 8 AwEAAa96jeuknZlaeSrvyAJj6ZHv28hhOKkx3rLGXVaC6rXTsDc449/cidltpkyGwCJNnOAlFNKF2jBosZBU5eeHspaQWOmOElZsjICMQMC3aeHbGiShvZsx4wMYSjH8e7Vrhbu6irwCzVBApESjbUdpWWmEnhathWu1jo+siFUiRAAxm9qyJNg/wOZqqzL/dL/q8PkcRU5oUKEpUge71M3ej2/7CPqpdVwuMoTvoB+ZOT4YeGyxMvHmbrxlFzGOHOijtzN+u1TQNatX2XBuzZNQ1K+s2CXkPIZo7s6JgZyvaBevYtxPvYLw4z9mR7K2vaF18UYH9Z9GNUUeayffKC73PYc=" >> /usr/lib/te-agent/dns-rootserver.key Please note: Customers will still need to complete Option 2 for a long-term fix, even if Option 1 is applied in the interim. Option 2: Upgrade the Agent Image Cisco Application-Hosting (CAF) Agent Run the following command on the Cisco device hosting the CAF Agent: If you have x86 architecture: app-hosting upgrade appid package https://downloads.thousandeyes.com/enterprise-agent/thousandeyes-enterprise-agent-x86_64-5.2.1.cisco.tar If you have ARM architecture: app-hosting upgrade appid package https://downloads.thousandeyes.com/enterprise-agent/thousandeyes-enterprise-agent-aarch64-5.2.1.cisco.tar Please note: Upgrading the Cisco App-Hosting Image (CAF) Agent image will clear the Enterprise Agent’s local Test Data cache. The Test timeline will have a data gap for the affected period, and the cached Test Data will not be backfilled. Docker Agent Use the latest image tag (or latest-agent if you are not using BrowserBot), or Docker image version 0.105.0 or later. docker pull docker stop docker rm docker run Linux Package Agent Upgrade the te-agent package to the latest version (1.247.2) using the system package manager (yum, dnf, or apt, depending on your distribution). For more details, see Upgrading Enterprise Agents.
Affected components
Update timeline
- identified Oct 11, 2026, 05:21 PM UTC
Cloud and enterprise agents are showing failure message in DNSSEC tests due intermediate validation. The issue started at 14:00 PM UTC.
- identified Oct 11, 2026, 08:19 PM UTC
We are continuing to investigate this issue.
- monitoring Oct 11, 2026, 10:31 PM UTC
A fix has been implemented. Operations will continue to monitor performance.
- monitoring Oct 12, 2026, 06:32 PM UTC
ThousandEyes is currently monitoring the fix. ThousandEyes Enterprise Agents will receive the fix and recover automatically. Commercial customers with FIPS-dependent configurations will not receive the fix automatically. Commercial customers with offline Agents should reach out to ThousandEyes Support to manually apply the fix. ThousandEyes for Government (FedRAMP) customers will have FIPS-dependent configurations, and will not receive the fix automatically. Please reach out to ThousandEyes FedRAMP Support to manually apply the fix.
- resolved Oct 13, 2026, 07:50 PM UTC
This incident is now resolved. Please follow the instructions using one of the options below if any of your Enterprise Agents cannot automatically update themselves. Option 1: Add the New DNSSEC Key Adding the new DNSSEC key can restore the Agent’s connection to the ThousandEyes platform. After the Enterprise Agent reconnects, the Test Data collected since the Enterprise Agent’s last check-in should be retroactively pushed to the Test timeline as long as the Test Data remains in the Enterprise Agent’s local cache. The local cache retains Test Data for up to 36 hours after connectivity is lost; data older than that may be permanently lost. Cisco Application-Hosting (CAF) Agent Access the CLI of the Cisco device hosting the CAF Agent. Open a terminal in the app container: app-hosting connect session bash Append the new DNSSEC key to the key file (note – this is all one line): echo ". 172800 IN DNSKEY 257 3 8 AwEAAa96jeuknZlaeSrvyAJj6ZHv28hhOKkx3rLGXVaC6rXTsDc449/cidltpkyGwCJNnOAlFNKF2jBosZBU5eeHspaQWOmOElZsjICMQMC3aeHbGiShvZsx4wMYSjH8e7Vrhbu6irwCzVBApESjbUdpWWmEnhathWu1jo+siFUiRAAxm9qyJNg/wOZqqzL/dL/q8PkcRU5oUKEpUge71M3ej2/7CPqpdVwuMoTvoB+ZOT4YeGyxMvHmbrxlFzGOHOijtzN+u1TQNatX2XBuzZNQ1K+s2CXkPIZo7s6JgZyvaBevYtxPvYLw4z9mR7K2vaF18UYH9Z9GNUUeayffKC73PYc=" >> /usr/lib/te-agent/dns-rootserver.key Docker Agent Run the following command (note – this is all one line) on the Docker host, replacing with the Agent’s container name: docker exec sh -c 'echo ". 172800 IN DNSKEY 257 3 8 AwEAAa96jeuknZlaeSrvyAJj6ZHv28hhOKkx3rLGXVaC6rXTsDc449/cidltpkyGwCJNnOAlFNKF2jBosZBU5eeHspaQWOmOElZsjICMQMC3aeHbGiShvZsx4wMYSjH8e7Vrhbu6irwCzVBApESjbUdpWWmEnhathWu1jo+siFUiRAAxm9qyJNg/wOZqqzL/dL/q8PkcRU5oUKEpUge71M3ej2/7CPqpdVwuMoTvoB+ZOT4YeGyxMvHmbrxlFzGOHOijtzN+u1TQNatX2XBuzZNQ1K+s2CXkPIZo7s6JgZyvaBevYtxPvYLw4z9mR7K2vaF18UYH9Z9GNUUeayffKC73PYc=" >> /usr/lib/te-agent/dns-rootserver.key' Linux Package Agent Append the new DNSSEC key to the key file (note – this is all one line): echo ". 172800 IN DNSKEY 257 3 8 AwEAAa96jeuknZlaeSrvyAJj6ZHv28hhOKkx3rLGXVaC6rXTsDc449/cidltpkyGwCJNnOAlFNKF2jBosZBU5eeHspaQWOmOElZsjICMQMC3aeHbGiShvZsx4wMYSjH8e7Vrhbu6irwCzVBApESjbUdpWWmEnhathWu1jo+siFUiRAAxm9qyJNg/wOZqqzL/dL/q8PkcRU5oUKEpUge71M3ej2/7CPqpdVwuMoTvoB+ZOT4YeGyxMvHmbrxlFzGOHOijtzN+u1TQNatX2XBuzZNQ1K+s2CXkPIZo7s6JgZyvaBevYtxPvYLw4z9mR7K2vaF18UYH9Z9GNUUeayffKC73PYc=" >> /usr/lib/te-agent/dns-rootserver.key Please note: Customers will still need to complete Option 2 for a long-term fix, even if Option 1 is applied in the interim. Option 2: Upgrade the Agent Image Cisco Application-Hosting (CAF) Agent Run the following command on the Cisco device hosting the CAF Agent: If you have x86 architecture: app-hosting upgrade appid package https://downloads.thousandeyes.com/enterprise-agent/thousandeyes-enterprise-agent-x86_64-5.2.1.cisco.tar If you have ARM architecture: app-hosting upgrade appid package https://downloads.thousandeyes.com/enterprise-agent/thousandeyes-enterprise-agent-aarch64-5.2.1.cisco.tar Please note: Upgrading the Cisco App-Hosting Image (CAF) Agent image will clear the Enterprise Agent’s local Test Data cache. The Test timeline will have a data gap for the affected period, and the cached Test Data will not be backfilled. Docker Agent Use the latest image tag (or latest-agent if you are not using BrowserBot), or Docker image version 0.105.0 or later. docker pull docker stop docker rm docker run Linux Package Agent Upgrade the te-agent package to the latest version (1.247.2) using the system package manager (yum, dnf, or apt, depending on your distribution). For more details, see Upgrading Enterprise Agents.
Your product runs on more than ThousandEyes
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.