Service Issue - Compute Hosts - IAD (Washington, DC)

Update timeline

1. investigating Mar 09, 2026, 06:39 PM UTC

   Our team is investigating an emerging service issue affecting compute hosts in IAD (Washington, DC). We will share additional updates as we have more information.

2. investigating Mar 09, 2026, 07:46 PM UTC

   Our team has identified the issue affecting connectivity in our IAD (Washington, DC) data center. We are working quickly to implement a fix, and we will provide an update as soon as the solution is in place.

3. monitoring Mar 09, 2026, 09:40 PM UTC

   At this time we have been able to correct the issues affecting connectivity in our IAD (Washington, DC) data center. We will be monitoring this to ensure that it remains stable. If you are still experiencing issues, please open a Support ticket for assistance.

4. resolved Mar 11, 2026, 07:29 PM UTC

   We haven’t observed any additional connectivity issues in our IAD (Washington, DC) data center, and will now consider this incident resolved. If you continue to experience problems, please open a Support ticket for assistance.

5. postmortem Mar 11, 2026, 07:29 PM UTC

   Beginning at 18:09 UTC on March 9, 2026, approximately 25% of Compute hosts in our IAD3 \(Washington DC\) data center experienced degraded network connectivity due to the application of an updated routing configuration intended to improve network performance. This updated router configuration was extensively tested and has been running in production in another region for several weeks without any issues. An investigation revealed that application of this change inadvertently caused loss of connectivity to these host due to a network configuration specific to the IAD3 data center. ‌ This resulted in loss of access and control of Linodes and clusters hosted on those machines until the issue was mitigated. Mitigation steps took longer than expected due to the specific networking nature of the issue and degradation of internal service visibility to impacted devices, which required additional planning to effectively apply a rollback to the prior configuration. ‌ We were able to successfully rollback the routing configuration and mitigate impact at 21:16 UTC on March 9, 2026. To prevent a reoccurrence of this issue, we are developing a new routing configuration deployment plan that avoids the identified failure modes and others inferred from our observations. ‌ This summary provides an overview of our current understanding of the incident given the information available. Our investigation is ongoing and any information herein is subject to change.