NZIX experienced a major incident on October 7, 2025 affecting AKL-IX, lasting 6h 30m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 07, 2025, 09:35 PM UTC
We are currently investigating an issue that caused a flap on v6 sessions toward the route servers.
- monitoring Oct 07, 2025, 10:00 PM UTC
The offending malformed route is filtered upstream so no further impact is expected, we will place this on monitoring for now whilst work is underway to schedule emergency upgrades to the route servers to avoid this in the future. The root cause appears to be a malformed OTC BGP update attribute typically 4-bytes but in this case was 1024 on the prefix 2402:2de0::/32, our route server software reflected this back to peers as expected and routers tore down their session due to it. The permanent fix for this is upgrading bird the BGP software on the route servers to not propagate these malformed attributes.
- resolved Oct 08, 2025, 04:05 AM UTC
This incident is resolved for now. A new notice will come out for route server maintenance to resolve this permanently by adding RFC 9234 role capability.