Infoblox incident

DNS resolution issue in multiple PoP locations

Major Resolved View vendor source →

Infoblox experienced a major incident on March 30, 2025 affecting DNS Resolution - Virginia, US and DNS Resolution - Ohio, US and 1 more component, lasting 4h 17m. The incident has been resolved; the full update timeline is below.

Started
Mar 30, 2025, 06:23 PM UTC
Resolved
Mar 30, 2025, 10:40 PM UTC
Duration
4h 17m
Detected by Pingoru
Mar 30, 2025, 06:23 PM UTC

Affected components

DNS Resolution - Virginia, USDNS Resolution - Ohio, USDNS Resolution - California, USDNS Resolution - São Paulo, BrazilDNS Resolution - Tokyo, JPDNS Resolution - Singapore, SGDNS Resolution - Mumbai, INDNS Resolution - Frankfurt, DEDNS Resolution - London, GB

Update timeline

  1. investigating Mar 30, 2025, 06:23 PM UTC

    We are investigating an issue affecting DNS resolution in multiple PoP locations. Our team is working diligently to identify the root cause and implement a solution quickly. We apologize for any inconvenience this may be causing. Impact: Some customers may experience intermittent DNS resolution issues due to this. We will provide updates on the progress of the investigation and resolution. If you are experiencing a service impact or require further assistance, kindly contact our support team at https://support.infoblox.com/

  2. investigating Mar 30, 2025, 07:08 PM UTC

    We are continuing to investigate this issue.

  3. identified Mar 30, 2025, 07:38 PM UTC

    The issue is likely to impact customers who are forwarding their DNS traffic using Anycast IP addresses 52.119.40.100 and 103.80.5.100. We are actively working with our network vendor to resolve the issue. Workaround: Customers experiencing DNS resolution issues using Anycast IP addresses 52.119.40.100 and 103.80.5.100 are requested to forward their DNS traffic to our fully operational network infrastructure using Anycast IP addresses 52.119.41.100 and 103.80.6.100.

  4. identified Mar 30, 2025, 08:12 PM UTC

    We have implemented a workaround for the affected Anycast IP addresses 52.119.40.100 and 103.80.5.100, and the DNS traffic has been redirected to the nearest location. Additionally, we identified NIOS-X servers that are connected to Infoblox Cloud (https://portal.infoblox.com) using the Anycast IP addresses mentioned above, may observe an unhealthy status. All other services remain unaffected, and there will be no impact on protocol services on these servers.

  5. monitoring Mar 30, 2025, 10:13 PM UTC

    We have implemented a fix for the affected Anycast IP addresses 52.119.40.100 and 103.80.5.100, and DNS traffic is now being resolved through the respective PoP sites. Additionally, the NIOS-X servers that were reporting unhealthy state connecting to the Portal now should be connected and showing healthy status.

  6. resolved Mar 30, 2025, 10:40 PM UTC

    This incident has been resolved.