US - Central - (Minneapolis) - Upstream Routing Issues

Update timeline

1. investigating Jul 17, 2020, 09:31 PM UTC

   We are seeing spotty reports from 1 of our 3 external monitoring systems that there maybe routing issues across one of our Minneapolis data centers uplink providers. We are not seeing any web site accessibility issues being reported by random URL monitoring, but are seeing it at a lower network level. But this may show as degraded performance or network latency for some clients depending on how you are routed to this data center. We are working with our data center provider at the moment to identify and possibly route around the issue shortly if it continues. We will update this incident as we have more details.

2. resolved Jul 17, 2020, 09:43 PM UTC

   Further investigation this looks to have only been affecting a limited amount of IPv6 traffic inbound to the data center, and not impacting IPv4 traffic at all. We are still monitoring things as always, but will be setting this issue now to be resolved as all monitoring is now showing Green from our external monitoring system(s). There was at the same time issues going on with Cloudflare.com's DNS services, which impacted a large section of the Internet too. Which may have lead to some seeing issues with their sites hosted with us but not directly related to any of our direct services or data centers. https://www.cloudflarestatus.com/incidents/b888fyhbygb8 Their bad route announcement may have been a side cause of our IPv6 issues above if other routers picked up the bad route leak from Cloudflare.