External Network Route Leak causing Inspection Failures

Update timeline

1. investigating Jun 24, 2019, 10:41 AM UTC

   We are currently investigating this issue.

2. identified Jun 24, 2019, 11:00 AM UTC

   We have identified an issue in network connectivity which leads to communication failures between different components. We are continuing to investigate the root cause.

3. monitoring Jun 24, 2019, 11:07 AM UTC

   We have temporarily mitigated and are continuing to investigate the root cause. Retried inspections should run through normally now.

4. monitoring Jun 24, 2019, 11:22 AM UTC

   We have tracked the root cause down to an upstream issue with Cloudflare which we use https://www.cloudflarestatus.com/incidents/46z55mdhg0t5 For the moment, this issue is mitigated on our end. However, we are seeing errors for other domains (such as node.js) and believe this might be a wider Internet connectivity issue. As a result, inspections that install a node.js version are currently failing. We are further investigating how to minimize the impact this has on your inspections and we will keep you updated here.

5. monitoring Jun 24, 2019, 12:40 PM UTC

   We are continuing to work on ways to mitigate this issue as best as possible on our end. We will keep you updated.

6. monitoring Jun 24, 2019, 12:46 PM UTC

   The issue has been resolved upstream and we are seeing reduced error rates for inspections. We are continuing to monitor the situation and will update as necessary. If you have a failed inspection related to node.js, a retry should now run through normally.

7. resolved Jun 24, 2019, 01:13 PM UTC

   This incident has been resolved. We are continuing to monitor and re-open if further updates are needed.