RIPE Atlas controller problems

Update timeline

1. investigating Mar 11, 2026, 02:11 PM UTC

   A part the RIPE Atlas probe population is unable to connect to our infrastructure. We're investigating the issue.

2. identified Mar 11, 2026, 02:28 PM UTC

   We identified the root cause of the issue and applied a fix, probes are connecting again.

3. monitoring Mar 11, 2026, 03:42 PM UTC

   A fix has been implemented and we're monitoring the situation.

4. resolved Mar 12, 2026, 08:03 AM UTC

   This incident has been resolved. It was ultimately caused by an overly restrictive permission check in the component that serves as the entry point for probe (the "registration server"). In itself this was not a major problem and would have been detected and resolved before it affected the probe population. However, independently of this, and as part of our normal software development process, we rolled out upgrades to our controlling infrastructure, which caused most of the probes to disconnect and try to re-register with the system. This is normal and expected - but in this case the registration server was not ready to answer them, and as a result they were unable to immediately re-join.