Resolved after 48 minutes, on July 10, 2025.
At 08:35 UTC we detected an issue with network connectivity on 2 nodes that were part of DNS and traffic routing infrastructure which caused disruption. Further investigation revealed that on 2 of our nodes networking was unstable - e.g. packet loss, no stable DNAT and so on. The reason was identified as the overlay networking \(flannel\) which wasn't correctly set the DNAT/SNAT rules on the pods running on those nodes The monitors have been properly triggered, and the offending components identified and restarted. 9:23 UTC - all nodes were fully operational. Incident was closed.
Update timeline
- resolved Jul 10, 2025, 09:39 AM UTC
There was an internal networking issue which briefly impact our web tools, and our API
- postmortem Jul 10, 2025, 09:40 AM UTC
At 08:35 UTC we detected an issue with network connectivity on 2 nodes that were part of DNS and traffic routing infrastructure which caused disruption. Further investigation revealed that on 2 of our nodes networking was unstable - e.g. packet loss, no stable DNAT and so on. The reason was identified as the overlay networking \(flannel\) which wasn't correctly set the DNAT/SNAT rules on the pods running on those nodes The monitors have been properly triggered, and the offending components identified and restarted. 9:23 UTC - all nodes were fully operational. Incident was closed.
Your product runs on more than Red Sift UK
Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.
Amazon Web Services