kube-proxy IPVS over IPSec

Lachlan Pease

unread,

Feb 2, 2019, 10:53:20 AM2/2/19

to kubernetes-sig-network

First off, apologies if this isn't the right place for this.

I'm currently prototyping a bare metal Kubernetes deployment where the control and data planes are encrypted by IPSec. Unfortunately, I can't get IPVS mode working at the same time as IPSec.

When using IPVS mode, ClusterIPs can't be connected to - any connection attempt results in a connect timeout.

I've traced the connection attempt and confirmed that the SYN is sent and the service responds with a SYN+ACK.

The SYN+ACK makes it through the sender's firewall, but is never delivered to the application.

I've debugged this as far as I can but I'm stumped, so I've attached a Vagrantfile in the hopes that someone with more networking knowledge can figure this out.

You can run "IPVS=0 vagrant up" to disable IPVS and/or "IPSEC=0 vagrant up" to disable IPSec - either one will cause the ClusterIP to work perfectly.

Any help would be appreciated, even just a pointer to more debugging tools.

Thanks,

Lachlan Pease

Vagrantfile

Ilya Dmitrichenko

unread,

Feb 4, 2019, 4:44:01 AM2/4/19

to Lachlan Pease, kubernetes-sig-network

Vagrant often causes problems to do with that default route points to NAT interface. If you have other platforms to test, please try those. Otherwise, there may exist workaround, but if you can test elsewhere please try.

--
You received this message because you are subscribed to the Google Groups "kubernetes-sig-network" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kubernetes-sig-ne...@googlegroups.com.
To post to this group, send email to kubernetes-...@googlegroups.com.
Visit this group at https://groups.google.com/group/kubernetes-sig-network.
For more options, visit https://groups.google.com/d/optout.

Lachlan Pease

unread,

Feb 4, 2019, 5:46:32 AM2/4/19

to Ilya Dmitrichenko, kubernetes-sig-network

Thanks for the suggestion,

I've retested my scenario in a simpler setup (2xUbuntu VMs and a router VM for internet access, all connected to the same network segment w/ single interfaces) with no change.

As before, the reply packet traverses the firewall but never gets delivered to the application.

I've attached the full routing tables and rules, along with the ipvsadm state and an iptables trace of the connection attempt, in case any of it helps.

Regards,

Lachlan

iptables trace.txt

routes.txt

ipvsadm.txt

Ahmed Sameh

unread,

Sep 27, 2021, 3:26:39 PM9/27/21

to kubernetes-sig-network

Hi,

I am facing the same issue using libreswan + calico ipvs, have you been able to solve it ?