Qubes VPN routing

[Supraleiter]

Hello Guys,

I have a problem concerning ipsec routing in qubes.
After setup of a network providing app-vm (former “Proxy-VM”) with a
strongswan client, I try to connect to a publicly available ikev2 vpn
server over WAN. The client tells me that a connection is established
and everything seems to work fine, but when trying to ping any host in
the private subnet, nothing happens.
I already tried to add following suggested iptable rules in proxyVM
(despite the fact that I believe it is not necessary for the client but
for a gateway):

iptables -t nat -A POSTROUTING -s 10.137.0.0/24 -o eth0 -m policy --dir
out --pol ipsec -j ACCEPT
iptables -t nat -A POSTROUTING -s 10.137.0.0/24 -o eth0 -j MASQUERADE
iptables -t nat -I POSTROUTING -m policy --pol ipsec --dir out -j ACCEPT

I also tried to use “sys-net” VM as network provider as alternative to
“sys-firewall” to circumvent any firewall related problem, but it does
not work, too.
Are there any other (maybe Qubes specific) settings I have to set up to
make the proxyVM send traffic over ipsec tunnel (strongswan does not
install a virtual network device)?

Several config files and shell outputs are given below.
Thank you very much.

Sincerely,
Supraleiter

Strongswan config file:
conn vpn-ikev2
auto=route
ike=aes256-sha256-sha512-modp4096!
esp=aes256-sha256-sha512-modp4096!
right=217.86.xxx.xxx
rightid="O=Foo GmbH,CN=217.86.xxx.xxx"
rightsubnet=0.0.0.0/0
rightauth=pubkey
rightca=217.86.xxx.xxx
rightcert=/etc/ipsec.d/certs/Foo_CA_IME.crt
leftid="O=Foo GmbH,CN=Foo VPN USER"
leftsourceip=%config
leftauth=pubkey
leftca="O=Foo GmbH,CN=Foo CA IME"
leftcert=/etc/ipsec.d/certs/Foo_VPN_User.crt


Initialisation of vpn connection:
IKE_SA vpn-ikev2[1] established between 10.137.0.10[O=Foo GmbH, CN=Foo
USER]...217.86.xxx.xxx[O=Foo GmbH, CN=217.86.xxx.xxx]
scheduling reauthentication in 9961s
maximum IKE_SA lifetime 10501s
installing DNS server 192.168.10.1 to /etc/resolv.conf
installing DNS server 192.168.10.52 to /etc/resolv.conf
installing new virtual IP 192.168.10.205
selected proposal: ESP:AES_CBC_256/HMAC_SHA2_512_256/NO_EXT_SEQ
CHILD_SA vpn-ikev2{2} established with SPIs cc3c6e40_i 0b663ca1_o and TS
192.168.10.205/32 === 0.0.0.0/0
connection 'vpn-ikev2' established successfully

iptables (proxy vm):
sudo iptables --list-rules
-P INPUT DROP
-P FORWARD DROP
-P OUTPUT ACCEPT
-N QBS-FORWARD
-A INPUT -i vif+ -p udp -m udp --dport 68 -j DROP
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i vif+ -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -i vif+ -j REJECT --reject-with icmp-host-prohibited
-A INPUT -j DROP
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -j QBS-FORWARD
-A FORWARD -i vif+ -o vif+ -j DROP
-A FORWARD -i vif+ -j ACCEPT
-A FORWARD -j DROP

iptables nat (proxy vm):
sudo iptables -t nat --list-rules
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P POSTROUTING ACCEPT
-P OUTPUT ACCEPT
-N PR-QBS
-N PR-QBS-SERVICES
-A PREROUTING -j PR-QBS
-A PREROUTING -j PR-QBS-SERVICES
-A POSTROUTING -m policy --dir out --pol ipsec -j ACCEPT
-A POSTROUTING -o vif+ -j ACCEPT
-A POSTROUTING -o lo -j ACCEPT
-A POSTROUTING -j MASQUERADE
-A POSTROUTING -s 10.137.0.0/24 -o eth0 -m policy --dir out --pol ipsec
-j ACCEPT
-A POSTROUTING -s 10.137.0.0/24 -o eth0 -j MASQUERADE
-A PR-QBS -d 10.139.1.1/32 -p udp -m udp --dport 53 -j DNAT
--to-destination 10.139.1.1
-A PR-QBS -d 10.139.1.1/32 -p tcp -m tcp --dport 53 -j DNAT
--to-destination 10.139.1.1
-A PR-QBS -d 10.139.1.2/32 -p udp -m udp --dport 53 -j DNAT
--to-destination 10.139.1.2
-A PR-QBS -d 10.139.1.2/32 -p tcp -m tcp --dport 53 -j DNAT
--to-destination 10.139.1.2

Ping:
ping 192.168.10.205
PING 192.168.10.205 (192.168.10.205) 56(84) bytes of data.
64 bytes from 192.168.10.205: icmp_seq=1 ttl=64 time=0.041 ms
64 bytes from 192.168.10.205: icmp_seq=2 ttl=64 time=0.041 ms
64 bytes from 192.168.10.205: icmp_seq=3 ttl=64 time=0.041 ms
64 bytes from 192.168.10.205: icmp_seq=4 ttl=64 time=0.044 ms
^C
--- 192.168.10.205 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 58ms
rtt min/avg/max/mdev = 0.041/0.041/0.044/0.008 ms

ping 192.168.10.1
PING 192.168.10.1 (192.168.10.1) 56(84) bytes of data.
^C
--- 192.168.10.1 ping statistics ---
7 packets transmitted, 0 received, 100% packet loss, time 188ms

ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=51 time=72.4 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=51 time=38.4 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=51 time=41.7 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=51 time=39.6 ms
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 8ms
rtt min/avg/max/mdev = 38.420/48.019/72.374/14.111 ms

[Chris Laprise]

On 11/5/19 8:41 AM, Supraleiter wrote:
> Hello Guys,
>
> I have a problem concerning ipsec routing in qubes.
> After setup of a network providing app-vm (former “Proxy-VM”) with a
> strongswan client, I try to connect to a publicly available ikev2 vpn
> server over WAN. The client tells me that a connection is established
> and everything seems to work fine, but when trying to ping any host in
> the private subnet, nothing happens.
> I already tried to add following suggested iptable rules in proxyVM
> (despite the fact that I believe it is not necessary for the client but
> for a gateway):
>
> iptables -t nat -A POSTROUTING -s 10.137.0.0/24 -o eth0 -m policy --dir
> out --pol ipsec -j ACCEPT
> iptables -t nat -A POSTROUTING -s 10.137.0.0/24 -o eth0 -j MASQUERADE
> iptables -t nat -I POSTROUTING -m policy --pol ipsec --dir out -j ACCEPT
>
> I also tried to use “sys-net” VM as network provider as alternative to
> “sys-firewall” to circumvent any firewall related problem, but it does
> not work, too.
> Are there any other (maybe Qubes specific) settings I have to set up to
> make the proxyVM send traffic over ipsec tunnel (strongswan does not
> install a virtual network device)?
>
> Several config files and shell outputs are given below.
> Thank you very much.

Hi,

I'm not familiar with IPsec routing policies, but have you checked the
routing table after connecting? Other VPN clients declare tunnel IPs as
gateways by default, and I'm wondering if a gateway route is what's
missing here.

Also make sure your initial ping attempts use the subnet's IP address
and not host names.

And after resolv.conf is modified, you should run
'/usr/lib/qubes/qubes-setup-dnat-to-ns' to setup nat for downstream VM DNS.

Finally, once you get the link working, you may want to setup an
anti-leak barrier in forwarding, such as what is shown in the Qubes
Network Manager VPN doc (i.e. block all forwarding to eth0).

--

Chris Laprise, tas...@posteo.net
https://github.com/tasket
https://twitter.com/ttaskett
PGP: BEE2 20C5 356E 764A 73EB 4AB3 1DC4 D106 F07F 1886

[Supraleiter]

Hey Chris,

Thanks for your reply. I checked presence of a gateway route with "ip
route get 192.168.10.0/24" and got the output
"192.168.10.1 dev eth0 src 10.137.0.10 uid 0" so I guess a route is set.
I also tried calling qubes-setup-dnat-to-ns and checked connection
again, but it seems like nothing happened. Do I have to run it on
proxyVM (as resolv.conf is only altered in this vm)?

I had another idea regarding network devices. If I list all devices with
ifconfig, only eth0 and lo are shown. Keeping in mind that every AppVM
has its own nat device, I looked up any net device with "cat
/proc/net/dev" and then another device "vif17.0" showed up. Does this
interfere with routing? Do I have to setup rules regarding this device?

Regards,
Supraleiter