VPN Link Up, NetVM set to VpnVM but AppVMs still don't have net access?

[gaikokuji...@gmail.com]

Hi, with quite a bit of help (thanks again) I was able to setup a VpnVM and have it work perferctly as a NetVM for AppVMs with KDE as my desktop env. I then backed up (zipped) the /rw/config dir and reinstalled 3.1 with just xfce, recreated the VpnVM and put all the needed vpn files from the config dir/sub-dir in thier place. I set an AppVM to use the VpnVM as a NetVM and started up the VpnVM, it seems to start up fine, I get the little notification that the VPN is up, but I can't connect to anything. I have also tried, instead of using backed up config file to just recreate everything from scratch, got the VPN notification/confirmation that its up and running, but still couldn't access the net. I can access the net when using the firewall as the netvm though? I am not sure how to diagnose this further, thoughts suggestions would really be apprecaited!

[Chris Laprise]

On 07/13/2016 12:36 PM, gaikokuji...@gmail.com wrote:
> Hi, with quite a bit of help (thanks again) I was able to setup a VpnVM and have it work perferctly as a NetVM for AppVMs with KDE as my desktop env. I then backed up (zipped) the /rw/config dir and reinstalled 3.1 with just xfce, recreated the VpnVM and put all the needed vpn files from the config dir/sub-dir in thier place. I set an AppVM to use the VpnVM as a NetVM and started up the VpnVM, it seems to start up fine, I get the little notification that the VPN is up, but I can't connect to anything. I have also tried, instead of using backed up config file to just recreate everything from scratch, got the VPN notification/confirmation that its up and running, but still couldn't access the net. I can access the net when using the firewall as the netvm though? I am not sure how to diagnose this further, thoughts suggestions would really be apprecaited!

I would start troubleshooting by opening a CLI in the appvm, and also in
sys-firewall: In each CLI try pinging a known server using domain name
(first) and then IP address (second). This should tell you whether the
blockage is complete or only DNS related.

Chris

[gaikokuji...@gmail.com]

ah that was a good idea. Well it seems to be DNS related in the VpnVM as I am able to ping a site/ip address from the firewallVM and ping an IP from in the VpnVM but not a site name. I am not sure why that would be though since as far as I can tell everything is the same as it was in the previous setup (obviously not I guess). It seems to netmanager is setting a DNS other than what my VPN provider normally sets? I tried manually editing the /etc/resolv.conf file but that didn't seem to help (automatically generated?) so am not sure where to go from here?

[Chris Laprise]

Hopefully you are not using Network Manager to connect... That would
only interfere with the script-based method described in the doc.

The qubes-vpn-handler ignores resolv.conf, but you can edit the latter
to test domain names within the vpn vm. Remember, in the vpn vm you will
need to prefix your ping commands with 'sudo sg qvpn -c' to get past the
firewall.

If you do this and pinging site names works (in the vpn vm), then the
problem may be with DNS dnat rules. After connecting, you can check
those with 'sudo iptables -L -v -t nat'. You should see the appropriate
DNS server IPs there (4 rules).

Chris

[gaikokuji...@gmail.com]

Thanks for that. While I could have sworn that the ping wnet through (pinged mult times) when I pinged an IP from the VpnVM this time it doesn't seem to be going through, or it pings once then hangs, eg:

[user@VPN ~]$ sudo sg qvpn -c 'ping 216.218.239.2'
PING 216.218.239.2 (216.218.239.2) 56(84) bytes of data.
^C
--- 216.218.239.2 ping statistics ---
0 packets transmitted, 0 received

[user@VPN ~]$

I tried the 'sudo iptables -L -v -t nat' anyway and to be honest I am not sure I understand the output:

[user@VPN ~]$ sudo iptables -L -v -t nat
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 PR-QBS all -- any any anywhere anywhere
0 0 PR-QBS-SERVICES all -- any any anywhere anywhere

Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination

Chain OUTPUT (policy ACCEPT 432 packets, 30668 bytes)
pkts bytes target prot opt in out source destination

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all -- any vif+ anywhere anywhere
3 192 ACCEPT all -- any lo anywhere anywhere
12 812 MASQUERADE all -- any any anywhere anywhere

Chain PR-QBS (1 references)
pkts bytes target prot opt in out source destination
0 0 DNAT udp -- any any anywhere 10.137.4.1 udp dpt:domain to:10.137.2.1
0 0 DNAT tcp -- any any anywhere 10.137.4.1 tcp dpt:domain to:10.137.2.1
0 0 DNAT udp -- any any anywhere 10.137.4.254 udp dpt:domain to:10.137.2.254
0 0 DNAT tcp -- any any anywhere 10.137.4.254 tcp dpt:domain to:10.137.2.254

Chain PR-QBS-SERVICES (1 references)
pkts bytes target prot opt in out source destination

[gaikokuji...@gmail.com]

Hi, I don't think I am using Network Manager to connect, that is I went only by the Qubes VPN wiki but while trying to diag the problem I read about /etc/resolv.conf in some other doc while searching so thought I'd try (obviously no luck).

As for the sudo sg qvpn -c ping whateversite, does returning one thing back and hanging count for anything? I am thinking not as I am not able to connect to the net via the VpnVM.

Any thoughts on the DNS dnat rules?

[Chris Laprise]

Pinging from my vpn vm is probably the same as yours, now that I've
checked it: I get a DNS response but the pings themselves aren't permitted.

I think the real problem is shown in your PR-QBS chain above. You see
that the 'to' addresses on the right are still pointing to a Qubes
internal subnet '10.137.x.x'. Something about the DHCP fetching of your
DNS servers or the way qubes-vpn-handler.sh is executing is not working.
You can verify this by taking the IP address for 'whateversite' and
pinging it from your appvm (connected to vpn vm)... that should work
even though DNS doesn't.

Cause of the problem should be a misconfigured .ovpn (the 3 lines for
scripting) or the qubes-vpn-handler.sh script itself can't execute
because the execute flag is not set, or the shebang at the start was
left out, etc.

Chris

[gaikokuji...@gmail.com]

well you are right about being able to ping an IP from the appvm that is connected to the vpnvm, it works fine.

As for the misconfigured .opvn I can't make heads or tails of that as the first time I just used the exact same file that I had backed up, I rechecked it and I think its ok (I also got a new pre-configured one from my vpn provider, c/p the needed edits in, and still get the same error). I checked the permissions user of the two files and I think they are ok?

-rw-r--r-- 1 root root 423 Jul 21 21:28 openvpn-client.ovpn
-rwxr-xr-x 1 root root 1089 Jul 10 21:15 qubes-vpn-handler.sh

I didn't quite follow you about the shebang? What parts at the begining do you think might have been left out? Are you refering to the configuration of the VM when I was creating it? (like setting as a proxyvm etc?)

[gaikokuji...@gmail.com]

The last three lines you refered to, of the .ovpn, I believe I added as the Qubes VPN doc instructed, anyway I just c/p'd from the .ovpn I have:

script-security 2
up 'qubes-vpn-handler.sh up'
down 'qubes-vpn-handler.sh down'

Is that what you were referring to?

[Chris Laprise]

The shebang refers to the '#!/bin/bash' at the start of the script; that
is required for it to run.

> The last three lines you refered to, of the .ovpn, I believe I added as the Qubes VPN doc instructed, anyway I just c/p'd from the .ovpn I have:
>
> script-security 2
> up 'qubes-vpn-handler.sh up'
> down 'qubes-vpn-handler.sh down'
>
> Is that what you were referring to?

Yes.

Something else you can try is to bypass the DHCP stuff and add the DNS
server manually in your .ovpn with a line like this:
setenv vpn_dns 'X.X.X.X'

Replace X's with DNS server address.

Then when you connect and list your nat table again, you should see the
DNS IP there.

Chris

[gaikokuji...@gmail.com]

Ahhhh. Ok. Well I checked and I had forgotten to remove the origonal #!/bin/sh but it was the same file I had used before that had worked? Anyway, I edited it and now only one line and it is bash. But still can't access anything other than direct ip addresses via the appvm that is using the vpnvm?

>
> > The last three lines you refered to, of the .ovpn, I believe I added as the Qubes VPN doc instructed, anyway I just c/p'd from the .ovpn I have:
> >
> > script-security 2
> > up 'qubes-vpn-handler.sh up'
> > down 'qubes-vpn-handler.sh down'
> >
> > Is that what you were referring to?
>
> Yes.
>
> Something else you can try is to bypass the DHCP stuff and add the DNS
> server manually in your .ovpn with a line like this:
> setenv vpn_dns 'X.X.X.X'
>
> Replace X's with DNS server address.

I tried this next, added both like

setenv vpn_dns 'X.X.X.X X.X.X.X'

(tried without quotes too) but still no go. I then noticed that there was a commented line in the qubes-vpn-handler.sh script so I added that line in that script and took it out of the ovpn file, still not able to ping non ip addresses...

>
> Then when you connect and list your nat table again, you should see the
> DNS IP there.
>
> Chris

and both times (restarting vpnvm/appvm) the new DNS didn't show up when i tried to list the nat tables?

I would have thought manually putting in the DNS would have been sufficent?

[gaikokuji...@gmail.com]

Should the openvpn-client.ovpn file have the #!/bin/bash in it as well? I have run out of ideas. I have also noticed that the VPN connects (the little notification at the top of the screen) then occasionally disconnects and reconnects again fairly quickly.

[Chris Laprise]

I thought I'd let you know another user was having the same symptoms (IP
access but no DNS) because Network Manager was running in the vpn vm...
https://groups.google.com/d/msgid/qubes-users/f35d90b9-2632-4f91-afff-3e1f8ac26302%40googlegroups.com

If you had enabled Network Manager in that vm you should disable it.
Other possible workarounds are putting a 7sec delay in handler script,
or renaming the qubes-setup-dnat-to-ns file before openvpn is run (see
linked thread for details).

Chris