Qubes 4 and VPN, client VMs cannot access Internet

[Eric S]

I have successfully created new netVM named 'sys-vpn' based on fedora-30, installed VPN functionality using qubes-vpn-support script, and have successfully connected to Internet with VPN's IP address, verified using browser in sys-vpn. My network connections look like this: 

     sys-vpn --> sys-firewall --> sys-net --> Internet

However, when I try to use another client VM with sys-vpn as netVM, the client is not able to connect to the Internet. Example network connection: 

     fedora-30 --> sys-vpn --> sys-firewall --> sys-net --> Internet (fedora-30 cannot access Internet).

I suspect this might be firewall rules, but am pretty noobish on how to troubleshoot and configure (all rules are default or configured as per qubes-vpn-support script). I have read the Firewall and VPN docs on qubes.org (firewall docs are a bit over my head), and scoured firewall and VPN threads on a number of discussion sites (reddit, qubes-users, stack overflow, etc.) to understand how to troubleshoot, I am simply at a loss for figuring out how to resolve.

I have attached graphic to illustrate problem. Any guidance and support greatly appreciated, thanks for assistance.

[Chris Laprise]

On 10/24/19 10:00 PM, Eric S wrote:
> I have successfully created new netVM named 'sys-vpn' based on
> fedora-30, installed VPN functionality using qubes-vpn-support

> <https://github.com/tasket/Qubes-vpn-support> script, and have

> successfully connected to Internet with VPN's IP address, verified using
> browser in sys-vpn. My network connections look like this:
>      sys-vpn --> sys-firewall --> sys-net --> Internet
>
> However, when I try to use another client VM with sys-vpn as netVM, the
> client is not able to connect to the Internet. Example network connection:
>      fedora-30 --> sys-vpn --> sys-firewall --> sys-net --> Internet
> (fedora-30 cannot access Internet).
>
> I suspect this might be firewall rules, but am pretty noobish on how to
> troubleshoot and configure (all rules are default or configured as per
> qubes-vpn-support script). I have read the Firewall and VPN docs on
> qubes.org (firewall docs are a bit over my head), and scoured firewall
> and VPN threads on a number of discussion sites (reddit, qubes-users,
> stack overflow, etc.) to understand how to troubleshoot, I am simply at
> a loss for figuring out how to resolve.
>
> I have attached graphic to illustrate problem. Any guidance and support
> greatly appreciated, thanks for assistance.

'fedora-30' would be the name of a template VM, not a regular app VM.
Templates are blocked from regular Internet access in Qubes.

If all you want fedora-30 to do is update or install software, it can be
done if an update proxy is added to the system (the existing update
proxy in sys-net can no longer see the template's requests bc its
traffic is encrypted by sys-vpn). This could be done by enabling the
Qubes service 'qubes-updates-proxy' for your sys-firewall-vpn VM.
Alternately, you could make the templates update directly by adding
'updates-proxy-setup' to their Qubes services tab and then un-checking
it (this has the effect of disabling the updates-proxy client).

A note about the firewall in qubes-vpn-support: If its configured
correctly with the example settings (using the 'vpn-handler-openvpn'
Qubes service) then you should not be able to browse Internet sites from
inside sys-vpn. Also, you should see a popup notification stating that
the VPN link is 'UP' when sys-vpn starts.

You can check on the VPN status in sys-vpn with 'sudo journalctl -u
qubes-vpn-handler'. You can also check firewall settings with 'sudo
iptables -L -v -t nat' and the 'Chain PR-QBS' should have ip addresses
pointing to your VPN provider's DNS server in the rightmost column
(traffic can appear to be blocked if this doesn't get set).

--

Chris Laprise, tas...@posteo.net
https://github.com/tasket
https://twitter.com/ttaskett
PGP: BEE2 20C5 356E 764A 73EB 4AB3 1DC4 D106 F07F 1886

[Eric S]

 

Thanks for quick response, see following replies.

'fedora-30' would be the name of a template VM, not a regular app VM.
Templates are blocked from regular Internet access in Qubes.

Sorry for the confusion. The actual app VM is named 'work' and is based on fedora-30 template. I also have two identical VMs named 'personal' and 'personal-vpn' based on ubuntu-18 template. The NetVM for 'personal' is sys-firewall, and I have full Internet access (i.e. 'ping 8.8.8.8' is success response). The NetVM for 'personal-vpn' is sys-vpn, and no Internet traffic goes through ('ping 8.8.8.8' is 100% packet loss).

 

If all you want fedora-30 to do is update or install software, it can be
done if an update proxy is added to the system (the existing update
proxy in sys-net can no longer see the template's requests bc its
traffic is encrypted by sys-vpn). This could be done by enabling the
Qubes service 'qubes-updates-proxy' for your sys-firewall-vpn VM.
Alternately, you could make the templates update directly by adding
'updates-proxy-setup' to their Qubes services tab and then un-checking
it (this has the effect of disabling the updates-proxy client).

Good to know, thanks. I did read this in the qubes documentation and had played around with it a bit on test VMs, but have not needed to perform any proxy updates as all the template updates are performing as expected, and I only need to restart my app VMs and net VMs to inherit software updates from the templates. I have not needed to add update proxy to any app VMs. 

A note about the firewall in qubes-vpn-support: If its configured
correctly with the example settings (using the 'vpn-handler-openvpn'
Qubes service) then you should not be able to browse Internet sites from
inside sys-vpn. Also, you should see a popup notification stating that
the VPN link is 'UP' when sys-vpn starts.

Great point. Initially I was having connection problems on sys-vpn. I was only able to get the popup notification and Internet access after I added 'vpn-handler-egress' service (I had already added 'vpn-handler-openvpn' when I created the VM).

You can check on the VPN status in sys-vpn with 'sudo journalctl -u
qubes-vpn-handler'. You can also check firewall settings with 'sudo
iptables -L -v -t nat' and the 'Chain PR-QBS' should have ip addresses
pointing to your VPN provider's DNS server in the rightmost column
(traffic can appear to be blocked if this doesn't get set).

I did use 'sudo journalctl -u qubes-vpn-handler' to troubleshoot problems when I first tried to install qubes-vpn-suport. On my first go-around I cloned sys-net and then installed the scripts as instructed. I had failures reported in journalctl (sorry, I cannot remember what the errors were), so I ended up deleting that VM and instead of cloning off sys-net, I created a new VM based on fedora-30, added the 'vpn-handler-openvpn' service, and installed qubes-vpn-support. This time no failures reported from journalctl. 

I verified that my VPN provider's DNS servers are listed correctly in iptables.

Open to additional suggestions or insights, will perform any commands you request for details. Thanks for your help

[Chris Laprise]

On 10/29/19 5:36 PM, Eric S wrote:
> Great point. Initially I was having connection problems on sys-vpn. I
> was only able to get the popup notification and Internet access after I
> added 'vpn-handler-egress' service (I had already added
> 'vpn-handler-openvpn' when I created the VM).
>
>
> You can check on the VPN status in sys-vpn with 'sudo journalctl -u
> qubes-vpn-handler'. You can also check firewall settings with 'sudo
> iptables -L -v -t nat' and the 'Chain PR-QBS' should have ip addresses
> pointing to your VPN provider's DNS server in the rightmost column
> (traffic can appear to be blocked if this doesn't get set).
>
> I did use 'sudo journalctl -u qubes-vpn-handler' to troubleshoot
> problems when I first tried to install qubes-vpn-suport. On my first
> go-around I cloned sys-net and then installed the scripts as instructed.
> I had failures reported in journalctl (sorry, I cannot remember what the
> errors were), so I ended up deleting that VM and instead of cloning off
> sys-net, I created a new VM based on fedora-30, added the
> 'vpn-handler-openvpn' service, and installed qubes-vpn-support. This
> time no failures reported from journalctl.
>
> I verified that my VPN provider's DNS servers are listed correctly in
> iptables.
>
> Open to additional suggestions or insights, will perform any commands
> you request for details. Thanks for your help

Two red flags here are that you needed to use '-egress'... normally only
needed for a kernenl-based protocol like Wireguard. Are you using openvpn?

Also, I don't know how copying sys-net got into it... this can cause
conflicts that would block net access for some dependent VMs. There
shouldn't be any sys-net clones unless you know precisely what you're doing.

I just signed up with Mullvad bc I hadn't seen it tested recently. Its
working fine on my end with openvpn and using plain fedora-20 for
sys-vpn and the appvm. The arrangement is appvm->sys-vpn->sys-net.

[Chris Laprise]

Correction, that is supposed to be fedora-30. :)

[seshu]

Chris a question. I just installed mullvad on my template debian-10. Set it up in there following the mullvad instructions. This is the template for my sys-vpn. When I start up sys-vpn, the connect to the mullvad server I wanted is operational and things seem to be working fine. When I test using the https://am.i.mullvad.net link, it says I have a DNS leak

So a few questions.

1. in looking at the instructions for qubes-vpn-support, you have it set for use of openvpn or for the vpn software provider. So, if I use the mullvad app but follow the instructions for qubes-vpn-support, I"m not sure how to get it to use the config files that mullvad provides?  So, mullvad app shouldn't be used with the qubes-vpn-support, rather use only openvpn and get the mullvad config files and setup as the instructions specify?

2. Mullvad provides instructions for installing thier product on Qubes OS 4 This is just using openvpn though, but the instructions are different then you have for open-vpn-support, so I'm wondering what the difference is?

3. Which leaves me more confused, as there are 3 options now to install mullvad on Qubes 4. Use the mullvad app but I seem to have a dns leak? Use mullvad's instructions for installing on Qubes 4 which uses openvpn but instructions seem different then qubes-vpn-support? or go with qubes-vpn-support but get the mullvad config files?

I'm not sure what would be the advantage or disadvantage? And is there anyother way to test if I have a dns leak?

Thanks in advance!

[Chris Laprise]

On 11/3/19 4:16 PM, seshu wrote:
> Chris a question. I just installed mullvad on my template debian-10. Set
> it up in there following the mullvad instructions. This is the template
> for my sys-vpn. When I start up sys-vpn, the connect to the mullvad
> server I wanted is operational and things seem to be working fine. When
> I test using the https://am.i.mullvad.net link, it says I have a DNS leak

FWIW, if using the proprietary mullvad app, a standalone VM is probably
a better target for installation. A template is probably not a good
place to install it.

>
> So a few questions.
> 1. in looking at the instructions for qubes-vpn-support, you have it set
> for use of openvpn or for the vpn software provider. So, if I use the
> mullvad app but follow the instructions for qubes-vpn-support, I"m not
> sure how to get it to use the config files that mullvad provides?  So,
> mullvad app shouldn't be used with the qubes-vpn-support, rather use
> only openvpn and get the mullvad config files and setup as the
> instructions specify?

Right, you shouldn't mix qubes-vpn-support with a proprietary VPN app.

To use qubes-vpn-support with the Mullvad service, you need to use
Mullvad's config downloader page. The Readme has a section with VPN
provider links and one points to the Mullvad config dl page:

https://github.com/tasket/Qubes-vpn-support#locating-and-downloading-vpn-config-files

>
> 2. Mullvad provides instructions for installing thier product on Qubes

> OS 4 <https://mullvad.net/en/help/qubes-os-4-and-mullvad-vpn/> This is

> just using openvpn though, but the instructions are different then you
> have for open-vpn-support, so I'm wondering what the difference is?

Their approach is rather basic and requires the user to manually find
and hardcode IP addresses into their config. It is often easier for the
author to give users a long list of more complicated steps than to code
an automated solution. A better question might be why didn't Mullvad
integrate Qubes support into their own automated solution (their Linux app)?

In case this sounds a bit critical of Mullvad, I'm sure they have good
reasons. Its often easier to contribute money to support a project like
Qubes (as they have) than to code something specifically for an unusual
OS with a small user base.

>
> 3. Which leaves me more confused, as there are 3 options now to install
> mullvad on Qubes 4. Use the mullvad app but I seem to have a dns leak?
> Use mullvad's instructions for installing on Qubes 4 which uses openvpn
> but instructions seem different then qubes-vpn-support? or go with
> qubes-vpn-support but get the mullvad config files?
>
> I'm not sure what would be the advantage or disadvantage? And is there
> anyother way to test if I have a dns leak?

A secured VPN config has many moving parts, and (unlike Tor) the parts
tend to be regular OS features. Different people have different ways of
pulling those parts into a whole solution that suits a specific kind of
user and their use cases. Even just with Linux, users also have several
options (an app, Network Manager, and direct openvpn setup for example).

After I wrote the scripted portion of the Qubes-hosted vpn doc, where a
constraint was to have it all DIY and 'educational', it was obvious many
people were intimidated by that. So I made qubes-vpn-support to reduce
the fuss to a reasonable minimum: The configuration? Let the VPN
provider supply it. Preventing leaks? Automated. Addresses? Automated.
Setup? Create the VM, run 'install' and copy a file. For each issue in a
VPN config, I chose what is simplest for the user without locking them
into a specific VPN provider.

Obviously, I recommend qubes-vpn-support over the other options. And
looking at the Mullvad Qubes doc, I think qubes-vpn-support is safer to
use bc any change in IP addresses won't break the config and the
anti-leak features are more robust.

[seshu]

This is really helpful, thanks so much Chris. I will try the qubes-vpn-support and see how that works!