Dns-over-TLS in sys-vpn. Is it possible? How?

[qubeslover]

Dear qubes users,
I wish you a good Sunday.

I'd like to use DoT on my qubes laptop. However, I am not sure how to do. I have followed a couple of pretty straightforward tutorials (https://www.techrepublic.com/article/how-to-use-dns-over-tls-on-ubuntu-linux/ and https://techrevelations.de/2019/01/11/encrypted-dns-and-how-to-use-it-in-linux/), installed stubby and configured NetworkManager - /etc/resolv.conf properly in sys-net.

Stubby connects to its default DoT servers and I can ping google from sys-net. However, I can't resolve addresses from other qubes (like sys-firewall etc). Has somebody managed to use DoT in Qubes? Which documents should I read in order to understand how networking, routing and name resolution work in QubesOS so that I can use DoT?

Sent with ProtonMail Secure Email.

[Chris Laprise]

Hi,

The vpn doc (step 3) has a good example of setting up DNS for a VPN
"proxy VM": The iptables nat/PR-QBS chain must be populated with dnat
rules for your DNS ips.

(A proxy VM is just like sys-firewall: Its an appVM created with the
'provides network' option set and acts like a router.)

https://www.qubes-os.org/doc/vpn/#set-up-a-proxyvm-as-a-vpn-gateway-using-iptables-and-cli-scripts

A version of this with more automatic setup is here:

https://github.com/tasket/Qubes-vpn-support

A shortcut you can take to setting up iptables for DNS is to populate
/etc/resolv.conf and then run '/usr/lib/qubes/qubes-setup-dnat-to-ns'.
This should configure the nat/PR-QBS chain with the DNS addresses you set.

A final note: There doesn't seem to be much demand for DoT over a VPN, I
think because VPN providers usually have their own DNS servers which are
protected by the VPN protocol. Something like DoT becomes useful only
when your link is generally insecure or you need to use a third-party
DNS for some other reason (i.e. you set up your own VPN server but not a
DNS server to go with it).

--

Chris Laprise, tas...@posteo.net
https://github.com/tasket
https://twitter.com/ttaskett
PGP: BEE2 20C5 356E 764A 73EB 4AB3 1DC4 D106 F07F 1886

[qubeslover]

Dear tasket,
today here is so hot that I feel like I am drunk. I typed the wrong title. The topic actually was

"Dns-over-TLS in *sys-net*. Is it possible? How?"

Obviously, as you correctly (and politely) pointed out, it doesn't make sense at all to run DoT over VPN. Actually, I want to run DoT in sys-net since my link is insecure.

Apologies for mistake. Suggestions are still appreciated.

Off Topic P.S: I use and love your scripts and extensions for Qubes. You made my life much easier. Look forward to test sparsebak once encryption will be deployed into it.

Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Chris Laprise,tas...@posteo.net

[Chris Laprise]

On 6/30/19 2:46 PM, 'qubeslover' via qubes-users wrote:
>
> Dear tasket,
> today here is so hot that I feel like I am drunk. I typed the wrong title. The topic actually was
>
> "Dns-over-TLS in *sys-net*. Is it possible? How?"
>
> Obviously, as you correctly (and politely) pointed out, it doesn't make sense at all to run DoT over VPN. Actually, I want to run DoT in sys-net since my link is insecure.
>
> Apologies for mistake. Suggestions are still appreciated.
>
> Off Topic P.S: I use and love your scripts and extensions for Qubes. You made my life much easier. Look forward to test sparsebak once encryption will be deployed into it.

Cool. Then this part still applies in sys-net:

>> A shortcut you can take to setting up iptables for DNS is to populate
>> /etc/resolv.conf and then run '/usr/lib/qubes/qubes-setup-dnat-to-ns'.
>> This should configure the nat/PR-QBS chain with the DNS addresses you set.

So check that your DoT setup is updating /etc/resolv.conf, then run
'/usr/lib/qubes/qubes-setup-dnat-to-ns'.

--

Chris Laprise, tas...@posteo.net

[Chris Laprise]

On 6/30/19 4:10 PM, Chris Laprise wrote:
>>> A shortcut you can take to setting up iptables for DNS is to populate
>>> /etc/resolv.conf and then run '/usr/lib/qubes/qubes-setup-dnat-to-ns'.
>>> This should configure the nat/PR-QBS chain with the DNS addresses you
>>> set.
>
> So check that your DoT setup is updating /etc/resolv.conf, then run
> '/usr/lib/qubes/qubes-setup-dnat-to-ns'.

Additional thought: The sys-net VM may not be the best place to secure
any data, DNS included. Putting DoT in sys-firewall or similar proxyVM
(and using qubes-setup-dnat-to-ns there) would be a better choice and
has a fair chance of working.

There is also a chance that configuring DoT to run in your AppVMs,
instead, could work and without any special Qubes steps.

[qubeslover]

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

On Sunday, June 30, 2019 10:36 PM, Chris Laprise <tas...@posteo.net> wrote:

> On 6/30/19 4:10 PM, Chris Laprise wrote:
>
> > > > A shortcut you can take to setting up iptables for DNS is to populate
> > > > /etc/resolv.conf and then run '/usr/lib/qubes/qubes-setup-dnat-to-ns'.
> > > > This should configure the nat/PR-QBS chain with the DNS addresses you
> > > > set.
> >
> > So check that your DoT setup is updating /etc/resolv.conf, then run
> > '/usr/lib/qubes/qubes-setup-dnat-to-ns'.

Thanks for you suggestion. Apparently, it does not work in sys-net.

Stubby is up, working and connected to its default DoT providers (as lsof -i asserts):


COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
stubby 534 stubby 3u IPv4 17946 0t0 UDP localhost:domain
stubby 534 stubby 4u IPv4 17947 0t0 TCP localhost:domain (LISTEN)
stubby 534 stubby 5u IPv6 17948 0t0 UDP localhost:domain
stubby 534 stubby 6u IPv6 17949 0t0 TCP localhost:domain (LISTEN)
stubby 534 stubby 7u IPv4 35444 0t0 TCP sys-net:46006->145.100.185.16:domain-s (ESTABLISHED)
stubby 534 stubby 8u IPv4 35447 0t0 TCP sys-net:45550->getdnsapi.net:domain-s (ESTABLISHED)
NetworkMa 564 root 17u IPv4 31022 0t0 UDP sys-net:bootpc
systemd-r 647 systemd-resolve 11u IPv4 19350 0t0 UDP *:hostmon
systemd-r 647 systemd-resolve 12u IPv4 19351 0t0 TCP *:hostmon (LISTEN)
systemd-r 647 systemd-resolve 13u IPv6 19353 0t0 UDP *:hostmon
systemd-r 647 systemd-resolve 14u IPv6 19354 0t0 TCP *:hostmon (LISTEN)
systemd-r 647 systemd-resolve 16u IPv4 19358 0t0 UDP 127.0.0.53:domain
systemd-r 647 systemd-resolve 17u IPv4 19359 0t0 TCP 127.0.0.53:domain (LISTEN)
tinyproxy 1547 tinyproxy 4u IPv4 32068 0t0 TCP *:us-cli (LISTEN)
tinyproxy 1547 tinyproxy 5u IPv6 32069 0t0 TCP *:us-cli (LISTEN)
tinyproxy 1548 tinyproxy 4u IPv4 32068 0t0 TCP *:us-cli (LISTEN)
tinyproxy 1548 tinyproxy 5u IPv6 32069 0t0 TCP *:us-cli (LISTEN)
tinyproxy 1549 tinyproxy 4u IPv4 32068 0t0 TCP *:us-cli (LISTEN)


Also, nano claims that everything is right in /etc/resolv.conf

# Generated by NetworkManager
nameserver 127.0.0.1
nameserver ::1


As root, I run /usr/lib/qubes/qubes-setup-dnat-to-ns . Everything looks fine.

I can ping the outside world but sys-net does not receive any request from my qubes :-(

> Additional thought: The sys-net VM may not be the best place to secure
> any data, DNS included. Putting DoT in sys-firewall or similar proxyVM
> (and using qubes-setup-dnat-to-ns there) would be a better choice and
> has a fair chance of working.

OK, will try tomorrow with sys-firewall and see what happens.

[qubeslover]

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

> Generated by NetworkManager
>
> ============================

>
> nameserver 127.0.0.1
> nameserver ::1
>
> As root, I run /usr/lib/qubes/qubes-setup-dnat-to-ns . Everything looks fine.
>
> I can ping the outside world but sys-net does not receive any request from my qubes :-(
>
> > Additional thought: The sys-net VM may not be the best place to secure
> > any data, DNS included. Putting DoT in sys-firewall or similar proxyVM
> > (and using qubes-setup-dnat-to-ns there) would be a better choice and
> > has a fair chance of working.
>
> OK, will try tomorrow with sys-firewall and see what happens.
>

Hello,

I tried but without results.

1. dnf install getdns-stubby in fedora-30-firewall (template).

2. servicectl enable stubby in fedora-30-firewall.

3. Shutdown fedora-30-firewall.

4. Restart sys-firewall

4. Sudo nano /etc/resolv.conf and change nameserver in 127.0.0.1 and ::1

5. Run /usr/lib/qubes/qubes-setup-dnat-to-ns as root.

I can ping the outside world and sys-firewall can resolve hostnames. However, the qubes behind it can't.

For sure, I am messing up somewhere. It is a sin: I would like to have a sys-dns qube running DoT or DoH.

Thanks a lot for your attention, interest and help. Again, very much appreciated.

[Chris Laprise]

On 7/1/19 3:40 PM, 'qubeslover' via qubes-users wrote:
> Hello,
>
> I tried but without results.
>
> 1. dnf install getdns-stubby in fedora-30-firewall (template).
>
> 2. servicectl enable stubby in fedora-30-firewall.
>
> 3. Shutdown fedora-30-firewall.
>
> 4. Restart sys-firewall
>
> 4. Sudo nano /etc/resolv.conf and change nameserver in 127.0.0.1 and ::1
>
> 5. Run /usr/lib/qubes/qubes-setup-dnat-to-ns as root.
>
> I can ping the outside world and sys-firewall can resolve hostnames. However, the qubes behind it can't.

Hmmm. I hate to keep tossing suggestions at you without having tried DoT
myself (though I hope to make time for it in the next couple weeks).

But... if 127.0.0.1/localhost is the dnat target, then the INPUT chain
comes into the picture. By default, Qubes configures INPUT to reject any
new requests (packets that don't satisfy 'related' or 'established'
conditions). As a quick workaround, you could try allowing DNS packets
in sys-firewall:

iptables -I INPUT -p udp --dport 53 -j ACCEPT
iptables -I INPUT -p tcp --dport 53 -j ACCEPT

>
> For sure, I am messing up somewhere. It is a sin: I would like to have a sys-dns qube running DoT or DoH.
>
> Thanks a lot for your attention, interest and help. Again, very much appreciated.
>

[Sphere]

With my experience of using DNSCrypt I actually think that Qubes' has some unique way of handling DNS queries given how the nameservers automatically put into /etc/resolv.conf are on a different subnet.

I actually think there must be some sort of bind or unbound being ran in there that resolves all the DNS queries for you by using sys-net or your netvm as a proxy.

In order to make a sys-dns qube or to turn any other qube into a sys-dns qube you must ensure that it is listening on port 53 UDP for any DNS queries.

This command alone given by Chris should be enough.

iptables -I INPUT -p udp --dport 53 -j ACCEPT

Afterwards you should change your /etc/resolv.conf to the IP address of your sys-dns qube. The IP address can be found out using Qubes Manager and try to ping that ip address first to verify if it is reachable by your AppVM in the first place.

If your sys-dns qube is not your sys-net or netvm then you should ensure that TCP port 853 outbound is allowed through if your firewall rules do not explicitly allow all outbound (all outbound is allowed by default for each qube)

(In dom0 terminal)
qvm-firewall [sys-firewall or/and sys-dns] add action=accept proto=tcp dstports=853 --before 0

If this doesn't solve it then it may be best to provide us with some logs of your stubby

[qubeslover]

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

Hi both,
thanks for your suggestions. I am a kind of busy for a couple of days. Once things get better I will try to set up a sys-dns qube running DoT following your indications and write a report for the mailing list.

Cheers

[Sphere]

You're welcome and good luck!
In any case, I was reminded that any sort of communication between non-interconnected qubes are not allowed. So even if both of your AppVM qubes and sys-dns qube are connected to sys-firewall then they won't be able to communicate with each other by default. Additional iptables rules must be added to allow it according to what's written here:
https://www.qubes-os.org/doc/firewall/#enabling-networking-between-two-qubes

[qubeslover]

Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐

Hello! Here I am again as promised.
In summary: I managed to create a sys-dns qube running DoT. Long story short, it is far from usable. Here are the steps I followed.

0. qvm-clone debian-10-minimal d10-minimal-dns.

1. Create a sys-dns qube which provides network and is based on d10-minimal-dns. This qube is behind sys-firewall.

2. qvm-run -u root d10-minimal-dns 'apt install qubes-core-agent-networking stubby'

3. In d10-minimal-dns 'nano /etc/stubby/stubby.yml' and add the following option >

listen_addresses:
- 127.0.0.1
- 0::1
- 10.137.0.xx # this is sys-dns IP address.


4. In d10-minimal-dns 'nano /etc/resolv.conf >
nameserver 127.0.0.1
namerserver ::1

5. qvm-shutdown d10-minimal-dns

6. qvm-start sys-dns

7. In sys-dns 'nano /rw/config/rc.local' >

iptables -I INPUT -p udp --dport 53 -j ACCEPT

iptables -I INPUT -p tcp --dport 53 -j ACCEPT

8. qvm-shutdown sys-dns

9. Set sys-dns as the network qube of a random app qubes (i.e. 'firefox')

firefox => sys-dns => sys-firewall => sys-net

10. In firefox 'nano /etc/resolv.conf' >
nameserver 10.137.xx # this is sys-dns IP address.

Check with dnsleaktest.com: DoT is working fine and firefox is resolving with the standard stubby provider.

Until step 9 every step is easily doable. However step 10 is kind of issue. Without step 10, the qube behind sys-dns is using the DNS of my Internet provider in order to resolv any address. I can't change resolv.conf everytime I open a qube, nor I think is a good idea to change resolv.conf in the template.

Thanks for any suggestions. I am just trying to find a suitable way to run DoT on Qubes.