Properly setup a qube dns cache server
nicholas roveda
I'd like to have an easy way to switch the DNS configurations from dom0, both via cli and GUI, maybe in the 'Qubes Global Settings' with another field in the 'System Defaults' section, that let me switch from 'sys-net' to other qubes.
The final goal is to make possible to specify a custom dnsVM backend based on tags and labels, for example to send all request from "trusted" VMs to a dnsVM with where a DNSCrypt is installed and all request from "untrusted" VMs to a dnsVM that apply a small set of filtering rules.
The problem is I don't know which configurations/files to change and how to make this configuration persist for a session or permanently, since I know ServiceVMs update dynamically the iptables rules, the nat table in particular, on interfaces UP and DOWN events.
Any ideas?
nicholas roveda
Of course Dom0 or a NetManagementVM (thanks to the new Admin API) has to store the DNS settings for the VMs with Networking and the easy way is to store only the infos about VMs that don't use the standars DNS (10.139.1.1 and 10.139.1.2).
So, adjustments have to be made to let the user specify a dnsVM in 'VM Settings', when Networking in not '(none)' and a default 'DNS Backend' field (new field in the GUI Qubes Global Settings) where the default value '(none)' or 'sys-net' corresponds to '10.139.1.1' and '10.139.1.2'.
Method a)
1) Every time a vif+ interface goes up in the FirewallVM (or the NetVM backend), this would have to run `qubes-setup-dnat-to-ns` like VMs that provide network do, but this time the script will retrieve the DNS through a rpc service instead of '/etc/resolv.conf'.
2) The rpc service will ask Dom0 (or the NetManagementVM) to update the net configurations, so Dom0 will retrive the list of current running VMs attached to the FirewallVM (or the NetVM backend) and provide it a list of VMs that don't use the default configurations and their the custom DNS backend IPs.
3) The FirewallVM (or the NetVM backend) will than update the iptables with secific DNAT of packets destinated to port 53 to specific IPs thanks to the infos received from Dom0 and maintain the general rule that DNAT all packets destinated to port 53 of '10.139.1.1' and '10.139.1.2' to the DNS specified in '/etc/resolv.conf'.
Method b)
Every time a qube is started from Dom0, Dom0 launch a rpc-service in its NetVM and passes to it any custom DNS configuration, if present, and the rpc service again, is the one responsible for executing a modified `qubes-setup-dnat-to-ns` to update the iptables.
So, the current 'sys-firewall' NAT table looks like this:
-N PR-QBS
-N PR-QBS-SERVICES
-A PREROUTING -j PR-QBS
-A PREROUTING -j PR-QBS-SERVICES
-A POSTROUTING -o vif+ -j ACCEPT
-A POSTROUTING -o lo -j ACCEPT
-A POSTROUTING -j MASQUERADE
-A PR-QBS -d 10.139.1.1/32 -p udp -m udp --dport 53 -j DNAT --to-destination 10.139.1.1
-A PR-QBS -d 10.139.1.1/32 -p tcp -m tcp --dport 53 -j DNAT --to-destination 10.139.1.1
-A PR-QBS -d 10.139.1.2/32 -p udp -m udp --dport 53 -j DNAT --to-destination 10.139.1.2
-A PR-QBS -d 10.139.1.2/32 -p tcp -m tcp --dport 53 -j DNAT --to-destination 10.139.1.2
And, the new 'sys-firewall' NAT table will look like this (with both methods):
-N PR-QBS
-N PR-QBS-SERVICES
-A PREROUTING -j PR-QBS
-A PREROUTING -j PR-QBS-SERVICES
-A POSTROUTING -o vif+ -j ACCEPT
-A POSTROUTING -o lo -j ACCEPT
-A POSTROUTING -j MASQUERADE
-A PR-QBS -i vifX -d 10.139.1.1/32 -p udp -m udp --dport 53 -j DNAT --to-destination <dnsVM-X-IP>
-A PR-QBS -i vifX -d 10.139.1.1/32 -p tcp -m tcp --dport 53 -j DNAT --to-destination <dnsVM-X-IP>
-A PR-QBS -i vifX -d 10.139.1.2/32 -p udp -m udp --dport 53 -j DNAT --to-destination <dnsVM-X-IP>
-A PR-QBS -i vifX -d 10.139.1.2/32 -p tcp -m tcp --dport 53 -j DNAT --to-destination <dnsVM-X-IP>
-A PR-QBS -i vifY -d 10.139.1.1/32 -p udp -m udp --dport 5353 -j DNAT --to-destination <dnsVM-Y-IP>
-A PR-QBS -i vifY -d 10.139.1.1/32 -p tcp -m tcp --dport 5353 -j DNAT --to-destination <dnsVM-Y-IP>
-A PR-QBS -i vifY -d 10.139.1.2/32 -p udp -m udp --dport 5354 -j DNAT --to-destination <dnsVM-Y-IP>
-A PR-QBS -i vifY -d 10.139.1.2/32 -p tcp -m tcp --dport 5354 -j DNAT --to-destination <dnsVM-Y-IP>
...
# Defaults
-A PR-QBS -d 10.139.1.1/32 -p udp -m udp --dport 53 -j DNAT --to-destination 10.139.1.1
-A PR-QBS -d 10.139.1.1/32 -p tcp -m tcp --dport 53 -j DNAT --to-destination 10.139.1.1
-A PR-QBS -d 10.139.1.2/32 -p udp -m udp --dport 53 -j DNAT --to-destination 10.139.1.2
-A PR-QBS -d 10.139.1.2/32 -p tcp -m tcp --dport 53 -j DNAT --to-destination 10.139.1.2
It seems to me that the first solution is more elegant and flexible, since it allows the FirewallVM to query other machines than Dom0 for networking configurations. That is the case described also in the Docs where, thanks to the new Admin API, a new ManagmmentVM can install a custom Template and manage all the AppVMs that use that Template and their settings.
ISSUES:
1) I haven't discovered yet, how retrive the vif backend interface name of a specific VM from Dom0 (querying the Xen DB I think), in order to pass to the FirewallVM (or the NetVM backend) the infos in this format: <vif_name> <DNS1> [<DNS2> ... <DNSX>].
Now, for testing purposed one could use the format: <VM_IP> <DNS1> [<DNS2> ... <DNSX>] and the iptables rules: `sudo iptables -t nat -A PR-QBS -s <VMX_IP> -d 10.139.1.1/32 -p udp -m udp --dport 53 -j DNAT --to-destination <dnsVM-X-IP>`
2) I don't know if I can use udev rules to or Fedora/Debian cutoms system launch a script that interact with Dom0 and launch `qubes-setup-dnat-to-ns`.
I've found the file '/etc/udev/rules.d/99-qubes-network.rules' wich cointains the following line:
SUBSYSTEMS=="xen", KERNEL=="eth*", ACTION=="add", ENV{ID_NET_DRIVER}=="vif", RUN+="/usr/lib/qubes/setup-ip"
, but it doesn't seem to do anything (I've tried to remove it, restart udev and start a VMs, but the connection works fine).
I really need some help, maybe from the developers, to understand better these mechanisms, to be able to implement a solution as more general as possible, that users can adopt without efforts.
David Hobach
On 10/28/2017 06:36 PM, nicholas roveda wrote:
> Up to now, I've thought of 2 possible solutions:
> I really need some help, maybe from the developers, to understand better these mechanisms, to be able to implement a solution as more general as possible, that users can adopt without efforts.
upstream services to downstream VMs.
The mechanisms are somewhat different in 3.2 and 4.0.
Your thoughts appear to be overly complicated. I e.g. didn't understand
the dom0 part:
dom0 doesn't store any DNS settings. Each VM forwards packets to the
next upstream VM and the DNS decision is made at your netvm resolv.conf.
Dynamic firewalls (iptables in 3.2, netfilter or so in 4.0) managed by
the dom0 qvm-firewall in your proxsy & firewall VMs ensure only allowed
traffic is going through. DNS uses dnat rules. Dom0 uses your firewall
VM to pull updates.
So you can just configure your DNS cache server in your net VM or if you
want to use a proxy VM you'll have to write a custom service that does
the iptables changes desired by the Qubes firewall (the default ones
will not work as they are designed to forward packets further upstream).
The latter should only be done by really experienced users.
nicholas roveda
> dom0 doesn't store any DNS settings. Each VM forwards packets to the
> next upstream VM and the DNS decision is made at your netvm resolv.conf.
> Dynamic firewalls (iptables in 3.2, netfilter or so in 4.0) managed by
> the dom0 qvm-firewall in your proxsy & firewall VMs ensure only allowed
> traffic is going through. DNS uses dnat rules. Dom0 uses your firewall
> VM to pull updates.
I'm aware of the Qubes networking mechanisms.
What I'm trying to achieve is the following:
1) New fields in 'Qubes Global Settings' menu, under 'System defaults' (Dom0) that let the user choose the default primary and secondary DNS (now '10.139.1.1' and '10.139.1.2').
How this will be implemented?
Dom0 will inform the first NetVM backends ('sys-firewall', 'sys-net' or a VpnVM) of the change though a rpc service and this will trigger `qubes-dnat-to-ns` which will procede to update the (NAT) iptables.
-> All the DNS requests destinated to '10.139.1.1' and '10.139.1.2' DNAT to the new DNS backend specified by the user <-
This method doesn't require any changes to the Templates or the AppVMs, but only to install a rpc service and a custom `qubes-dnat-to-ns` script in the VMs that provide network (ProxyVMs and NetVMs).
2) New fields in 'VM Settings' menu (Dom0) that let the user choose the primary and secondary DNS for each VM, if the machine has network.
How this will be implemented?
Dom0 will inform the NetVM backend to which the VM is attached and the (NAT) iptables will be modified to DNAT all the DNS requests from a the specific VM vif interface (-i vifX.0) to the new addresses provided.
Again, no changes are required to the Templates or the AppVMs, but only to the VMs that provide network (ProxyVMs and NetVMs).
*
ISSUES:
1) I solved the first issue and I discovered how to retrieve vif names from xen database,
2) I still don't know how to make all VMs that provide network launch a script when a new vif interface came up.
As I said before, I've found the this file '/etc/udev/rules.d/99-qubes-network.rules' that I believed was the solution, but it seem not to work at all,
3) My code still rely on files, because the Qubes API and Qubesadmin API has confused me a lot.
I need to add:
- the attributes 'default_primary_dns' and 'default_secondary_dns' to 'qubesadmin.app.QubesBase' and
- the properties 'primary_dns' and 'secondary_dns' to the base Class that represent a VM object.
I don't know if I have to modify 'Qubes' or 'Qubesadmin' and if this is enough for the `qubesd` calls or if it's not possible to add new properties at all.
I really need help from the developers, expecially for the point n.3.
Thanks!