Qubes Routing and NAT
Epinsion Polickye
I've been following this guide to set up routing and NAT to an internal machine: https://www.qubes-os.org/doc/firewall/
sys-net and my machines are currently working on the internal network, and the internet perfectly outbound, but I'm having troubles exposing services from sys-net (which I only intend to do for testing with this VM), or internal VMs (what I actually want to do).
My machine is connecting directly to sys-net (no sys-firewall middle man).
My first step is to simply run "nc -nlv 444" on sys-net to telnet into it from and internal device, and be able to ping the sys-net machine from an internal device. I'm having issues just at this step however, even if I've disabled nftables and iptables. qvm-ls -n lists a different sys-net ip than what's set up as the internal private address on the network.
I can ping from sys-net to a particular machine, but not back to the machine.
I suspect I'm misunderstanding networking and firewalls in Qubes. I expect sys-net to function like a router (along with any other intermediary VMs for firewalls, net/vpns and the like), and for the firewalls to be handled by nftables and iptables, which shouldn't function when the services are disabled on systems.
Thanks.
Epinsion Polickye
You may be able to help me troubleshoot the below. I'm yet to learn iptables, nftables, and qubes-firewall in depth. The below seems to make sense to me - accept traffic on 443 to 10.1.113.0 and forward to 443 on 10.138.22.124 and ensure the machine accepts the traffic on its firewall.
Interface ens5 is 10.1.113.0 on sys-net
Interface eth0 is 10.138.22.124 on a downstream vm.
I'm wanting to forward 443 from sys-net to the vm.
The PREROUTING rule is numbered 3
On Sys-Net
----------
sudo iptables -t nat -A PREROUTING -i ens5 -p tcp --dport 443 -d 10.1.113.0 -j DNAT --to-destination 10.138.22.124
sudo iptables -I FORWARD 3 -i ens5 -d 10.138.22.124 -p tcp --dport 443 -m conntrack --ctstate NEW -j ACCEPT
On Downstream VM
----------------
sudo iptables -N MY-HTTPS
sudo iptables -I INPUT 5 -d 10.1.113.0 -p tcp --dport 443 -m conntrack --ctstate NEW -j MY-HTTPS
Epinsion Polickye
Current workaround is an ssh tunnel. Example:
On sys-net and VM: sudo iptables -I INPUT -p tcp --dport 444 -j ACCEPT
On VM: sudo iptables -I INPUT -s 10.1.113.0 -j ACCEPT
And finally on sys-net: sudo ssh -L 0.0.0.0:444:10.138.30.53:444 us...@10.138.30.53
Then I can connect to tcp 444 on 10.1.113.0 (sys-net) which will tunnel it through to the local vm.
Epinsion Polickye
And also on VM: sudo iptables -I INPUT -s 10.137.0.5 -j ACCEPT
Unman
You haven't said what Qubes version you have or what template you are
using for sys-newt and sys-firewall. Can you add that?
Epinsion Polickye
I'm using qubes-4. sys-net and sys-firewall are fedora-26.
Unman
If you switched the sys-net and sys-firewall to Debian, I suspect that
your existing rules might work. Otherwise recast them in nftables.
qvm-ls -n shows the IP address of sys-net within Qubes - not the IP
address of the external interface on the network, as you have already
noticed.
So you will need to open the firewall to allow traffic to tcp 444 on
ens5.
You can run a sniffer like tcpdump on the external interface of sys-net
to make sure that traffic from the local networking is actually
arriving.
You might find this and the linked projects interesting:
https://github.com/QubesOS/qubes-issues/issues/3556
unman
Epinsion Polickye
Thanks for your help and for sharing the link. I'm going to set aside some time to read the Qubes doco on networking, learn nftables and iptables, and read the link.
Regardless I've found ssh tunneling from sys-net to be very handy, and could be very useful with ssh jumps if there are intermediary firewalls and network services.