sys-firewall based on debian-10-minimal not recognized

[Sven Semmler]

Hi,

I created a sys-firewall based on debian-10-minimal:

* qvm-clone debian-10-minimal deb-10-sys-firewall
* qvm-create --template deb-10-sys-firewall --label blue dvm-sys-firewall
* qvm-prefs dvm-sys-firewall template_for_dispvms True
* qvm-create --class DispVM --template dvm-sys-firewall --lable blue sys-firewall
* qvm-prefs sys-firewall provides_network True
* qvm-prefs sys-firewall netvm sys-net

Then in deb-10-sys-firewall (template):

* sudo apt-get install qubes-core-agent-networking qubes-core-agent-dom0-updates
* attempting to install iproute tells me that this package no longer exists and I shall try iproute2
* iproute2 does exist and was already installed

Then in dvm-sys-firewall (template for disposable):

* added "iptables -I FORWARD 2 -s 10.137.0.21 -d 10.137.0.25 -j ACCEPT" to /rw/config/qubes-firewall-user-script

Then shut everything down and started sys-firewall.

Result:

* network connectivity working
* the above mentioned iptables rule is working (.21 can connect to .25)
* qubes-qube-manager gives me this error when I try to edit the firewall rules of any qube connected to sys-firewall: "Networking qube does not support 'qubes-firewall' - firewall restrictions will not be applied."
* however it does not give me this error when I try to edit other qubes connected to sys-whonix

Any ideas?

/Sven

--
public key: https://www.svensemmler.org/0x8F541FB6.asc
fingerprint: D7CA F2DB 658D 89BC 08D6 A7AA DA6E 167B 8F54 1FB6

[Sven Semmler]

On Tue, Feb 04, 2020 at 09:59:57AM -0600, Sven Semmler wrote:
> As far as I can tell the firewall rules are enacted / the firewall does work. So it's "just" Qubes Manager thinking it's not. How does it check?

$ qvm-service sys-firewall
qubes-firewall on
clocksync on
qubes-updates-proxy on

$ qvm-features sys-firewall
service.qubes-firewall 1
check-updates
service.clocksync 1
service.qubes-updates-proxy 1
appmenus-dispvm 1

[awokd]

Sven Semmler:

Maybe it doesn't like the "disposable" part? Try it with a regular AppVM
based on that same minimal template.


--
- don't top post
Mailing list etiquette:
- trim quoted reply to only relevant portions
- when possible, copy and paste text instead of screenshots

[Sven Semmler]

On Sun, Feb 09, 2020 at 01:46:12PM +0000, 'awokd' via qubes-users wrote:
> Maybe it doesn't like the "disposable" part? Try it with a regular AppVM
> based on that same minimal template.

I did: makes no difference. Also I've been running debian-based and
fedora-based disposable sys-firewall for years. It has most likely to do
with the -minimal template and me being oblivious to what the Qubes
Manger looks for.

After verifying that the firewall does indeed work and does what it's
supposed to do I see this as a minor annoyance. I saw some discussion on
github about how the Qubes Manager recognizes firewall functionality but
all those things I already checked - I think (see previous postings).

Could you point out to me please how I would verify that the qubes-firewall service is up and running?

[awokd]

Sven Semmler:

> On Sun, Feb 09, 2020 at 01:46:12PM +0000, 'awokd' via qubes-users wrote:
>> Maybe it doesn't like the "disposable" part? Try it with a regular AppVM
>> based on that same minimal template.
>
> I did: makes no difference. Also I've been running debian-based and
> fedora-based disposable sys-firewall for years. It has most likely to do
> with the -minimal template and me being oblivious to what the Qubes
> Manger looks for.

I had double-checked my debian-10-minimal firewall template against the
packages you installed and they matched, so I'm trying to think what
else is different. Maybe because you manually edited the rules? I've
seen that confuse the GUI before.

> After verifying that the firewall does indeed work and does what it's
> supposed to do I see this as a minor annoyance. I saw some discussion on
> github about how the Qubes Manager recognizes firewall functionality but
> all those things I already checked - I think (see previous postings).
>
> Could you point out to me please how I would verify that the qubes-firewall service is up and running?

Systemctl in sys-firewall tells me qubes-firewall.service is "loaded
active running", if that helps.

[Sven Semmler]

On Mon, Feb 10, 2020 at 04:55:40PM +0000, 'awokd' via qubes-users wrote:
> I had double-checked my debian-10-minimal firewall template against the
> packages you installed and they matched, so I'm trying to think what
> else is different. Maybe because you manually edited the rules? I've
> seen that confuse the GUI before.

I decided to redo everything from scratch as a sanity check:

dom0: sudo qubes-dom0-update qubes-template-debian-10-minimal
dom0: qvm-clone debian-10-minimal tpl-deb-10-min
dom0: sudo dnf erase qubes-template-debian-10-minimal
dom0: qvm-run -u root tpl-deb-10-min xterm
tpl-deb-10-min: apt-get update
tpl-deb-10-min: apt-get install qubes-core-agent-passwordless-root
qubes-core-agent-networking qubes-core-agent-dom0-updates
tpl-deb-10-min: apt-get dist-upgrade
tpl-deb-10-min: apt-get autoremove
tpl-deb-10-min: apt-get autoclean
tpl-deb-10-min: sudo poweroff
dom0: qvm-create --template tpl-deb-10-min --label blue sys-firewall2
dom0: qvm-prefs sys-firewall2 provides_network true
dom0: qvm-prefs sys-firewall2 netvm sys-net
dom0: qvm-create --template tpl-deb-10-min --label red app-test
dom0: qvm-prefs app-test netvm sys-firewall2
dom0: qvm-start app-test
dom0: qubes-qube-manager

When checking the Firewall tab of app-test the Qube Manager does not
complain. ... so I must have done something wrong the first time around.

I'll clean this up. Thank you for your help!

[54th Parallel]

On Tuesday, 11 February 2020 at 00:56:07 UTC+8 awokd wrote:

Systemctl in sys-firewall tells me qubes-firewall.service is "loaded
active running", if that helps.

--
- don't top post
Mailing list etiquette:
- trim quoted reply to only relevant portions
- when possible, copy and paste text instead of screenshots

I'm having the same issue with disposable firewalls built on debian-10-minimal, with the minimum amount of packages, on brand-spanking-new installations (plural) being unreliable firewalls. They sometimes function but not all the time--and this is what's scary, because there's no way of knowing without manually checking all the time. The warning prompts when editing firewall rules aren't useful indicators since they always appear regardless of whether filtering is happening.

I ran systemctl in both and found that qubes-firewall.service is not running in either, despite having manually activated them. I'm not a technical person, but this seems like a pretty critical issue to me (unreliable firewall with no indicator)--a warning about using minimal debian as templates for firewalls should be put up somewhere highly visible.

This unreliability has been bugging me for a while and I've been testing and testing (to the best of my abilities) before realizing that this is almost certainly not a user issue, so Sven, the OP,  probably either ran into the issue again, didn't know about his deactivated firewalls, or didn't report the issue. 

[54th Parallel]

On Friday, 21 August 2020 at 16:58:48 UTC+8 54th Parallel wrote:

I'm having the same issue with disposable firewalls built on debian-10-minimal, with the minimum amount of packages, on brand-spanking-new installations (plural) being unreliable firewalls. They sometimes function but not all the time--and this is what's scary, because there's no way of knowing without manually checking all the time. The warning prompts when editing firewall rules aren't useful indicators since they always appear regardless of whether filtering is happening.

I ran systemctl in both and found that qubes-firewall.service is not running in either, despite having manually activated them. I'm not a technical person, but this seems like a pretty critical issue to me (unreliable firewall with no indicator)--a warning about using minimal debian as templates for firewalls should be put up somewhere highly visible.

This unreliability has been bugging me for a while and I've been testing and testing (to the best of my abilities) before realizing that this is almost certainly not a user issue, so Sven, the OP,  probably either ran into the issue again, didn't know about his deactivated firewalls, or didn't report the issue. 

After some more probing around, I think I've found the issue, and that what I wrote earlier contains inaccuracies. The unreliable firewall might not be a debian-10-minimal issue, though the warning prompt that appears whenever editing firewall rules in a connected VM is. 

My setup has two firewalls--one behind sys-net and another behind a VPN VM. Though the two firewalls are clones of one another, the sys-net firewall works (responds to rules set in appVMs) and the proxyVM firewall doesn't. This is what caused me to think that deb-10-min firewalls in general are unreliable--some things are connected to the net-firewall (sometimes) and most are connected to the VPN-firewall. This makes it look like the firewalls work sometimes. 

I have two laptops running Qubes with the same setup. Of the four firewalls, all with qubes-firewall explicitly enabled, only one actually has the qubes-firewall.service show up after typing 'systemctl | grep firewall'. Each of these firewalls were created in fresh but updated installations of 4.0.3 with the absolute minimum amount of packages (qubes-core-agent-passwordless-root (so I can configure sudo prompt), qubes-core-agent-networking, apparmor*) and the typical settings, along with qubes-vm-hardening (vm-boot-protect enabled).

Any insight into this would be greatly appreciated since this is a massive headache for me.

[Frank]

On 22. Aug 2020, at 11:01, 54th Parallel wrote:



On Friday, 21 August 2020 at 16:58:48 UTC+8 54th Parallel wrote:

I'm having the same issue with disposable firewalls built on debian-10-minimal, with the minimum amount of packages, on brand-spanking-new installations (plural) being unreliable firewalls. They sometimes function but not all the time--and this is what's scary, because there's no way of knowing without manually checking all the time. The warning prompts when editing firewall rules aren't useful indicators since they always appear regardless of whether filtering is happening.

I ran systemctl in both and found that qubes-firewall.service is not running in either, despite having manually activated them. I'm not a technical person, but this seems like a pretty critical issue to me (unreliable firewall with no indicator)--a warning about using minimal debian as templates for firewalls should be put up somewhere highly visible.

This unreliability has been bugging me for a while and I've been testing and testing (to the best of my abilities) before realizing that this is almost certainly not a user issue, so Sven, the OP,  probably either ran into the issue again, didn't know about his deactivated firewalls, or didn't report the issue. 

After some more probing around, I think I've found the issue, and that what I wrote earlier contains inaccuracies. The unreliable firewall might not be a debian-10-minimal issue, though the warning prompt that appears whenever editing firewall rules in a connected VM is. 

My setup has two firewalls--one behind sys-net and another behind a VPN VM. Though the two firewalls are clones of one another, the sys-net firewall works (responds to rules set in appVMs) and the proxyVM firewall doesn't. This is what caused me to think that deb-10-min firewalls in general are unreliable--some things are connected to the net-firewall (sometimes) and most are connected to the VPN-firewall. This makes it look like the firewalls work sometimes.

I am not sure what you mean with „behind a vpn vm“.

My setup is such that I have the sys-net VM which is used as network vm in sys-firewall and a few sys-vpn-xxx. sys-firewall in turn is used as network vm for all app VMs that connect directly to the internet and the sys-vpn-xxx VMs are used by various VMs that connect through the various VPNs. There is no need for extra proxy or firewall VMs, since the sys-vpn-xxx can themselves double as firewall VMs, since they are proxy VMs already. Therefore any rules specified in any of the app VMs (no matter if directly connected through sys-firewall or indirectly through a vpn) are taken care of by the network vm they are connected to. And I have never had a problem with this setup. At least none, that I am aware of.  ;-)

Also don’t forget, that a proxy vm set as network vm for your sys-vpn, will never see anything other than the sys-vpn connecting to your vpn provider’s server. Therefore any rules specified in your sys-vpn are pretty much useless.

Regards, Frank

I have two laptops running Qubes with the same setup. Of the four firewalls, all with qubes-firewall explicitly enabled, only one actually has the qubes-firewall.service show up after typing 'systemctl | grep firewall'. Each of these firewalls were created in fresh but updated installations of 4.0.3 with the absolute minimum amount of packages (qubes-core-agent-passwordless-root (so I can configure sudo prompt), qubes-core-agent-networking, apparmor*) and the typical settings, along with qubes-vm-hardening (vm-boot-protect enabled).

Any insight into this would be greatly appreciated since this is a massive headache for me.

--
You received this message because you are subscribed to the Google Groups "qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/4ce48a47-2bec-4d61-a855-5ef83b61d1f9n%40googlegroups.com.

[54th Parallel]

On Saturday, 22 August 2020 at 18:57:22 UTC+8 Frank Schäckermann wrote:

I am not sure what you mean with „behind a vpn vm“.

My setup is such that I have the sys-net VM which is used as network vm in sys-firewall and a few sys-vpn-xxx. sys-firewall in turn is used as network vm for all app VMs that connect directly to the internet and the sys-vpn-xxx VMs are used by various VMs that connect through the various VPNs. There is no need for extra proxy or firewall VMs, since the sys-vpn-xxx can themselves double as firewall VMs, since they are proxy VMs already. Therefore any rules specified in any of the app VMs (no matter if directly connected through sys-firewall or indirectly through a vpn) are taken care of by the network vm they are connected to. And I have never had a problem with this setup. At least none, that I am aware of.  ;-)

Thank you for your time, Frank.

Right now my setup is as follows:

Internet -- sys-net (disp) -- sys-firewall-1 (disp) -- sys-vpn (disp) -- sys-firewall-2 (disp) -- appVMs

I chose to separate sys-vpn and sys-firewall-2 despite knowing that sys-vpn can also act as a firewall. This is because I want to have another layer of isolation, so that a compromised sys-vpn wouldn't bring down the firewall that filters the contents of the VPN connection. Is this setup causing issues for the upstream firewalls? If so, why would it be, since the number of downstream firewalls are the same in both our setups.

Also don’t forget, that a proxy vm set as network vm for your sys-vpn, will never see anything other than the sys-vpn connecting to your vpn provider’s server. Therefore any rules specified in your sys-vpn are pretty much useless.

The way I understand how firewalls works in Qubes is that the firewall executes rules for upstream VMs, so using my setup as an example, firewall-1 takes orders from sys-vpn, while firewall-2 takes orders from the appVMs. Firewall-1 is therefore important for stopping unwanted inbound connections from reaching sys-vpn and also acts as a final gatekeeper against unwanted outbound leaks. Having only one whitelist rule for the VPN set in sys-vpn, which itself doesn't have qubes-firewall.service started by default (I checked) makes it so that only the VPN connection can go past firewall-1 (assuming the firewall is functioning), so it's not useless.

Tl;dr FW-1 restricts connection to VPN only, FW-2 filters connections from the VPN, sys-VPN doesn't start qubes-firewall.service (at least with debian-10-minimal), even though it technically should. 

 

[unman]

To deal with the question here, I use debian minimal templates
extensively, (NOT with qubes-vm-hardening), and have never seen this
issue - neither the unreliable firewall nor the warning prompts issue.
I've asked fellow users who also use debian-minimal and they do no not
recognise the issue.
So either it's qubes-vm-hardening, or your proxyVM setup.
Insight will follow (perhaps) if you give more information. How have you
set up the proxyVM? Since that does not work n either machine we should
be able to dig into that problem relatively easily.
What is different about the setup and laptop where the "sys-net
firewall" (please explain what this means) does NOT work compared to the
one where it DOES work?

For the sake of sanity I suggest you answer in ONE place and then
cross-post the solution, rather than pursue the issue in both the
forum and the mailing list. And please don't post it in reddit too.

unman

[Sven Semmler]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 8/22/20 10:10 AM, unman wrote:
> To deal with the question here, I use debian minimal templates
> extensively, (NOT with qubes-vm-hardening), and have never seen
> this issue - neither the unreliable firewall nor the warning
> prompts issue. I've asked fellow users who also use debian-minimal
> and they do no not recognise the issue.

As mentioned earlier ... I have seen it. Both with debian-10-minimal
and debian-10-minimal + kicksecure. However, just like before... when
I retraced my steps the problem went away.

I suspect that maybe one of the packages that need to be installed
runs a post-install script that sometimes fails (I have no proof).

Another thing that caused many issues for me (until I recognized and
fixed it) was that the time in my sys-net was wrong.

Sorry, got no answers... just observations.

/Sven

- --

public key: https://www.svensemmler.org/0x8F541FB6.asc
fingerprint: D7CA F2DB 658D 89BC 08D6 A7AA DA6E 167B 8F54 1FB6

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEE18ry22WNibwI1qeq2m4We49UH7YFAl9F0CQACgkQ2m4We49U
H7YfYBAAjlyS/ehbhbztuYNMMqU7HLoUk3LGCwJpBoe67Nz5+DB1H3gbM7l9T1xc
7opa7aRbOUa1aI88sZUuOgFhvQHbokho9X9ebbADjjTYQHhiiBXqO3pULgh+P8+g
jHCObPgID/OX7ceiS7hp7Cy+SiDm1ZrUQIdSFlCduZJYC44a6BauSaF5P9o+fpnq
FlFSbLGq1XyLQcKtr6Gl5rgYVa52UfTWzKt9+TGzIyAV/TRbLisInvI9RhkmVBvj
yMJYbgYsjyjKLqFMvLzeMeeJoriHzNzWz6Zqwbmw6NXkGFAqmqCSfYLyLprgnPyf
g8DGkGhRmp9awK77C8HfruqyfADPsdXYrHdVxBJbiSOB7CKbst3ZIBCVvVfwD0oY
8DGNXAfP5hN7wWuRas6IshRArgk0ujuU0uR8EQpOMgI/qutZYY1LiA8BQ0o7RhUf
o3Rc4JM4lU10EQPafZOeWDQueD4v837dd5IkCuNKeidRmCvFzIHu7/SJFlOvnbWB
PNi23k3Hxx8x5hHDUDK72tXcEWawdiUJ6NMfmQG31N/zptBu8FGBFROn8Ww+XJRb
ePIfHmcsdNJFekWzDe4Fa8B2EpioL4QJ8Gcqy3rFmn7ZQ870prAOpT8WbNYk49BW
fgd/H8PBXB09aa0oHYc2KBGphrbXToIDlBLULCe5HeCKxtp/tj0=
=hvfa
-----END PGP SIGNATURE-----

[54th Parallel]

Hi Sven,

Thanks for returning to this thread. As per unman's request, I have continued the discussion at the discourse.groups forum (https://qubes-os.discourse.group/t/debian-10-minimal-firewall-issues/146/7). My sys-net time has also slipped, like yours, on occasion. Sometimes it's 12+ minutes behind. This is apparently a common issue that's due to be fixed  (https://github.com/QubesOS/qubes-issues/issues/4939), so if this was a cause of firewall issues, more people should have noticed it. 

Also, unman correctly pointed out that I misspoke when I said that my issues are the same as yours, as you seem to focus on the warning prompt that shows when attempting to edit firewall rules, while I have that problem plus qubes-firewall.service being unreliable. Anyways, as a non-technical user, I'm in the same boat as you--no answers, just observations. I managed to get Mirage firewall running and I highly recommend it to you in particular since it takes a lot of uncertainty out of the issue while being easy to use.