Firewall antispoofing in ingress hook

[qubist]

Hi,

With the current firewall, a spoofed packet can travel all the way to
the ip/ip6 hook, where it is dropped.

Considering the general security principle to stop attacks as far as
possible, why is it not dropped early in the ingress hook instead, thus
also saving additional CPU cycles?

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

The "antispoof" chain is hooked via the "raw" priority, which happens
before all kind of connection tracking, routing etc, basically as early
as firewall can see whole IP packets (not for example only their
fragments). Theoretically it might be moved a bit earlier, but I don't
think it saves much processing, but on the other hand you may run into
some issues since not all packet fields are available at this stage yet.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmYmy4QACgkQ24/THMrX
1yx2KQf/coLIf6whMj5JfoCF4X3E+x0liALkjPNNC7GART8HkZzCA+9w+We6bpg6
09VOWEEmMPa7PNjEFKg+frMAWT2gCFye0civCucSvmVtSmzRpZL3eWPmWZPlj8pt
ITPE79fyVXBEIsvuKQaZloGFeyINQQSW64Szf+k+sWy+uTrcyV6Kx0tWpDi5KTVK
TSc28ujclSWRcDCCo836lMLmd4NgBZ63C0MAdLQfzP97wwzqbFrj7FBT+r41B/ES
QZE0W2zanxmA/DDoP7NOZ4BjHFSk62ueQ8xgeAyQRdNFFDwVNv2crRtjlmgolZ5z
9RgNwR9cD4/EDxhM7aP0D+fkurnaUg==
=98ms
-----END PGP SIGNATURE-----

[qubist]

On Mon, 22 Apr 2024 22:41:40 +0200 Marek Marczykowski-Górecki wrote:

> The "antispoof" chain is hooked via the "raw" priority, which happens
> before all kind of connection tracking, routing etc, basically as
> early as firewall can see whole IP packets (not for example only their
> fragments).

I can see how it is done but why would one need to (potentially)
defragment the spoofed packets first just to drop them later? How does
that match the mentioned principle?

> Theoretically it might be moved a bit earlier,

What do you mean theoretically? It can be done practically. I have done
it and it works. What I am lacking is the mechanism of dynamically
adding interface . address paired elements to the set, which comes with
Qubes, so my approach is not as flexible as the original, e.g. I can
drop hosts "pretending to be me" and bogons but that's pretty much all.

> but I don't think it saves much processing,

Have you made any actual tests/comparisons? In a high speed attack
(many spoofed packets) it may turn out significant.

> but on the other hand you may run into some issues since not all
> packet fields are available at this stage yet.

What issues? Which fields are necessary to drop spoofed packets?

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Tue, Apr 23, 2024 at 08:40:04AM -0000, qubist wrote:
> On Mon, 22 Apr 2024 22:41:40 +0200 Marek Marczykowski-Górecki wrote:
>
> > The "antispoof" chain is hooked via the "raw" priority, which happens
> > before all kind of connection tracking, routing etc, basically as
> > early as firewall can see whole IP packets (not for example only their
> > fragments).
>
> I can see how it is done but why would one need to (potentially)
> defragment the spoofed packets first just to drop them later? How does
> that match the mentioned principle?
>
> > Theoretically it might be moved a bit earlier,
>
> What do you mean theoretically? It can be done practically. I have done
> it and it works.

Care to open a pull request then?

> What I am lacking is the mechanism of dynamically
> adding interface . address paired elements to the set, which comes with
> Qubes

This is done in the vif hotplug script:
https://github.com/QubesOS/qubes-core-agent-linux/blob/main/network/vif-route-qubes#L222-L223

> , so my approach is not as flexible as the original, e.g. I can
> drop hosts "pretending to be me" and bogons but that's pretty much all.
>
> > but I don't think it saves much processing,
>
> Have you made any actual tests/comparisons? In a high speed attack
> (many spoofed packets) it may turn out significant.

Have you measured it? I'd say it's up to ones who propose a change to
justify it. (but, if the change has no downsides, I'm okay with
accepting it without detailed benchmark too)

> > but on the other hand you may run into some issues since not all
> > packet fields are available at this stage yet.
>
> What issues? Which fields are necessary to drop spoofed packets?

For antispoofing probably none. But other things can be added to the
"prerouting" chain. Or do you mean moving just "antispoof" chain out of
"prerouting" and have it hooked directly?

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmYnh6YACgkQ24/THMrX
1yyVEwf9HaVqh1fIgTeuYoTE6WK76O8CKSJaXIkm13b8br22OnvwhdzMlD74b/Y6
0nJloGncWtzn2mkGQsSH1azQTa8aroZqzqCa5GnnfiAPyXvDl4q/nM3bzw+V+SRE
wISQolnH/0nPhsvUENGj2nJGVG9fR6izFxIfj/gO+6dro1GRANkHb2m2F/GIsRKu
ORw5MIcprWb7t1aCrxPkCDArFt3NAsg53uxTZLsO1Tr/nH6T5GxRiPsE8U1T7pcj
HWctIVJXh3JYUi4mrVVjIz9GhLSFI8Q9LykBJ+EG7vcabn5D+GN8/n5ycC1Cokx2
eJLrnfZHGQzgZ4BuN26tjQ9U0I7UDA==
=2DVw
-----END PGP SIGNATURE-----

[qubist]

On Tue, 23 Apr 2024 12:04:22 +0200 Marek Marczykowski-Górecki wrote:

> Care to open a pull request then?

A few things:

1. The customizations I am working on are atomic nftables tables,
chains and rules. So far, they work fine independently and along with
the current default firewall.

2. I still don't know how to work with git efficiently, so this "pull
request" is something I need to learn about first. All I know so far is
how to create a project and commit changes.

3. My firewall work (which involves other improvements, I hope) is
still not complete. As soon as it is, I would gladly share it.

> This is done in the vif hotplug script:
> https://github.com/QubesOS/qubes-core-agent-linux/blob/main/network/vif-route-qubes#L222-L223

Thanks! Bash is fine for me, so I will see what modification works
best. What calls that script BTW?

> Have you measured it?

Not yet. At least not in a way deserving to be called measurement. All
I have done is to trace the result and see that ingress works with less
steps. If you know a good way to measure it correctly, please let me
know.

> I'd say it's up to ones who propose a change to justify it.

Of course.

> Or do you mean moving just "antispoof" chain out of "prerouting" and
> have it hooked directly?

I am testing my stuff in the netdev with priority -500. However,
ingress is available in prerouting too, so that move might work as well.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Tue, Apr 23, 2024 at 11:16:56AM -0000, qubist wrote:
> On Tue, 23 Apr 2024 12:04:22 +0200 Marek Marczykowski-Górecki wrote:
>
> > Care to open a pull request then?
>
> A few things:
>
> 1. The customizations I am working on are atomic nftables tables,
> chains and rules. So far, they work fine independently and along with
> the current default firewall.
>
> 2. I still don't know how to work with git efficiently, so this "pull
> request" is something I need to learn about first. All I know so far is
> how to create a project and commit changes.
>
> 3. My firewall work (which involves other improvements, I hope) is
> still not complete. As soon as it is, I would gladly share it.
>
> > This is done in the vif hotplug script:
> > https://github.com/QubesOS/qubes-core-agent-linux/blob/main/network/vif-route-qubes#L222-L223
>
> Thanks! Bash is fine for me, so I will see what modification works
> best. What calls that script BTW?

xendriverdomain daemon (xl devd), when the vif interface is
created/removed.

> > Have you measured it?
>
> Not yet. At least not in a way deserving to be called measurement. All
> I have done is to trace the result and see that ingress works with less
> steps. If you know a good way to measure it correctly, please let me
> know.

I'd use some packet generator, maybe even ping (or nping) will be
enough. You can easily try spoofing by changing IP inside some test VM.

> > I'd say it's up to ones who propose a change to justify it.
>
> Of course.
>
> > Or do you mean moving just "antispoof" chain out of "prerouting" and
> > have it hooked directly?
>
> I am testing my stuff in the netdev with priority -500. However,
> ingress is available in prerouting too, so that move might work as well.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmYnnqEACgkQ24/THMrX
1ywroAf/U0Iq2WsIFs1HPv/PGobjCdEz5cSFsL+4fWACYydi+Cqtj8EOQmc1cRh/
ei9xYpGcQYFD69/suZZuostkb87LIuwNsA/jJ2NDQjnCdM1G/i0Z/lUoPAuNNb07
h3csaDGQPWtKuLAM30ADsVSa+dneAyu9tkK+xAOI+Kq8R+UjkVPr1Dxdnf5RXHSL
niFIicJB6U9+cVCO8jr/3O4DgYp8xf1GsL6uCXx9bPgO25R57Y+Mcga5Jam1SeG7
P5f3REk91kiZ2BZG5kmlfg9ipAE/Fyx9y4QLcz9fcafzwu17NFZlP+jRDShqLKAx
ijh44R63lS1nT5TiPJFYLbsrMmtQEg==
=xJVU
-----END PGP SIGNATURE-----

[qubist]

On Tue, 23 Apr 2024 13:42:25 +0200 Marek Marczykowski-Górecki wrote:

> xendriverdomain daemon (xl devd), when the vif interface is
> created/removed.

Thanks.

> I'd use some packet generator, maybe even ping (or nping) will be
> enough. You can easily try spoofing by changing IP inside some test
> VM.

Well, I have been using hping3 for spoofing but it doesn't provide a
measure. I thought that by measure you mean something along the lines
of network throughput during a heavy spoofing load.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I'd simply generate some traffic and observe CPU load in the firewall
vm.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmYnpzAACgkQ24/THMrX
1yz1BAf/TUImu5wA+HKsonG0hRnTZCxBQ57/oZn2H/8fX8k6B84a9USBaZghygvD
G41usqvZM/9mBUCMNk1o0ayCIseYOwsvp2bPOLX9WmSNMamL+rPCKcFTTQ6Jmwnv
XPGV//UQ9EwwWYuSf4ApeTO3NgeNI/NThhLyq99WUxz+CkNqBuPI4TZ9VtEmhAZy
KkVNT/PDCIpv09l8gjm8NIwkSAzsbT1wZTYZgd2ygIwZdz+ClsqPwlpZZ8R41YGy
YYiqgSxXAHE89foMzAYvq4UmaOrCReJBChJpwyPwxVLeBBqqyYD0pMe14Gmw7LJA
DE4VhNZA9sBmwHPvD9uvAahyqodWXA==
=9g30
-----END PGP SIGNATURE-----

[qubist]

I will see what I can do.

[qubist]

Marek,

I think I have made the necessary changes and I can see vif-route-qubes
works the new way. Before sharing the result, I need to have something
clarified, please.

Currently, 'ip a' shows that all interfaces have both an IPv4 and an
IPv6 address. The current ruleset, through chain 'antispoof' in table
'ip6 qubes' drops all IPv6 traffic for the particular interface.
Example packet:

IN=vif32.0 OUT= MAC=33:33:00:00:00:16:00:16:3e:5e:6c:00:86:dd SRC=0000:0000:0000:0000:0000:0000:0000:0000 DST=ff02:0000:0000:0000:0000:0000:0000:0016 LEN=76 TC=0 HOPLIMIT=1 FLOWLBL=0 PROTO=ICMPv6 TYPE=143 CODE=0

According to RFC 4890, type 143 is "traffic that must not be dropped",
but it is dropped. There are other such ICMPv6 packets with different
codes - also types which must not dropped but do get dropped. It is not
clear how these will be handled in case one enables IPv6. I also can't
figure where the DST address in the above packet comes from, as neither
of the 2 qubes, between which I am testing, has that.

While I am aware of https://www.qubes-os.org/doc/networking/#ipv6, it
still not clear:

- If IPv6 is disabled by default, why in-qube interfaces have IPv6
addresses too?

- Looking at the check on line 208: does it mean that if one enables
IPv6, the IPv4 antispoof chain will start dropping instead, i.e. that
the IPv6 antispoof and IPv4 antispoof will "switch roles"? More
specifically: is this expected and intentional? I am asking because
the ingress dropping, which I implemented, is based on the same check
and the result is the same (just dropped earlier, as discussed), but
the concern regarding RFC 4890 remains.

- Actually, I am also working separately on ICMPv6 as per RFC 4890
(which is currently not the case). Would you be interested to have a
look at it too when I have it ready?

I hope you can shed some light, so that I know how to proceed.

[qubist]

On Tue, 23 Apr 2024 12:04:22 +0200 Marek Marczykowski-Górecki wrote:

> Have you measured it? I'd say it's up to ones who propose a change to
> justify it.

Now, I have.

Setup:

2 VMs: firewall and client (vif interface). TCP and UDP port 1000 is
explicitly open on firewall's input for the purpose of testing.

Testing procedure:

- run a test for at least 1 minute
- note load average in firewall VM
- wait 1 minute before next test

======================================================================

Original Qubes firewall
-----------------------

### Spoof

# time (timeout 65s hping3 10.137.0.88 --flood --spoof 10.137.0.88)
HPING 10.137.0.88 (eth0 10.137.0.88): NO FLAGS are set, 40 headers + 0 data bytes
hping in flood mode, no replies will be shown

--- 10.137.0.88 hping statistic ---
14397433 packets transmitted, 0 packets received, 100% packet loss
round-trip min/avg/max = 0.0/0.0/0.0 ms

real 1m5.015s
user 0m6.268s
sys 0m47.634s

load average: 0.21, 0.07, 0.02

----------------------------------------------------------------------

### iperf3 between the 2 VMs

# iperf3 -c 10.137.0.88 -p 1000 -t 65
...
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-65.00 sec 37.4 GBytes 4.94 Gbits/sec 1158 sender
[ 5] 0.00-65.01 sec 37.4 GBytes 4.94 Gbits/sec receiver

load average: 1.44, 0.49, 0.17


======================================================================

Antispoof in ingress
--------------------

This is what the modified vif-route-qubes creates. The policy is 'drop'
because only traffic from the original (non-spoofed) IP address is
allowed anyway and the chain has no other function:

# nft list table netdev antispoof
table netdev antispoof {
chain antispoof-vif19-0-10-137-0-11 {
type filter hook ingress device "vif19.0" priority -500; policy drop;
iifgroup 2 ip saddr 10.137.0.11 accept
counter packets 13 bytes 796
}
}

Before testing, remove unused rules too:

# nft flush chain ip qubes antispoof
# nft flush chain ip6 qubes antispoof
# nft flush chain ip qubes prerouting
# nft flush chain ip6 qubes prerouting


### Spoof

# time (timeout 65s hping3 10.137.0.88 --flood --spoof 10.137.0.88)
HPING 10.137.0.88 (eth0 10.137.0.88): NO FLAGS are set, 40 headers + 0 data bytes
hping in flood mode, no replies will be shown

--- 10.137.0.88 hping statistic ---
13443086 packets transmitted, 0 packets received, 100% packet loss
round-trip min/avg/max = 0.0/0.0/0.0 ms

real 1m5.016s
user 0m5.999s
sys 0m47.699s

load average: 0.03, 0.13, 0.09

----------------------------------------------------------------------

### iperf3 between the 2 VMs

# iperf3 -c 10.137.0.88 -p 1000 -t 65
...
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-65.00 sec 37.4 GBytes 4.95 Gbits/sec 887 sender
[ 5] 0.00-65.01 sec 37.4 GBytes 4.95 Gbits/sec receiver

load average: 1.21, 0.47, 0.22

======================================================================

Side by side summary:

Spoof:

current Qubes firewall: load average: 0.21
antispoof in ingress: load average: 0.03


iperf3:

current Qubes firewall: 4.94 Gbits/sec, load average: 1.44
antispoof in ingress: 4.95 Gbits/sec, load average: 1.21

======================================================================

The code:

# This is the new /etc/qubes/qubes-antispoof.nft:

#!/usr/sbin/nft -f

table netdev antispoof
delete table netdev antispoof

table netdev antispoof {
}

I am attaching the modified vif-route-qubes.

[qubist]

Marek,

It's been quite some time.
Should I hope for feedback?

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Ok, so it does improve performance a bit. The scenario of flooding with
spoofed traffic is not very realistic, but does show the impact. But
even on more realistic scenario of allowed traffic, the difference is
noticeable. IMO worth changing.

> ======================================================================
>
> The code:
>
> # This is the new /etc/qubes/qubes-antispoof.nft:
>
> #!/usr/sbin/nft -f
>
> table netdev antispoof
> delete table netdev antispoof
>
> table netdev antispoof {
> }
>
> I am attaching the modified vif-route-qubes.

As for the implementation, few remarks:
- - you create separate chain per IP, each with policy drop - it will
fail for multiple IPs (for example both IPv4 and IPv6) - it should be
one chain with possibly multiple rules (or one with a set?)
- - "downstream" map is gone (without a replacement using the new method),
opening spoofing on eth0 - see QSB-056 for details:
https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-056-2019.txt

If you open a pull request on github, we have quite extensive set of
tests I can schedule from there.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZOiPQACgkQ24/THMrX
1yzCXQf/SvshTsK2mloZ7XngZRvQw4jBZa2STU2eNAp4yC94VkSCLsu4SrM8GzLw
Hy940Pm5Z6IBSpkIjaqvcZ+DUqmd5dipZk8BvwehqAG947hTiAVoNly+R4sMArs9
mw9YT5o+tFDvelrbd+R4glRPhWPZhl1flILG2GU75WVqJnJGnM1sXlZM6YTdMGOA
U1j02SJE6tlA7pb145Aqeg69c8pdWrL5HBu/D/Fe8ACAEeWQnTgBXcbF9FKePkNf
8OjRa4JfKg6Z0dtaDnWpIaB3KSclSPm/ldSpssyYxEJBd47WuwQ0sOF/fmnVy4wo
X+DlqaElycm1msmqIvSnuIlqERYm6A==
=VdWn
-----END PGP SIGNATURE-----

[qubist]

On Thu, 23 May 2024 02:08:20 +0200 Marek Marczykowski-Górecki wrote:

> As for the implementation, few remarks:
> - - you create separate chain per IP, each with policy drop - it will
> fail for multiple IPs (for example both IPv4 and IPv6)

What do you mean by fail?

The suggested implementation successfully creates separate
uniqely-named chains for the IPv4 and IPv6 addresses of the same vif
and automatically removes them upon vif's disconnect. Example:

# nft list table netdev antispoof
table netdev antispoof {

chain antispoof-vif16-0-10-137-0-91 {
type filter hook ingress device "vif16.0" priority -500; policy drop;
iifgroup 2 ip saddr 10.137.0.91 accept
counter packets 0 bytes 0
}

chain antispoof-vif16-0-fd09-24ef-4179-a89-5b {
type filter hook ingress device "vif16.0" priority -500; policy drop;
iifgroup 2 ip6 saddr fd09:24ef:4179::a89:5b accept
counter packets 8 bytes 640

}
}

> - it should be one chain with possibly multiple rules (or one with a
> set?)

Using a single chain for all vifs and IP addresses is quite challenging
for the following reasons:

1. It would practically require a replacement of the whole chain upon
each connect/disconnect of a new vif. => A more complex algorithm.

2. Rules need to be more complex as mapping of the interface and the
IP address would be required.

3. I suppose (my speculation) that may introduce a security problem.

Although in nftables there is this concept of atomic processing, there
may be a very short time during the "switching" from the old to the new
version of the chain, where spoofing of all connected qubes may be
possible. In a high-performance attack, this may theoretically allow to
exploit such vulnerability e.g. through a high-frequency execution of
something like "ip link set eth0 down && ip link set eth0 up" inside
one qube, and flood with spoofed packets from another with the idea to
sneak those in during the mentioned short time. This kind of DoS-y
procedure would be compromising the protection of the firewall qube. I
don't know how realistic or effective that may be, just sharing my
first thoughts.

Unless there are strong facts against the reasons above, it seems to me
safer and simpler to have 2 stable and non-changeable chains per vif.

In case I have misunderstood your idea, please let me know.



There is something else that bothers me, which I would like to discuss
with you:

While working on a more meticulous IPv6 filtering, I noticed the
following. This is both about existing implementation, as well as about
the suggested one (which simply inherits the algorithm):

Certain ICMPv6 messages (Router Advertisement and Redirect), currently
cannot transit the firewall, because, as per RFC 4861, they MUST have a
link-local source address. Considering the ${ip} address list, sent by
Xen to the script, does not contain these addresses, the antispoofing
mechanism blocks the possibility for creating a fully functional local
IPv6 network between 2 or more qubes without hacking the nftables chain
structure and custom scripting to detect vif name etc. This BTW makes
the existing _icmpv6 chain simply unreachable by any external packet.


I also notice that all qubes receive the same link-local address for
eth0:

inet6 fe80::216:3eff:fe5e:6c00/64

obviously based on the same MAC address they receive too.

This practically means that to distinguish packets during filtering,
matching of the interface name (which is dynamic) is also required and
that makes things fairly complicated if qube A filters traffic from
qube B. This seems similar to issue #8833, just for a different
component.

So, I wonder what is the correct approach in regards to that. On one
hand, it is very much desired to disallow qube-to-qube network chatter
by default, even if the qubes have IPv6 enabled. On the other, there
needs to be a mechanism allowing the user to easily configure a custom
IPv6 qubes-LAN.

I would very much like to know what you think about that.

> - - "downstream" map is gone (without a replacement using the new
> method), opening spoofing on eth0 - see QSB-056 for details:
> https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-056-2019.txt

Are you suggesting that we should antispoof eth0 too? The bigger
firewall hardening I am working on already includes eth0 bogon
filtering. In case you mean something else, please explain.

> If you open a pull request on github, we have quite extensive set of
> tests I can schedule from there.

I will see how to do that but let's please clarify first. If you can
share what these tests are, I could probably run them locally too.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Thu, May 23, 2024 at 09:39:55AM -0000, qubist wrote:
> On Thu, 23 May 2024 02:08:20 +0200 Marek Marczykowski-Górecki wrote:
>
> > As for the implementation, few remarks:
> > - - you create separate chain per IP, each with policy drop - it will
> > fail for multiple IPs (for example both IPv4 and IPv6)
>
> What do you mean by fail?

I mean one of them will drop packets that would be allowed by the
other. So, no traffic will be allowed.

> The suggested implementation successfully creates separate
> uniqely-named chains for the IPv4 and IPv6 addresses of the same vif
> and automatically removes them upon vif's disconnect. Example:
>
> # nft list table netdev antispoof
> table netdev antispoof {
> chain antispoof-vif16-0-10-137-0-91 {
> type filter hook ingress device "vif16.0" priority -500; policy drop;
> iifgroup 2 ip saddr 10.137.0.91 accept
> counter packets 0 bytes 0
> }
>
> chain antispoof-vif16-0-fd09-24ef-4179-a89-5b {
> type filter hook ingress device "vif16.0" priority -500; policy drop;
> iifgroup 2 ip6 saddr fd09:24ef:4179::a89:5b accept
> counter packets 8 bytes 640
> }
> }
>
> > - it should be one chain with possibly multiple rules (or one with a
> > set?)
>
> Using a single chain for all vifs and IP addresses is quite challenging
> for the following reasons:

No, I mean one chain per vif (but for all its IPs), instead of one per
IP.

(...)

> There is something else that bothers me, which I would like to discuss
> with you:
>
> While working on a more meticulous IPv6 filtering, I noticed the
> following. This is both about existing implementation, as well as about
> the suggested one (which simply inherits the algorithm):
>
> Certain ICMPv6 messages (Router Advertisement and Redirect), currently
> cannot transit the firewall,

Those are actually intentional, we don't do dynamic IP configuration on
purpose (attack surface reduction), and even if for some reason one VM
would try to sent such packets (could be an attempt to change
sys-firewall's configuration by one of AppVMs...), they should be
blocked. Same for Redirect.

> because, as per RFC 4861, they MUST have a
> link-local source address. Considering the ${ip} address list, sent by
> Xen to the script, does not contain these addresses, the antispoofing
> mechanism blocks the possibility for creating a fully functional local
> IPv6 network between 2 or more qubes without hacking the nftables chain
> structure and custom scripting to detect vif name etc. This BTW makes
> the existing _icmpv6 chain simply unreachable by any external packet.

The link-local traffic should not be forwarded anyway. And definitely
shouldn't be used to communicate between two separate qubes.

> I also notice that all qubes receive the same link-local address for
> eth0:
>
> inet6 fe80::216:3eff:fe5e:6c00/64
>
> obviously based on the same MAC address they receive too.

Yes, but the MAC address can be changed (see "mac" property). And the
vif script should have access to it too I think.

> This practically means that to distinguish packets during filtering,
> matching of the interface name (which is dynamic) is also required and
> that makes things fairly complicated if qube A filters traffic from
> qube B. This seems similar to issue #8833, just for a different
> component.
>
> So, I wonder what is the correct approach in regards to that. On one
> hand, it is very much desired to disallow qube-to-qube network chatter
> by default, even if the qubes have IPv6 enabled. On the other, there
> needs to be a mechanism allowing the user to easily configure a custom
> IPv6 qubes-LAN.

Still, link-local addresses shouldn't be used to communicate between two
different links. There may be some cases where app-qube <-> sys-firewall
communication is affected by the unexpected filtering of all link-local
traffic, but I don't think it affects anything in practice (haven't seen
any issues). That said, allowing specific link-local address in the
antispoofing rules (and later filter undesirable packets using normal
rules) might be a good idea.

> I would very much like to know what you think about that.
>
> > - - "downstream" map is gone (without a replacement using the new
> > method), opening spoofing on eth0 - see QSB-056 for details:
> > https://github.com/QubesOS/qubes-secpack/blob/master/QSBs/qsb-056-2019.txt
>
> Are you suggesting that we should antispoof eth0 too?

Yes, the change you propose should not regress one of the issues
classified as security bugs before...
For example, sys-net should not be able to pretend to be one of your
AppVMs. That's especially relevant if you'd configure sys-firewall to
allow traffic between some AppVMs.

> The bigger
> firewall hardening I am working on already includes eth0 bogon
> filtering. In case you mean something else, please explain.
>
> > If you open a pull request on github, we have quite extensive set of
> > tests I can schedule from there.
>
> I will see how to do that but let's please clarify first. If you can
> share what these tests are, I could probably run them locally too.

Sure: https://www.qubes-os.org/doc/automated-tests/
For network specifically, there is qubes.tests.integ.network
and qubes.tests.integ.network_ipv6.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZPFKIACgkQ24/THMrX
1ywunQf/dKBcABY6r9ojW1M6fl58OtBsgWaZp7TTEIVXZ9oUpos6srpFA0tXfVuL
99zeGb/Wi4nU0nlwGv2cQlIPd1ZeUXa2aZMbF0EyLh2M1VYRg3K1tOvgqiqOzwjL
6kG8wrvlucf+mWyjf2QzUWV7/lzs+h79yFix81uZLHseb8SqsvReOzlCUcIWgIQ+
V2wWrJzN9QenmJIWRkprv8hto7CLmB/46iJMq+xv5yuuYyvXwL2RW9C9jSYd/S+E
1fL/B/y5c/HMGkmAjMnFzR2rApfW6/6kavaycq0fLXcf9/4jh3e6Ph1rpEPnpUnj
Dg2tmtL1afXsZAIMun5H760h6S036A==
=nfpb
-----END PGP SIGNATURE-----

[qubist]

On Thu, 23 May 2024 12:04:18 +0200 Marek Marczykowski-Górecki wrote:

> I mean one of them will drop packets that would be allowed by the
> other. So, no traffic will be allowed.

Indeed. I will fix that.

> Those are actually intentional, we don't do dynamic IP configuration
> on purpose (attack surface reduction), and even if for some reason
> one VM would try to sent such packets (could be an attempt to change
> sys-firewall's configuration by one of AppVMs...), they should be
> blocked. Same for Redirect.

According to RFC 4890 section 4.4.1 (local configuration traffic),
router advertisement messages (ICMPv6 type 134) must not be dropped by
the end host (Qubes drops them). In 4.3.3 (transit traffic), they list
both types (134 and 137) as "Traffic That Will Be Dropped Anyway -- No
Special Attention Needed" and in the bash script in the appendix they
actually drop them + quite a few other types - traffic which Qubes
allows. So, except for type 137 (redirect), there is some significant
discrepancy between these recommendations and Qubes.

That, of course, is off-topic we can discuss later. I am just
mentioning it along the lines of what should/not be dropped by the
antispoof and its potential effect on IPv6.

> Still, link-local addresses shouldn't be used to communicate between
> two different links.

If I am reading the RFC correctly, they are not used for communicating
(i.e. fully-fledged data transfer), but rather for establishing
communication. I am not a network expert, so I am not closely familiar
with all the inner workings of IPv6.

Section 6.1.2 of RFC 4861 says:

- IP Source Address is a link-local address. Routers must use
their link-local address as the source for Router Advertisement
and Redirect messages so that hosts can uniquely identify
routers.

Assuming sys-firewall is the router, perhaps there needs to be a way to
advertise it. However, that is a data-collision problem because all
qubes have the same link-local address and the node would confuse its
own with that of sys-firewall. No?

> That said, allowing specific link-local address in the antispoofing
> rules (and later filter undesirable packets using normal rules) might
> be a good idea.

My thoughts too. The question is what is the appropriate way to do
this, considering that currently all MAC addresses are the same => all
link-local IPv6 addresses too. IOW, what would be the mechanism of
getting the specific link-local address, so that it can be used in the
antispoofing rules?

> > Are you suggesting that we should antispoof eth0 too?
>
> Yes, the change you propose should not regress one of the issues
> classified as security bugs before...

It does not propose that. It does not allow traffic which is blocked by
current implementation.

> For example, sys-net should not be able to pretend to be one of your
> AppVMs. That's especially relevant if you'd configure sys-firewall to
> allow traffic between some AppVMs.

In my mentioned bigger firewall hardening setup, which may probably fit
better in a separate thread, I have this:

table netdev hardening {
chain ingress {
type filter hook ingress device "eth0" priority -500;
ip saddr @bogons_ipv4 counter drop
ip6 saddr @bogons_ipv6 counter drop
# ... fragments handling
# ... early invalid drop
# ... TCP MSS segment filtering
# ... SYN tarpit
}
# ...
}

and the sets are populated using:

https://team-cymru.org/Services/Bogons/fullbogons-ipv4.txt
https://team-cymru.org/Services/Bogons/fullbogons-ipv6.txt

Qubes' internal IP addresses are covered by these lists.

> Sure: https://www.qubes-os.org/doc/automated-tests/
> For network specifically, there is qubes.tests.integ.network
> and qubes.tests.integ.network_ipv6.

Thanks. This is somewhat complicated for me.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Thu, May 23, 2024 at 12:33:02PM -0000, qubist wrote:
> On Thu, 23 May 2024 12:04:18 +0200 Marek Marczykowski-Górecki wrote:
>
> > I mean one of them will drop packets that would be allowed by the
> > other. So, no traffic will be allowed.
>
> Indeed. I will fix that.
>
> > Those are actually intentional, we don't do dynamic IP configuration
> > on purpose (attack surface reduction), and even if for some reason
> > one VM would try to sent such packets (could be an attempt to change
> > sys-firewall's configuration by one of AppVMs...), they should be
> > blocked. Same for Redirect.
>
> According to RFC 4890 section 4.4.1 (local configuration traffic),
> router advertisement messages (ICMPv6 type 134) must not be dropped by
> the end host (Qubes drops them). In 4.3.3 (transit traffic), they list
> both types (134 and 137) as "Traffic That Will Be Dropped Anyway -- No
> Special Attention Needed" and in the bash script in the appendix they
> actually drop them + quite a few other types - traffic which Qubes
> allows. So, except for type 137 (redirect), there is some significant
> discrepancy between these recommendations and Qubes.

There will be some intentional discrepancies, that document describes a
network using IPv6 autoconfiguration (where indeed packets like 133 and
134 are crucial), which we explicitly decide to not use.

> That, of course, is off-topic we can discuss later. I am just
> mentioning it along the lines of what should/not be dropped by the
> antispoof and its potential effect on IPv6.
>
> > Still, link-local addresses shouldn't be used to communicate between
> > two different links.
>
> If I am reading the RFC correctly, they are not used for communicating
> (i.e. fully-fledged data transfer), but rather for establishing
> communication. I am not a network expert, so I am not closely familiar
> with all the inner workings of IPv6.
>
> Section 6.1.2 of RFC 4861 says:
>
> - IP Source Address is a link-local address. Routers must use
> their link-local address as the source for Router Advertisement
> and Redirect messages so that hosts can uniquely identify
> routers.
>
> Assuming sys-firewall is the router, perhaps there needs to be a way to
> advertise it.

RD etc shouldn't really be needed in our case. But neighbor discovery
might.

> However, that is a data-collision problem because all
> qubes have the same link-local address and the node would confuse its
> own with that of sys-firewall. No?

No, link-local addresses don't need to be globally unique, they need to
be unique only within a scope of a _single_ link (eth0<->vif pair in
this case). Linux (or any other routing-capable OS for that matter)
needs to consider which interface to use for which control packet anyway
- - in fact, I think it's not valid to target link local address without
explicit interface name (on which link you want it to be). For example
ping warns about it:

$ ping6 fe80::fcff:ffff:feff:ffff
ping6: Warning: IPv6 link-local address on ICMP datagram socket may require ifname or scope-id => use: address%<ifname|scope-id>

> > That said, allowing specific link-local address in the antispoofing
> > rules (and later filter undesirable packets using normal rules) might
> > be a good idea.
>
> My thoughts too. The question is what is the appropriate way to do
> this, considering that currently all MAC addresses are the same => all
> link-local IPv6 addresses too. IOW, what would be the mechanism of
> getting the specific link-local address, so that it can be used in the
> antispoofing rules?

Build it based on the qube MAC address (which should be known to the vif
script already)? Those antispoofing rules are meant only to prevent qube
using IP that it's not allowed to use. So, for IPv6 a qube would be
allowed to use either its "main" IPv6 or its link-local IPv6. Those
rules are not meant to ensure any uniqueness, and link-local addresses
don't need to be unique across different links.

> > > Are you suggesting that we should antispoof eth0 too?
> >
> > Yes, the change you propose should not regress one of the issues
> > classified as security bugs before...
>
> It does not propose that. It does not allow traffic which is blocked by
> current implementation.

Are you sure? The current implementation has this rule in prerouting:

ip saddr @downstream counter drop

and a matching "downstream" set. Your version removes that without
equivalent replacement.

> > For example, sys-net should not be able to pretend to be one of your
> > AppVMs. That's especially relevant if you'd configure sys-firewall to
> > allow traffic between some AppVMs.

I think you can do an experiment:
1. Create two qubes (I'll name them test1 and test2).
2. Add a rule in sys-firewall that would allow traffic from test2 to
test1 and vice versa: https://www.qubes-os.org/doc/firewall/#enabling-networking-between-two-qubes
3. Start test1 and observe incoming traffic on its eth0 (you may want to
clean its firewall for the experiment, to allow all).
4. Do _not_ start test2.
5. Try to spoof IP of test2 from sys-net - like, add to the vif interface
going to sys-firewall and try to use as a source when pinging test1 from
there.

This should normally be blocked, but I think your version won't block
it.

> In my mentioned bigger firewall hardening setup, which may probably fit
> better in a separate thread, I have this:
>
> table netdev hardening {
> chain ingress {
> type filter hook ingress device "eth0" priority -500;
> ip saddr @bogons_ipv4 counter drop
> ip6 saddr @bogons_ipv6 counter drop
> # ... fragments handling
> # ... early invalid drop
> # ... TCP MSS segment filtering
> # ... SYN tarpit
> }
> # ...
> }
>
> and the sets are populated using:
>
> https://team-cymru.org/Services/Bogons/fullbogons-ipv4.txt
> https://team-cymru.org/Services/Bogons/fullbogons-ipv6.txt
>
> Qubes' internal IP addresses are covered by these lists.

Well, this is too broad, as for example sys-net is allowed to use its
own IP to send packets down the network (like to sys-firewall or other
qubes). This happens for example for ICMP error packets. It would also
break any communication to your LAN (like, using network printer in your
LAN)...

This list may work on internet-only router without any kind of LAN
involved only.

> > Sure: https://www.qubes-os.org/doc/automated-tests/
> > For network specifically, there is qubes.tests.integ.network
> > and qubes.tests.integ.network_ipv6.
>
> Thanks. This is somewhat complicated for me.

It is a bit (at least for the first time). The tl;dr is:

sudo systemctl stop qubesd; sudo -E python3 -m qubes.tests.run qubes.tests.integ.network; sudo systemctl start qubesd

But note it will remove any qube with name starting with "test-" prefix.

Anyway, that's why I asked for a PR, as we have scripts doing this all
automatically.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZPSmMACgkQ24/THMrX
1yxXsAf9HDxIJQdd8LVWEOb4FmrLNZAr8FLxK7FTE/2OrUBloQWNxtWvpdy3vZ4i
5EMT1vGetm5PgJrVi8M/5tiRjJHbYRsPMknXPojtYkL+D5d/2ymS2sKRPAmCbduw
QItUMLLNkbtNixm1dOBbpdDPmguw20bd0G5+cIK0dFj7tYcYNK59mNY861eAd6Bb
CuY8dkcKnAndaRApV3ouF0UIQjgxgEYU+tKOLDzem+FroTWuKKzlGbXc+1rOICPb
F4YWnwPoyZedwLsMZznfedxmgF3Nz777ivXzDz4s5qTkndYGlnDMw/ONdqyoMMkl
9QMxaEN/Dm2bo2bBdO1PJwktk8twzw==
=+6DR
-----END PGP SIGNATURE-----

[qubist]

On Thu, 23 May 2024 15:53:39 +0200 Marek Marczykowski-Górecki wrote:

> There will be some intentional discrepancies, that document describes
> a network using IPv6 autoconfiguration (where indeed packets like 133
> and 134 are crucial), which we explicitly decide to not use.

Thanks for explaining. I will get back to ICMPv6 later.

> ping6: Warning: IPv6 link-local address on ICMP datagram socket
> may require ifname or scope-id => use: address%<ifname|scope-id>

Interesting. I don't see this warning. Debian 12 template.

> Build it based on the qube MAC address (which should be known to the
> vif script already)? Those antispoofing rules are meant only to
> prevent qube using IP that it's not allowed to use. So, for IPv6 a
> qube would be allowed to use either its "main" IPv6 or its link-local
> IPv6. Those rules are not meant to ensure any uniqueness, and
> link-local addresses don't need to be unique across different links.

So, it is doable then. I will see how exactly.

> Are you sure? The current implementation has this rule in prerouting:
>
> ip saddr @downstream counter drop
>
> and a matching "downstream" set. Your version removes that without
> equivalent replacement.

I will test this again to double check. However, the way I understand
it:

- Current implementation: the policy is accept and the device is not
defined explicitly (prerouting). That's why it is necessary to
explicitly drop saddr @downstream and to explicitly match the address
against the interface.

- Proposed implementation: the policy is drop and the chain is bound to
the device (netdev). This means only the explicitly allowed IP source
addresses on the particular interface will be accepted, i.e. this is
the equivalent of the allowed map. As for downstream => no need to
define or drop a downstream set, because of the drop policy.

Please let me know if I misunderstood anything.

> Well, this is too broad, as for example sys-net is allowed to use its
> own IP to send packets down the network (like to sys-firewall or other

> qubes). [...]

> This list may work on internet-only router without any kind of LAN
> involved only.

True. There is a reason why the Team Cymru adds a warning to know one's
network.

It is possible to use a whitelist and a permissive rule before bogon
dropping, then process these particular packets later down the hooks.
Creating this whitelist has a usability burden, so perhaps it can be
defined through some kind of UI, similar to that for "Default and
service qubes" in global config. What do you say?

> > > Sure: https://www.qubes-os.org/doc/automated-tests/
> > > For network specifically, there is qubes.tests.integ.network
> > > and qubes.tests.integ.network_ipv6.
> >
> > Thanks. This is somewhat complicated for me.
>
> It is a bit (at least for the first time). The tl;dr is:

What is the proper way to do it only inside a domU? I prefer not to
mess with dom0, as this is not a test-only system.

[qubist]

BTW, I found something while watching journal:

This line:

echo 1 >"/proc/sys/net/ipv6/conf/${vif}/proxy_ndp"

always results in an error messages in journal upon removal of the vif
(removing netvm from the qube in dom0). It doesn't seem to cause any
observable issue, however it would be good to have some check to avoid
false alarms in logs. Or, if it is safe, adding a " || true" at the end
works too.

Also, shellcheck shows 2 warnings about unused variables.

There are some other possible improvements about the code as a whole,
e.g. "${variable}" is better than "$varible", conditional blocks and
loops are quite lengthy, etc. So, some refactoring might be a good
idea. I am not touching that, just saying, as I have no idea how and
when this was created.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Yes, but also note the antispoof chain (used for all vif+ interfaces)
ends with a drop rule.
So, for vif only explicitly allowed (source) IPs are accepted, others
are blocked. And for eth0 it's the opposite: IPs used elsewhere are
blocked and others are allowed.

> - Proposed implementation: the policy is drop and the chain is bound to
> the device (netdev). This means only the explicitly allowed IP source
> addresses on the particular interface will be accepted, i.e. this is
> the equivalent of the allowed map. As for downstream => no need to
> define or drop a downstream set, because of the drop policy.

Can you clarify your description of the downstream set here? I think you
do need to have some drop rule somewhere.

> Please let me know if I misunderstood anything.
>
> > Well, this is too broad, as for example sys-net is allowed to use its
> > own IP to send packets down the network (like to sys-firewall or other
> > qubes). [...]
> > This list may work on internet-only router without any kind of LAN
> > involved only.
>
> True. There is a reason why the Team Cymru adds a warning to know one's
> network.
>
> It is possible to use a whitelist and a permissive rule before bogon
> dropping, then process these particular packets later down the hooks.
> Creating this whitelist has a usability burden, so perhaps it can be
> defined through some kind of UI, similar to that for "Default and
> service qubes" in global config. What do you say?

That can be as some optional configuration user can opt-in to. I don't
want this part in the default rules, even if user could adjust it make
it work in their particular network.

> > > > Sure: https://www.qubes-os.org/doc/automated-tests/
> > > > For network specifically, there is qubes.tests.integ.network
> > > > and qubes.tests.integ.network_ipv6.
> > >
> > > Thanks. This is somewhat complicated for me.
> >
> > It is a bit (at least for the first time). The tl;dr is:
>
> What is the proper way to do it only inside a domU? I prefer not to
> mess with dom0, as this is not a test-only system.

Those tests are written to be launched from dom0. They create new qubes
(with "test-" prefix) for performing the test, and then remove them
afterwards. In fact, it operates on a copy of qubes.xml, the original
one remains unchanged. Qubes other than with "test-" prefix are not
changed. But it's not safe to do other stuff when a test is running.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZQnSgACgkQ24/THMrX
1yyMQQf9Hmsfz1784CHnBqY5kqcRf25ONajWYRewDMdhyQ/cA9MhcUZ015/H/7WK
sdysvQuTMHiFyBSApET+Op0KjpmIEeUgYaH7lxMOQ7anFB5ZSs+5w557gDik2Lb8
qMsgDtIy8G6WiEQoCOZb3AMdvEeM9ZnikGMy3NPYbLBdK0vg6v3tCqZqaiQxKbKe
/HIRSA4OW4lw6Kr5IrZaS/voNl+oroAkL4JROKU6atJM7gtGLfv9pr3kU2RFVyX0
KfwuIJkd2MP6GSXp+pLl14ZDRrfzmCiAUH+NoeCjkiOdAZcLLu3wC5x8OsgxtcQR
InlvcbbO3efRftm0BMG5pSqGbU+z3g==
=5xvf
-----END PGP SIGNATURE-----

[qubist]

On Fri, 24 May 2024 15:59:04 +0200 Marek Marczykowski-Górecki wrote:

> Yes, but also note the antispoof chain (used for all vif+ interfaces)
> ends with a drop rule.
> So, for vif only explicitly allowed (source) IPs are accepted, others
> are blocked. And for eth0 it's the opposite: IPs used elsewhere are
> blocked and others are allowed.

Yes, I understand.

To do this downstream thing in ingress, we should use a separate
ingress chain for eth0. I have already done the per-vif chain, as you
suggested. I will do this too. Please give me some time, as I am quite
busy these days. I will send back the updated script, so you can have a
look.

> > True. There is a reason why the Team Cymru adds a warning to know
> > one's network.
> >
> > It is possible to use a whitelist and a permissive rule before bogon
> > dropping, then process these particular packets later down the
> > hooks. Creating this whitelist has a usability burden, so perhaps
> > it can be defined through some kind of UI, similar to that for
> > "Default and service qubes" in global config. What do you say?
>
> That can be as some optional configuration user can opt-in to. I don't
> want this part in the default rules, even if user could adjust it make
> it work in their particular network.

I understand you want to avoid extra complexity and that makes sense.
Re. your earlier:

> Well, this is too broad, as for example sys-net is allowed to use its
> own IP to send packets down the network (like to sys-firewall or other

> qubes). This happens for example for ICMP error packets. It would also
> break any communication to your LAN (like, using network printer in
> your LAN)...
>

> This list may work on internet-only router without any kind of LAN
> involved only.

The problem is that this implies trust in the infrastructure and in the
[generally distrusted] sys-net too, i.e. a compromised device/qube can
send malicious traffic and the current permissive implementation allows
everything except the downstream, it means unwanted packets travel all
the way to the filter hook (including bogons), rather than be dropped
early.

Speaking of this, are there any plans for D/DoS protection for Qubes?
(XDP, BPF)

[qubist]

On Fri, 24 May 2024 15:59:04 +0200 Marek Marczykowski-Górecki wrote:

> Can you clarify your description of the downstream set here? I think you
> do need to have some drop rule somewhere.

Here is what the updated script gives for 2 client qubes: one IPv4-only
and one IPv6:

table netdev antispoof {
set downstream_ipv4 {
type ipv4_addr
elements = { 10.137.0.11, 10.137.0.88 }
}

set downstream_ipv6 {
type ipv6_addr
elements = { fd09:24ef:4179::a89:b }
}

chain downstream {
type filter hook ingress device "eth0" priority -500; policy accept;
ip saddr @downstream_ipv4 counter packets 0 bytes 0 drop
ip6 saddr @downstream_ipv6 counter packets 0 bytes 0 drop
}

chain antispoof-vif18-0 {
type filter hook ingress device "vif18.0" priority -500; policy drop;
iifgroup 2 ip saddr 10.137.0.11 accept
iifgroup 2 ip6 saddr fd09:24ef:4179::a89:b accept
counter packets 14 bytes 968
}

chain antispoof-vif30-0 {
type filter hook ingress device "vif30.0" priority -500; policy drop;
iifgroup 2 ip saddr 10.137.0.88 accept
counter packets 11 bytes 704
}
}

Maybe we should get rid of the "iifgroup 2", since it is clear which
exact device each chain handles?

[Demi Marie Obenour]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Yes, the "iffgroup 2" is redundant in this case. I initially added
interface groups to support VPN setups, as matching on them is faster
and simpler than matching on source interface by name.

How do you plan to handle sys-net and VPN qubes? These have additional
interfaces that are not managed by Qubes OS, and they might also try to
inject spoofed packets. The simplest solution is something like

chain antispoof_downstream {
type filter hook prerouting priority raw; policy accept;
iifgroup != 2 ip saddr @downstream counter drop
}

but this might defeat the performance improvements as it brings back a
prerouting hook. A netdev hook on every interface would work, but it
would need to block the interface being brought online if something went
wrong, and I’m not aware of any way to do that.
- --
Sincerely,
Demi Marie Obenour (she/her/hers)

Invisible Things Lab
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEdodNnxM2uiJZBxxxsoi1X/+cIsEFAmZWQ28ACgkQsoi1X/+c
IsHUMA//UjP30UKPx5xIl+ZZZlBA+wEJ+pxXvsm3pdFDhqc3YtIvLQYJKvLzJrHJ
0POMm1s1ySb8coSSduE1+CK3Hk1jSEerP87tbbBJ1jUmkBugYSMNRHEBMMGmQWlg
wzfeSp/+kr0UzQ8LP/Y3i7/xP3kCgKDAMVa5Q57SUJW6FeTRqUNTlRfeqBScwK21
n/lKRnc1LR3S/bPB+je+TDGnKNqXxgBOdkReHXT6VlOPRHBqmr0OeZUgMRPmR6rx
d2pnz+0hJkjJ5mQc3Qx2X8fhEe5wWvsX+L5XT0+BIFmbohGzXkoar4sdlopSLO/q
88VuudkkAeP1USE0PfGPjhi1S9yvVFPalSSgAvOVlT7Ho/DpZJxA7ZPkmLHyt709
DDUSoNRRB4M5LLh2DREOUPHNz+rMtK1ETMzSgmEHP+x1TaF1Dplh1Kh9uL+M75ox
ptdpcjDy6W5iyeFSLYk69V/rJxHUTaVbcddfVolvEvUrg6JSzkjJbyNf4V5LvCgy
2katS4gOSdLiB3q6TLhv1q8WqWqPQcfObb71Yq+k//ltKxgOZqKdhLavIiBQ3lSI
DieWpNIazDH4ikuFIbSjLJKHyPjc2kXIfca276wp18tKhYyZD6nf5d52jq1HhNf8
vGiDGOR0OnEOiJzjZsUVU5Pmjaf60j/BEGRMTBuqNgAzhlob7hA=
=wFwP
-----END PGP SIGNATURE-----

[qubist]

On Tue, 28 May 2024 16:49:51 -0400 Demi Marie Obenour wrote:

> additional interfaces that are not managed by Qubes OS

Why not?
Can't we have them managed, similar to the vifs?

> to block the interface being brought online if something went wrong

Could you elaborate?

[Demi Marie Obenour]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On Wed, May 29, 2024 at 04:07:45PM -0000, qubist wrote:
> On Tue, 28 May 2024 16:49:51 -0400 Demi Marie Obenour wrote:
>
> > additional interfaces that are not managed by Qubes OS
>
> Why not?
> Can't we have them managed, similar to the vifs?

Qubes OS only knows how to manage Xen interfaces. It does not (and,
realistically, cannot) know how to manage every kind of network
interface imaginable.

> > to block the interface being brought online if something went wrong
>
> Could you elaborate?

If there is an error setting up the anti-spoofing firewall rules, the
network interface must not be brought up at all, to prevent a spoofing
vulnerability.

- --
Sincerely,
Demi Marie Obenour (she/her/hers)
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEdodNnxM2uiJZBxxxsoi1X/+cIsEFAmZXeTcACgkQsoi1X/+c
IsHIxw//W9Ri+GHmabTqGQWiVcvAT76gJFWZIEpUVkHJ4e3R9yXpQpcI+8EfLnQ6
bZe6nq2QGsKxMA2hvkzo70Oh/a1oY/cyY9iOPCBD463Vi3y9f5SxoWce2Z0AdEfK
UFou1LuELyQoQqfXEob5Zl8BeWN9cWFieC6HJwHe7fFetacsQKAfHei7+n9grEEc
Dc+DKYnX2pdx+Erc0CJMr75nr2wSTAx/0oAVJXdGFK0HMMgOgpoN0i1X9YXk6wEF
F7BIpHZnFaHS8p9vTWxTKYIcPlgz14cxIs/CDZ2RNpRy/F5oaZL29XnzxgkIyJ2V
U+oKO0IyjMvWx+aU7PeBa0kNi2YFKt3AFjQy050D8dbumWMjNIhPvnofeSFqY3rc
+YZQAOfaCo7Ck9zUwRNTsY3NlO+SXOKB1w0jba88qzqMFQ8pfcm8sGjkfIV6Vpu6
UGyNuSEG/MHMGTEN4I97xzLggBYo9OzZ/Zu9lyymnuJ5nfvwIJk3RiST3h6XFWME
h/gCpxc9pQlM5lV7kWTqN3xcVfqjlim1joxC+6B382GGFyCRFqN3MepvUOTL7f1b
/E7vyw6UJqG+8sUuq6w5vJkTjTEMXEFCUxxPjOoJwhwWEFSw8FEcRpYchYBWUodd
PWbEIFvJc+p1SP0pzMyPSwu4hJe6uTtxuRqCMl7Nw4gcvoHu+hY=
=pddU
-----END PGP SIGNATURE-----

[qubist]

On Wed, 29 May 2024 14:51:36 -0400 Demi Marie Obenour wrote:

> Qubes OS only knows how to manage Xen interfaces. It does not (and,
> realistically, cannot) know how to manage every kind of network
> interface imaginable.

In such case, it cannot do this too:

> [...] the network interface must not be brought up at all, to prevent
> a spoofing vulnerability.

as it is not up to Qubes to do it, right?

Or how will it bring an interface up/down if it cannot control it?

As for sys-net and its self-managed (uplink) interfaces: this means any
such interface can have its group changed, thus circumvent the
group-based antispoofing chain you suggest. Actually, sys-net can
simply 'sudo nft flush ruleset' which is extremely easy with default
passwordless root, so does it really matter what firewall rules we
attempt to enforce?

I don't know if I am overthinking it, but for all that to work
properly, an external control mechanism is necessary (another qube). I
don't know if there is ARP antispoofing mechanism either.

I hope you can clarify.

[Demi Marie Obenour]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On Thu, May 30, 2024 at 05:20:04PM -0000, qubist wrote:
> On Wed, 29 May 2024 14:51:36 -0400 Demi Marie Obenour wrote:
>
> > Qubes OS only knows how to manage Xen interfaces. It does not (and,
> > realistically, cannot) know how to manage every kind of network
> > interface imaginable.
>
> In such case, it cannot do this too:
>
> > [...] the network interface must not be brought up at all, to prevent
> > a spoofing vulnerability.
>
> as it is not up to Qubes to do it, right?

Correct.

> Or how will it bring an interface up/down if it cannot control it?
>
> As for sys-net and its self-managed (uplink) interfaces: this means any
> such interface can have its group changed, thus circumvent the
> group-based antispoofing chain you suggest. Actually, sys-net can
> simply 'sudo nft flush ruleset' which is extremely easy with default
> passwordless root, so does it really matter what firewall rules we
> attempt to enforce?

> I don't know if I am overthinking it, but for all that to work
> properly, an external control mechanism is necessary (another qube). I
> don't know if there is ARP antispoofing mechanism either.
>
> I hope you can clarify.

Qubes OS's current firewall treats any interface in group 2 as if it
were a Xen vif* interface, and prevents downstream spoofing by any
interface _not_ in group 2. Common network management packages assign
all devices to group 0 by default, so this works fine.

Being able to assign non-managed devices to group 2 is intentional, not
a bug.

- --
Sincerely,
Demi Marie Obenour (she/her/hers)
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEdodNnxM2uiJZBxxxsoi1X/+cIsEFAmZYygkACgkQsoi1X/+c
IsGSSw/+P7nD30ot5cHsZPgKz5DR6+zX+YYTMjZm1k5hC3sCOKM7AmyyaIxCk4OV
/79JA1OOu/J0IC7vzlrc3o92Ix0/3segH4Ag6XbtCnLAxgJX8K6t5wgg3ySCxmMt
mIRlhUXCTL3t36H7l2wjpLGcq4iMqOUw0mcdudmgEW0YwDkR7GoI1sOZfr4f/kle
QQlnLkvjHojL/Fhit41XyQCwEarHlAVXAZ3ACWMlzBwfwxaaXQ4KM4QZTUnEkRDM
MX6zriV8cOnv5ZnQFgBZ04u7UfQJN/XvpCnp9CKqAJDg9j1QDQiBnUL4rgoqJgDr
vzBysAAgdReB4WoTnHV5DWfnp4GiQgI+Vx4NXOOqMS5LIyIG2cMMkoZVBRP00CWy
0q0ToCHQCpsReQo0xzUvNPnYJPBoSrTmi5JJai4r3Gd9g+W7jKtJ/yrPAJ+VLRMY
cKSRQ8q2rUJhCiwbvqsZHa71o+LOYPQapJ0fvpd45HNJI6WgIVTMpIfspkaVVanM
qDLDyxLIkmWABtWjrMjFaaSb7KlTii7RaM+9wPH6VFlUkYrPADzb7O0+a92xVODj
ezrsgzafQyphacv+rdzIMQJt+msLKYhismuv/75RSndFoGBXT80T1qi3fyYqlSZT
Gk+vMrN0hgdM7OYHzLsaTDo4fduEA48L9LNbT9fJHKhjsLoamBo=
=kHQG
-----END PGP SIGNATURE-----

[qubist]

On Thu, 30 May 2024 14:48:41 -0400 Demi Marie Obenour wrote:

> Correct.

Then:

On Tue, 28 May 2024 16:49:51 -0400 Demi Marie Obenour wrote:

> How do you plan to handle sys-net and VPN qubes?

I can think of 2 options:

1. Stick with prerouting for those interfaces

2. Have some internal (in-qube) monitoring mechanism watching for new
interfaces and create chains based on such events.

The problem with both options is that the firewall running inside
sys-net is just as reliable as sys-net being free from userspace
malware.

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

That's always the case. After all, your ingress rules are managed by
userspace too.

IMO static rules (the first option) is easier and more reliable (no need
for that monitoring mechanism to keep working).

- --
Best Regards,
Marek Marczykowski-Górecki

Invisible Things Lab
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmZaPrsACgkQ24/THMrX
1yyJ8wgAg6pN3GfeqUYsXhnnflE/lNERsyo8DJ/6Y94OUZLFNZsQpFaM5vz0EAZn
bbRsRE74qA7+1Q2+RRDyicgF0tK5Co1SHQ6FY4NjkNlk+eYBIv5+FKy/s8v34Pve
tO+Pf3uIkELJKQrkAzHeatIw+FoqlmScVUWZ9s3e7Y+hruduj2iLP4CV3J6IiT15
TKkjxLUJMA+vDK9Q4pOiykSJAhHBfPQkDByJo7Hf9ZQvxd1T7cVwGrQYfdCa+fF5
F0NC1WGXJ1GG6qmx+Je81Cp0NYC+Dj/Prr82L04rDd9e3mIlCq42m/LKuU98OWF5
gvht3aTmWJ3hio8Ip3LbHfJ5+aKQow==
=BNlh
-----END PGP SIGNATURE-----

[qubist]

On Fri, 31 May 2024 23:18:51 +0200 Marek Marczykowski-Górecki wrote:

> That's always the case. After all, your ingress rules are managed by
> userspace too.

Sorry, I meant user-account malware, which (with passwordless root) can
instantly escalate to root and modify anything (interfaces, firewall).
Considering nothing sits between sys-net and the uplink(s), and it is a
firewall for itself, that makes it particularly vulnerable.

> IMO static rules (the first option) is easier and more reliable (no
> need for that monitoring mechanism to keep working).

Yes.

However, it would be good to have some external control mechanism to
protect sys-net more efficiently. What can be done about that?