IPv6 Policy Routing doesn't work as expected

[Marc Haber]

Hi,

I have a Linux box that is on a network segment with two IPv6-enabled
routers. Each of the routers has Internet connectivity and its own
prefix. I want to route everything via router A except when the Linux
box decides to use a source address that belongs to router B's prefix.
I want the connectivity via one router to continue working even if the
othre router has failed, therefore I cannot set one of the routers as
default gateway and rely on the machine forwarding the traffic and/or
issueing redirects.

Here is my configuration:
|$ ip -6 rule
|0: from all lookup local
|29000: from 2001:db8:42bc:a18e::/64 lookup ka51
|30000: from all lookup main
|$ ip -6 route show table ka51
|2001:db8:42bc:a11d::/64 via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1024 pref medium
|2001:db8:42bc:a180::/59 via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1024 pref medium
|2001:db8:42bc:a1b0::/60 via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1024 pref medium
|default via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1000 pref medium
|$ ip -6 route show table main
|::1 dev lo proto kernel metric 256 pref medium
|2001:16b8:3037:6900::/64 dev unt381 proto ra metric 1024 expires 6801sec pref medium
|2001:16b8:3037:6900::/56 via fe80::cece:1eff:fe29:7745 dev unt381 proto ra metric 1024 expires 1401sec pref medium
|2001:db8:42bc:a18e::/64 dev unt381 proto kernel metric 256 pref medium
|fe80::/64 dev unt381 proto kernel metric 256 pref medium
|default via fe80::cece:1eff:fe29:7745 dev unt381 proto ra metric 1024 expires 1401sec mtu 1492 pref medium
|$

However, this doesn't work. When I ping 2001:db8:42bc:a182::1d:100,
the following packet gets sent out:

|20:38:22.569941 02:48:05:c1:4b:81 > cc:ce:1e:29:77:45, ethertype IPv6 (0x86dd), length 154: 2001:db8:42bc:a18e::9:100 > 2001:db8:42bc:a182::1d:100: ICMP6, echo request, seq 1, length 100

cc:ce:1e:29:77:45 is the MAC address of the default router in the
routing table main, while I would expect the packet to be sent to the
default router in the routing table "ka51".

Why is my routing rule wth preference 29000 not honored by the kernel?

Am I doing something wrong?

Greetings
Marc
--
-------------------------------------- !! No courtesy copies, please !! -----
Marc Haber | " Questions are the | Mailadresse im Header
Mannheim, Germany | Beginning of Wisdom " |
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

[Pascal Hambourg]

Hello,

Le 09/08/2020 à 21:35, Marc Haber a écrit :
>
> |$ ip -6 route show table ka51
> |2001:db8:42bc:a11d::/64 via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1024 pref medium
> |2001:db8:42bc:a180::/59 via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1024 pref medium
> |2001:db8:42bc:a1b0::/60 via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1024 pref medium
> |default via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1000 pref medium

What is the point of the three first routes which have the same
interface and router as the default route ?

(...)

> However, this doesn't work. When I ping 2001:db8:42bc:a182::1d:100,

How do you send the ping ? Do you force the source address ?

[Marc Haber]

Pascal Hambourg <pas...@plouf.fr.eu.org> wrote:
>Le 09/08/2020 à 21:35, Marc Haber a écrit :
>> |$ ip -6 route show table ka51
>> |2001:db8:42bc:a11d::/64 via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1024 pref medium
>> |2001:db8:42bc:a180::/59 via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1024 pref medium
>> |2001:db8:42bc:a1b0::/60 via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1024 pref medium
>> |default via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1000 pref medium
>
>
>What is the point of the three first routes which have the same
>interface and router as the default route ?

Being more specific than the default route just in case the default
route incidentally points to the other uplink. Those three networks
are a bunch of infrastructure LANs that can only be reached via one of
the two routers.

>
>(...)
>> However, this doesn't work. When I ping 2001:db8:42bc:a182::1d:100,
>
>How do you send the ping ? Do you force the source address ?

with /bin/ping from iputils-ping 20180629-2. No, source address not
forced, the source address should be selected automatically and
according to where the chosen route points to.

[Pascal Hambourg]

Le 30/08/2020 à 15:10, Marc Haber a écrit :
> Pascal Hambourg <pas...@plouf.fr.eu.org> wrote:
>> Le 09/08/2020 à 21:35, Marc Haber a écrit :
>>> |$ ip -6 route show table ka51
>>> |2001:db8:42bc:a11d::/64 via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1024 pref medium
>>> |2001:db8:42bc:a180::/59 via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1024 pref medium
>>> |2001:db8:42bc:a1b0::/60 via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1024 pref medium
>>> |default via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1000 pref medium
>>
>>
>> What is the point of the three first routes which have the same
>> interface and router as the default route ?
>
> Being more specific than the default route just in case the default
> route incidentally points to the other uplink.

Err, IIUC this is a user-defined routing table, filled with user-defined
routes. How would its default route point to the other router ?

>> How do you send the ping ? Do you force the source address ?
>
> with /bin/ping from iputils-ping 20180629-2. No, source address not
> forced, the source address should be selected automatically and
> according to where the chosen route points to.

Then AFAIK the source address is only an output of the routing decision
(along with the interface and next hop), not an input key.

[Marc Haber]

Pascal Hambourg <pas...@plouf.fr.eu.org> wrote:
>Le 30/08/2020 à 15:10, Marc Haber a écrit :
>> Pascal Hambourg <pas...@plouf.fr.eu.org> wrote:
>>> Le 09/08/2020 à 21:35, Marc Haber a écrit :
>>>> |$ ip -6 route show table ka51
>>>> |2001:db8:42bc:a11d::/64 via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1024 pref medium
>>>> |2001:db8:42bc:a180::/59 via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1024 pref medium
>>>> |2001:db8:42bc:a1b0::/60 via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1024 pref medium
>>>> |default via 2001:db8:42bc:a18e::70:100 dev unt381 proto static metric 1000 pref medium
>>>
>>>
>>> What is the point of the three first routes which have the same
>>> interface and router as the default route ?
>>
>> Being more specific than the default route just in case the default
>> route incidentally points to the other uplink.
>
>Err, IIUC this is a user-defined routing table, filled with user-defined
>routes. How would its default route point to the other router ?

Think an automatism to change routing in some situations, e.g. an
uplink or router failure. This example is stripped down to be
understandeable.

>>> How do you send the ping ? Do you force the source address ?
>>
>> with /bin/ping from iputils-ping 20180629-2. No, source address not
>> forced, the source address should be selected automatically and
>> according to where the chosen route points to.
>
>Then AFAIK the source address is only an output of the routing decision
>(along with the interface and next hop), not an input key.

This is unhelpful nitpicking, obviously the routing decision is taken
before the source address is selected and the mechanisms dont consider
that the chosen route is the wrong one for the chosen source address.

[Pascal Hambourg]

Le 31/08/2020 à 18:34, Marc Haber a écrit :
> Pascal Hambourg <pas...@plouf.fr.eu.org> wrote:
>> Le 30/08/2020 à 15:10, Marc Haber a écrit :
>>> Pascal Hambourg <pas...@plouf.fr.eu.org> wrote:
>>>>
>>>> How do you send the ping ? Do you force the source address ?
>>>
>>> with /bin/ping from iputils-ping 20180629-2. No, source address not
>>> forced, the source address should be selected automatically and
>>> according to where the chosen route points to.
>>
>> Then AFAIK the source address is only an output of the routing decision
>> (along with the interface and next hop), not an input key.
>
> This is unhelpful nitpicking,

Nitpicking ? You seemed to have expectations about advanced routing
which do not match how it actually works, so I just tried to explain you
why.

> obviously the routing decision is taken
> before the source address is selected and the mechanisms dont consider
> that the chosen route is the wrong one for the chosen source address.

Whatever. The bottom line is that the routing decision is not iterative.
It won't loop back through the routing rules after selecting the source
address.

[Marc Haber]

Pascal Hambourg <pas...@plouf.fr.eu.org> wrote:
>Le 31/08/2020 à 18:34, Marc Haber a écrit :
>> Pascal Hambourg <pas...@plouf.fr.eu.org> wrote:
>>> Le 30/08/2020 à 15:10, Marc Haber a écrit :
>>>> Pascal Hambourg <pas...@plouf.fr.eu.org> wrote:
>>>>>
>>>>> How do you send the ping ? Do you force the source address ?
>>>>
>>>> with /bin/ping from iputils-ping 20180629-2. No, source address not
>>>> forced, the source address should be selected automatically and
>>>> according to where the chosen route points to.
>>>
>>> Then AFAIK the source address is only an output of the routing decision
>>> (along with the interface and next hop), not an input key.
>>
>> This is unhelpful nitpicking,
>
>Nitpicking ? You seemed to have expectations about advanced routing
>which do not match how it actually works, so I just tried to explain you
>why.

Sorry for misinterpreting your intentions.

>> obviously the routing decision is taken
>> before the source address is selected and the mechanisms dont consider
>> that the chosen route is the wrong one for the chosen source address.
>
>Whatever. The bottom line is that the routing decision is not iterative.
>It won't loop back through the routing rules after selecting the source
>address.

That takes a lot of usefulness out of policy routing.

[Pascal Hambourg]

Le 01/09/2020 à 08:37, Marc Haber a écrit :
> Pascal Hambourg <pas...@plouf.fr.eu.org> wrote:
>> Le 31/08/2020 à 18:34, Marc Haber a écrit :
>
>>> obviously the routing decision is taken
>>> before the source address is selected and the mechanisms dont consider
>>> that the chosen route is the wrong one for the chosen source address.
>>
>> Whatever. The bottom line is that the routing decision is not iterative.
>> It won't loop back through the routing rules after selecting the source
>> address.
>
> That takes a lot of usefulness out of policy routing.

Maybe policy routing is not useful for your use case.
It seems that basic routing based on the destination address is able to
select the expected source address, so why wouldn't it be able to also
select the expected gateway ?

[Grant Taylor]

On 8/9/20 1:35 PM, Marc Haber wrote:
> Hi,

Hi,

> I have a Linux box that is on a network segment with two IPv6-enabled
> routers. Each of the routers has Internet connectivity and its own
> prefix. I want to route everything via router A except when the Linux
> box decides to use a source address that belongs to router B's prefix.

I believe this should be doable. There are some caveats and gotchas to
be aware of. (More below.)

> I want the connectivity via one router to continue working even if the
> othre router has failed, therefore I cannot set one of the routers
> as default gateway and rely on the machine forwarding the traffic
> and/or issueing redirects.

Okay.

That just means that your machine needs to be actively involved in the
routing and not rely on the external routers.

> Am I doing something wrong?

Based on your comment about two different routers, each with their own
prefix, I'm going to assume that two globally routed IPv6 addresses on
the system, one for each router.

As Pascal pointed out, the crux of the problem has to do with the source
IP or lack there of when choosing routes.

1) Unknown / Yet to be determined source (new outgoing connections)
2) Replies from source A / new outgoing connections tied to source A.
3) Replies from source B / new outgoing connections tied to source B.

Numbers 2 and 3 are easy to deal with. The from rule covers this.

Number 1 is the bugbear. Typically there is /a/ default in the main (or
default) routing table. But how do does the system choose the
non-default as the source if there isn't a destination route that matches?

You can easily have a routing table for each router that is fairly
simple. (One of these can even overload the main (default) routing
table.) Rules to select them are easy as long as a source IP is known.

If the source IP is unknown or hasn't been selected yet, you must rely
on some other aspect, typically destination. There are other things
that can be used to choose a routing table. The things that come to
mind are fwmark, ipproto, sport, and dport. I don't know if ipproto,
sport, or dport are viable options. If they are not, you will likely
need to rely on fwmark.

/If/ you need to have specific services / daemons send traffic, then you
might be able to do things based on uid(range).

The other thing you might be able to do is leverage l3mdev if uid(range)
won't work. l3mdev takes things to a new level. I'd need to know more
about the traffic that falls into category #1 before going down that
rabbit hole.



--
Grant. . . .
unix || die