Bug#1050418: Conntrackd in Bookworm reverts byte order in src address sent by conntrackd in Bullseye

[Pavel Matěja]

Package: conntrackd
Version: 1:1.4.7-1+b2
Conntrackd package on Bullseye is 1:1.4.6-2.

I'm upgrading our servers from Bullseye to Bookworm. Some of them act as load balancers and they are using conntrackd to synchronize TCP connection states
using FTFW sync mode.
I've noticed when I have primary server running Bullseye (conntrack v1.4.6) and secondary Bookworm (conntrack v1.4.7) I get

bullseye:~$ sudo conntrack -L
..
tcp      6 430554 ESTABLISHED src=x.y.49.137 dst=x.y.48.169 sport=35570 dport=636 src=10.170.0.153 dst=x.y.49.137 sport=636 dport=35570 [ASSURED] mark=0 use=1
..

bookworm:~$ sudo conntrack -L
..
tcp      6 431388 ESTABLISHED src=x.y.49.137 dst=x.y.48.169 sport=35570 dport=636 src=153.0.170.10 dst=x.y.49.137 sport=636 dport=35570 [ASSURED] mark=0 use=1
..

Notice order of the 'src' address bytes.
When failover occures all TCP connections via secondary balancer are broken as packets source addresses don't match those in conntrack table anymore.

Downgrade of conntrack and conntrackd packages on Bookworm server solved this problem.
I was unable to create 1.4.7 package for Bullseye.
I'm not sure which version is considered to be acting correctly.

Core of this problem might be related to
https://git.netfilter.org/conntrack-tools/commit/?id=b55717d46ae3b7c3769192a66e565bc7c2d833a1
but I'm not familiar with conntrackd source code.

I'm sorry I had to mask the public ip.

[Jeremy Sowden]

Control: tags -1 confirmed upstream

I believe you are correct in identifying b55717d46ae3 ("conntrackd: fix
endianness bug in IPv4 and IPv6 address"). Before 1.4.7, conntrackd
sent IP addresses in host byte order. However, this prevented one from
running conntrackd instances on hosts of different endianness:

https://marc.info/?l=netfilter&m=161886262729364&w=2

This commit changes conntrackd to use network byte order instead. The
consequence, of course, is that little-endian 1.4.6 instances are not
compatible with little-endian 1.4.7 instances.

I believe the upstream switch to NBO is correct, but I'm afraid that we
in Debian didn't spot this consequence. I'll see about getting a notice
added to the package documentation.

J.

[Jeremy Sowden]

Control: tags -1 + patch

On 2023-08-24, at 20:33:13 +0100, Jeremy Sowden wrote:
> On 2023-08-24, at 12:55:30 +0200, Pavel Matěja wrote:
> > I'm upgrading our servers from Bullseye to Bookworm. Some of them
> > act as load balancers and they are using conntrackd to synchronize
> > TCP connection states using FTFW sync mode. I've noticed when I
> > have primary server running Bullseye (conntrack v1.4.6) and
> > secondary Bookworm (conntrack v1.4.7) I get
> >
> > bullseye:~$ sudo conntrack -L
> > ..
> > tcp 6 430554 ESTABLISHED src=x.y.49.137 dst=x.y.48.169
> > sport=35570 dport=636 src=10.170.0.153 dst=x.y.49.137
> > sport=636 dport=35570 [ASSURED] mark=0 use=1
> > ..
> >
> > bookworm:~$ sudo conntrack -L
> > ..
> > tcp 6 431388 ESTABLISHED src=x.y.49.137 dst=x.y.48.169
> > sport=35570 dport=636 src=153.0.170.10 dst=x.y.49.137
> > sport=636 dport=35570 [ASSURED] mark=0 use=1
> > ..
> >
> > Notice order of the 'src' address bytes.
> > When failover occures all TCP connections via secondary balancer are
> > broken as packets source addresses don't match those in conntrack
> > table anymore.
> >

> > [...]

> >
> > Core of this problem might be related to
> > https://git.netfilter.org/conntrack-tools/commit/?id=b55717d46ae3b7c3769192a66e565bc7c2d833a1
> > but I'm not familiar with conntrackd source code.
>
> I believe you are correct in identifying b55717d46ae3 ("conntrackd:
> fix endianness bug in IPv4 and IPv6 address").
>

> [...]

>
> I believe the upstream switch to NBO is correct, but I'm afraid that
> we in Debian didn't spot this consequence. I'll see about getting a
> notice added to the package documentation.

Something like this patch.

J.