Bug#1013259: samba-libs: Possible policy violation

[Hannes Eberhardt]

Package: samba-libs
Version: 2:4.16.1+dfsg-8~bpo11+1
Severity: normal
X-Debbugs-Cc: hannes.e...@neumannmueller.com

Dear Maintainer,

I noticed that sssd service of a freeipa-client installation broke with the following error message because of missing library.
sssd[651553]: /usr/libexec/sssd/sssd_pac: error while loading shared libraries: libndr.so.1: cannot open shared object file: No such file or directory

I previously reported the issue on the backports mailing list. This is the thread: https://lists.debian.org/debian-backports/2022/06/msg00012.html

I was asked to file a bug report against the samba-libs package as the root cause seems to be here. Please find the corresponding reply here: https://lists.debian.org/debian-backports/2022/06/msg00015.html

-- System Information:
Debian Release: 11.3
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable-security'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 5.10.0-15-amd64 (SMP w/2 CPU threads)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US:en
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages samba-libs depends on:
ii libacl1 2.2.53-10
ii libavahi-client3 0.8-5
ii libavahi-common3 0.8-5
ii libbsd0 0.11.3-1
ii libc6 2.31-13+deb11u3
ii libcap2 1:2.44-1
ii libgnutls30 3.7.1-5
ii libicu67 67.1-7
ii libjansson4 2.13.1-1.1
ii libldap-2.4-2 2.4.59+dfsg-1~bpo11+1
ii libldb2 2:2.5.0+smb4.16.1-8~bpo11+1
ii libpam0g 1.4.0-9+deb11u1
ii libpopt0 1.18-2
ii libsystemd0 247.3-7
ii libtalloc2 2.3.3-4~bpo11+1
ii libtdb1 1.4.6-3~bpo11+1
ii libtevent0 0.11.0-1~bpo11+1
ii libwbclient0 2:4.16.1+dfsg-8~bpo11+1
ii zlib1g 1:1.2.11.dfsg-2+deb11u1

samba-libs recommends no packages.

samba-libs suggests no packages.

-- no debconf information

Thank you,

Hannes

[Andrew Bartlett]

On Mon, 2022-06-20 at 10:39 +0200, Hannes Eberhardt wrote:
> Package: samba-libs
> Version: 2:4.16.1+dfsg-8~bpo11+1
> Severity: normal
> X-Debbugs-Cc: hannes.e...@neumannmueller.com
>
> Dear Maintainer,
>
> I noticed that sssd service of a freeipa-client installation broke
> with the following error message because of missing library.
> sssd[651553]: /usr/libexec/sssd/sssd_pac: error while loading shared
> libraries: libndr.so.1: cannot open shared object file: No such file
> or directory
>
> I previously reported the issue on the backports mailing list. This
> is the thread:
> https://lists.debian.org/debian-backports/2022/06/msg00012.html
>
> I was asked to file a bug report against the samba-libs package as
> the root cause seems to be here. Please find the corresponding reply
> here: https://lists.debian.org/debian-backports/2022/06/msg00015.html

I don't claim to understand debian policy but I think that freeIPA
should, just as sssd should, expect to need to rebuild after any Samba
version upgrade. (And therefore Samba backports may not be possible,
and this may be why Samba is often installed in a custom prefix by 3rd
party packagers).

I changed the libndr.so version upstream in Samba 4.15 to properly
address regressions from earlier fixes in that library. Almost all
users of the library will have been impacted by the changed function
signatures, no fallbacks are/were possible.

Upstream Samba doesn't have the resources to provide a stable ABI to
libndr, but the output of PIDL (eg compile-time API) should still
work.

Andrew Bartlett

--
Andrew Bartlett (he/him) https://samba.org/~abartlet/
Samba Team Member (since 2001) https://samba.org
Samba Team Lead, Catalyst IT https://catalyst.net.nz/services/samba

Samba Development and Support, Catalyst IT - Expert Open Source
Solutions

[Michael Stone]

The issue here is that packages built against samba-libs get a
dependency on samba-libs >= version, and they really either need a
dependency on samba-libs == version or the samba-libs package needs to
be versioned (e.g., samba-libs2, samba-libs3, etc.) and conflict with
other versions, or samba-libs needs to Breaks: every dependent package
on every update, or somesuch. The current state of affairs results in
every change to the samba-libs libraries breaks other packages compiled
against them (namely sssd) until those packages are recompiled, but
there is nothing in the dependencies to enforce that.

[Michael Tokarev]

Michael, are you sure about that?

It is not that simple.

Yes, there were an incompatible change in libndr.so which resulted in changing
its soname, and this is bullseye->bookworm issue indeed. I know about this
issue. But this does not mean sssd needs to be recompiled on every samba
upload (or even 4.16=>4.17 update), sssd should work just fine being compiled
with older samba libs while more recent samba-libs are installed.

Or so it looks like, to me, anyway.

There was another issue here, when modifying samba to build libldb out of
samba sources, there was a bug resulted with older sssd to become non-functional
(this was together with the soname of libndr change too, but unrelated): the
plugin directory has changed. But later on I found a way how to search both
old and new plugins directories, so that issue has been resolved, tho I still
have (iirc) Breaks for older sssd.

But over than this soname change and a bug in directory rename, there should
be no reason why sssd needs to be recompiled every time samba changes, and
why samba-libs dependency should be =version, not >=version.

Samba does have more or less stable public libraries (in /usr/lib/, or actually
/usr/lib/$triple/ but let's use the former for brevity). If another package
uses one of these (like libndr, lisamba-util, libsmbldap, etc), it will have
>= $version dependency. And there are quite some internal libraries,
in /usr/lib/samba/, which, once used, gets =$version dependency, because for
these, there's just no ABI whatsoever, and stuff can change significantly.

I yet to figure out how to solve the soname change issues like we have.
I can add Breaks against existing previous sssd releases for that, I guess,
but it wont be fair, since a recompile of old sssd would fix it but the
Breaks will prevent it from being used.

/mjt

[Michael Stone]

If you can come up with a better solution than a strict dependency, great. But the current situation, in which samba upgrades randomly render systems unusable is, in my opinion, much much much worse than an overly strict dependency. Fundamentally the problem is that you're promising future compatibility but not providing that. One way or another samba-libs either needs to not suggest that linked binaries will work with future versions, or make sure that they do.
--
Michael Stone
(From phone, please excuse typos)

[Michael Tokarev]

01.11.2022 10:59, Michael Tokarev wrote:
..
> And this revealed one more issue here, now with samba 4.17.  Where, the
> same libndr.so again, has changed soname from libndr.so.2 to libndr.so.3!
>
> And it looks like *this* is what you're talking about now, once 4.17 with
> this new libndr.so.3 hits unstable.
>
> *Sigh*.
>
> So now, samba-libs breaks not only bullseye sssd-ad, but also *bookworm*
> sssd-ad!
>
> So:
>  samba-libs 4.13: libndr.so.1
>  samba-libs 4.15: libndr.so.2
>  samba-libs 4.17: libndr.so.3
>
> and this is what we're facing now, 4.16=>4.17 update breaks things again.
> and this new breakage went unnoticed, and I knew nothing about the soname
> change before this very moment.
>
> Andrew, can you share some info about the new 2=>3 soname bumb in 4.17?
>
> I wonder if we should provide old libndr.so.1 and libndr.so.2 interface
> in samba-libs forever... this shouldn't be that difficult.

This has come in 7b9f87b877bd385e8cec893cd282d4b3fc00206d:

Author: Pavel Filipenský <pfil...@redhat.com>
Date: Wed Jun 22 11:13:34 2022 +0200

librpc:ndr: Update ndr_print_debug() and add macro NDR_PRINT_DEBUG_LEVEL

Bumping the ABI to 3.0.0

This is enhancement of NDR_PRINT_DEBUG macro with following new features:

* debug level can be specified (NDR_PRINT_DEBUG always uses level 1)
* the trace header shows the location and function of the caller
instead of function 'ndr_print_debug', which is not really useful.

Signed-off-by: Pavel Filipenský <pfil...@redhat.com>
Reviewed-by: Andreas Schneider <a...@samba.org>

Is it not possible to keep the soname after this change?
I'm reviewing the changes now..

/mjt

[Michael Tokarev]

31.10.2022 19:14, Michael Stone wrote:
> If you can come up with a better solution than a strict dependency, great. But the current situation, in which samba upgrades randomly render systems unusable is, in my opinion, much much much worse than an overly strict dependency. Fundamentally the problem is that you're promising future compatibility but not providing that. One way or another samba-libs either needs to not suggest that linked binaries will work with future versions, or make sure that they do.

Heck.

I did some modifications to the packaging, added the right Breaks: against
an "old" sssd-ad, which should close this very bugreport.

But. And this is a big fat BUT.

I also adjusted list of samba-libs files in samba-libs.install to include
all sonames explicitly.

And this revealed one more issue here, now with samba 4.17. Where, the
same libndr.so again, has changed soname from libndr.so.2 to libndr.so.3!

And it looks like *this* is what you're talking about now, once 4.17 with
this new libndr.so.3 hits unstable.

*Sigh*.

So now, samba-libs breaks not only bullseye sssd-ad, but also *bookworm*
sssd-ad!

So:
samba-libs 4.13: libndr.so.1
samba-libs 4.15: libndr.so.2
samba-libs 4.17: libndr.so.3

and this is what we're facing now, 4.16=>4.17 update breaks things again.
and this new breakage went unnoticed, and I knew nothing about the soname
change before this very moment.

Andrew, can you share some info about the new 2=>3 soname bumb in 4.17?

I wonder if we should provide old libndr.so.1 and libndr.so.2 interface
in samba-libs forever... this shouldn't be that difficult.

/mjt

[Michael Tokarev]

Control: tag -1 + upstream

Original context was at http://bugs.debian.org/1013259 , but whole thing *now*
has is about completely unnecessary soname bump of libndr in 4.17 due to debugging
refinements.

01.11.2022 11:07, Michael Tokarev wrote:
> 01.11.2022 10:59, Michael Tokarev wrote:
> ..

>> And this revealed one more issue here, now with samba 4.17.  Where, the
>> same libndr.so again, has changed soname from libndr.so.2 to libndr.so.3!
>>
>> And it looks like *this* is what you're talking about now, once 4.17 with
>> this new libndr.so.3 hits unstable.
>>
>> *Sigh*.
>>
>> So now, samba-libs breaks not only bullseye sssd-ad, but also *bookworm*
>> sssd-ad!
>>
>> So:
>>   samba-libs 4.13: libndr.so.1
>>   samba-libs 4.15: libndr.so.2
>>   samba-libs 4.17: libndr.so.3
>>
>> and this is what we're facing now, 4.16=>4.17 update breaks things again.
>> and this new breakage went unnoticed, and I knew nothing about the soname
>> change before this very moment.
>>
>> Andrew, can you share some info about the new 2=>3 soname bumb in 4.17?
>>
>> I wonder if we should provide old libndr.so.1 and libndr.so.2 interface
>> in samba-libs forever... this shouldn't be that difficult.
>

> This has come in 7b9f87b877bd385e8cec893cd282d4b3fc00206d:
>
> Author: Pavel Filipenský <pfil...@redhat.com>
> Date:   Wed Jun 22 11:13:34 2022 +0200
>
>     librpc:ndr: Update ndr_print_debug() and add macro NDR_PRINT_DEBUG_LEVEL
>
>     Bumping the ABI to 3.0.0
>
>     This is enhancement of NDR_PRINT_DEBUG macro with following new features:
>
>     * debug level can be specified (NDR_PRINT_DEBUG always uses level 1)
>     * the trace header shows the location and function of the caller
>       instead of function 'ndr_print_debug', which is not really useful.
>
>     Signed-off-by: Pavel Filipenský <pfil...@redhat.com>
>     Reviewed-by: Andreas Schneider <a...@samba.org>
>
> Is it not possible to keep the soname after this change?
> I'm reviewing the changes now..

And it is/was definitely possible and was even trivial to do.
The only real change there is:

-void ndr_print_debug(ndr_print_fn_t fn, const char *name, void *ptr);
+bool ndr_print_debug(int level, ndr_print_fn_t fn, const char *name, void *ptr, const char *location, const char *function);

In order to avoid soname bump, a new function should be created, say,
ndr_print_debug_level(), with a new signature, and old function will
become a trivial wrapper around the new one. This way, only the single
new signature should be added to the ABI file, and that's it.

C'mon... there's really no reason to bump the soname just to refine
debugging output..

I can produce a patch to do just that, so the ABI will become compatible
with previous releases. But what to do with samba-4.17.[012] which were
released with libndr.so.3 already?

I'll sure revert this patch to librpc/ABI/ in Debian (after refining
the ndr_print_debug stuff)...

Anyway, I'll submit both changes, and let's see how it goes.

Thanks,

/mjt

[Ralph Boehme]

On 11/1/22 09:15, Michael Tokarev via samba-technical wrote:
> Control: tag -1 + upstream
>
> Original context was at http://bugs.debian.org/1013259 , but whole
> thing *now* has is about completely unnecessary soname bump of libndr
> in 4.17 due to debugging refinements.

to me this looks like a packaging bug.

[Andreas Schneider]

On Tuesday, 1 November 2022 11:21:55 CET Michael Tokarev wrote:

> It indeed is, I fixed it already.
>
> The question remains though: why the .3 bump happened in the first
> place, due to a change just in a debug helper routine, which was
> trivial to avoid (like I've shown in the patches just posted to
> the list).

libndr is not a stable API it can change any time!

The only stable APIs in Samba are:

libtalloc
libtevent
libtdb
libsmbclient
libwbclient
libnetapi



Andreas


--
Andreas Schneider a...@samba.org
Samba Team www.samba.org
GPG-ID: 8DFF53E18F2ABC8D8F3C92237EE0FC4DCC014E3D

[Andreas Schneider]

On Tuesday, 1 November 2022 11:21:55 CET Michael Tokarev via samba-technical
wrote:

> 01.11.2022 13:14, Ralph Boehme wrote:

> It indeed is, I fixed it already.

See

https://src.fedoraproject.org/rpms/samba/blob/rawhide/f/samba.spec#_148