Adding SONAME to a shared object in KLEE

[Marko Dimjašević]

# Please CC me in replies as I'm not on d-devel

Hi all,

I would need help with fixing a SONAME issue with my ITP package. It's a
tool for development, which systematically explores as many as possible
execution paths in a given program, potentially triggering crashes
unknown before.

The package is available at:

http://mentors.debian.net/package/klee

You can fetch it with:

dget -x
http://mentors.debian.net/debian/pool/main/k/klee/klee_1.1.0-1.dsc

(The package targets Jessie and additionally relies on the
python-tabulate package available from Jessie backports. Once I fix the
issue mentioned below, I will move onto fixing other upstream issues
such that I can target Sid.)


I asked for help on d-mentors [1], but the help I got there was not
enough to resolve the issue.

The issue is reported by Lintian:

E: klee: sharedobject-in-library-directory-missing-soname
usr/lib/libkleeRuntest.so
N:
N: A shared object was identified in a library directory (a directory
in
N: the standard linker path) which doesn't have a SONAME. This is
usually
N: an error.
N:
N: SONAMEs are set with something like gcc -Wl,-soname,libfoo.so.0,
where 0
N: is the major version of the library. If your package uses libtool,
then
N: libtool invoked with the right options should be doing this.
N:
N: To view the SONAME of a shared library, run readelf -d on the
shared
N: library and look for the tag of type SONAME.
N:
N: Severity: important, Certainty: possible
N:
N: Check: shared-libs, Type: binary, udeb
N:


The packaged tool doesn't use Libtool. I understand what the error
message is saying, but even with the help I got on d-mentors I wasn't
able to resolve the issue.

Therefore, I'd appreciate if someone could take a look and help with
resolving it.


[1] https://lists.debian.org/debian-mentors/2016/01/msg00542.html


--
Kind regards,
Marko Dimjašević <ma...@cs.utah.edu> . University of Utah
https://dimjasevic.net/marko . PGP key ID: 1503F0AA
Learn email self-defense! https://emailselfdefense.fsf.org

[Filippo Rusconi]

[ longish mail, bear with me ]

Greetings, Marko,

On Thu, Feb 18, 2016 at 10:43:27AM -0700, Marko Dimjašević wrote:
># Please CC me in replies as I'm not on d-devel
>
>Hi all,
>
>I would need help with fixing a SONAME issue with my ITP package. It's a
>tool for development, which systematically explores as many as possible
>execution paths in a given program, potentially triggering crashes
>unknown before.

[...]

I packaged a library (libwildmagic) that involved re-writing partially
the makefile shipped by upstream. In the process I tried to explain
what involves packaging a UNIX/LINUX-based shared object file to the
upstream developer who was only knowledgeable of Windows-based
development. I put that text that I modified to suit your case below.

~~~~~ soname stuff explained ~~~~~

Shared libraries are one of the most interesting/complex fields in
GNU/Linux systems. I'll try to be clear. However, if you want a
document that is pretty neat, then head for this page:

http://www.dwheeler.com/program-library/Program-Library-HOWTO/

by David Wheeler, who is a respected hacker.

But, I'm willing to recap some things here, akin to an "exec summary":

Shared libraries in GNU/Linux are versioned, contrary to what used to
be with Windows-based dlls.

We work on a library that is named kleeRuntest and the file on disk is
thus libkleeRuntest.

SONAME
======

The library file proper should store a specific bit of information
that is called a SONAME (shared object name). That SONAME should be in
the form lib<name>.so.<major-version-number>. In our example, that would be
libkleeRuntest.so.1 (as suggested by then file klee_1.1.0-1.dsc).

REAL-NAME
=========

Note however, that the SONAME is not the "real name" of the library,
that is, the name of the actual file containing the binary code
offered by the library. The real name of the library typically is the
soname *plus* a minor version number, that is actually called the
release number. That REAL-NAME is the name that should be used when
actually *creating* the library file upon building it (which is not
the case presently with your build setup, see below).

Thus, the REAL-NAME would be lib<name>.so.<major-version-number>.<minor-version.number>

libkleeRuntest.so.1.1 (again, klee_1.1.0-1.dsc).

OK, with that out of the way, let's go on with use/dev operations involving
the library:

LINKER-NAME
===========

When building a software that makes use of a library, the linker
should be told the library name to use. That library name should be in
the form of the SONAME *without the version number*. That is, the
linker should only know the name of the library to use for the
linkage.

For your library, that would be:

libkleeRuntest.so

But, then, how to tell from the LINKER-NAME, which actual file should
be used, as we've seen above that the actual file is named with the
REAL-NAME? Well, the bridge between LINKER-NAME and REAL-NAME is a
simple symbolic link.

For your lib, the real binary library file what we should ship is:

/usr/lib/libkleeRuntest.so.1.1


Then, the packager is reponsible for creating the LINKER-NAME symbolic
link:

/usr/lib/libkleeRuntest.so ---> /usr/lib/libkleeRuntest.so.1.1

so that, when the linkder searches for /usr/lib/libkleeRuntest it ends
up actually using /usr/lib/libkleeRuntest.so.1.1.

The linker name is a development/build-time name. This is not the name
that is actually used when dynamically linking the library upon
execution of the program code.

The dynamic loading process needs yet another name, that would be the
SONAME. But, as above, the SONAME is *not* the name of the library as
we ship it, so again we should provide a symbolic link:

/usr/lib/libkleeRuntest.so.1 ---> /usr/lib/libkleeRuntest.so.1.1

The SONAME is the name that the dynamic loader gets from the program
that is being executed and that requires to load library stuff. For
example, my massXpert program has these library NEEDS:

objdump -x /usr/bin/massxpert | grep NEEDED
NEEDED libQtSvg.so.4
NEEDED libQtGui.so.4
NEEDED libQtXml.so.4
NEEDED libQtCore.so.4
NEEDED libstdc++.so.6
NEEDED libm.so.6
NEEDED libgcc_s.so.1
NEEDED libc.so.6

Now, as you can see, the listed names are all SONAMES :
lib<name>.so.<version>.

For example, if I now want to see the files relating to libQtCore.so.4:

ls /usr/lib/x86_64-linux-gnu/libQtCore.so* -l
18 Jan 12 05:15 /usr/lib/x86_64-linux-gnu/libQtCore.so -> libQtCore.so.4.8.6
18 Jan 12 05:15 /usr/lib/x86_64-linux-gnu/libQtCore.so.4 -> libQtCore.so.4.8.6
18 Jan 12 05:15 /usr/lib/x86_64-linux-gnu/libQtCore.so.4.8 -> libQtCore.so.4.8.6
3033168 Jan 12 05:17 /usr/lib/x86_64-linux-gnu/libQtCore.so.4.8.6

You can see that the real library name is indeed libQtCore.so.4.8.6,
of size 3033168, and the other ones are only symbolic links.


OK, NOW THAT WE HAVE THESE FUNDAMENTALS HOW WOULD WE BE DOING WITH
libkleeRuntest library ?

The real name
=============


Generate the library right from the build with the proper name:

libkleeRuntest.so.1.1

How would we do that ? We need to tell the linker how to create the
library file:

compilation flags to build your libkleeRuntest shared object lib:

-shared -fPIC -Wl,-soname,libkleeRuntest.so.1.1 -o libkleeRuntest.so.1.1

These flags need to go in the makefile that drives the build of the library.

This will build the properly-named lib file: libkleeRuntest.so.1.1.

You can check that by running this command:

readelf -d libkleeRuntest.so.1.1

will output something like:
Dynamic section at offset 0xb8c0 contains 26 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000e (SONAME) Library soname: [libkleeRuntest.so.1]
0x000000000000000c (INIT) 0x4e68
0x000000000000000d (FINI) 0x965c

As you can see, the SONAME is correctly mentioned now.

This is all we need to make real-world UNIX-based libs.

Now, we still need the symbolic links, and this is your debian
packager duty. How is this implemented in Debian and Debian-based
systems ?

When packaging libraries, we always start from a source package that
somehow only ships the original source tarball plus methods to build
the binary packages.

A random user will only want binary packages because she would not
even know what to do with a source package.

The software will be shipped as source in the debian source package
named libkleeruntest.

This source package will contain an attached archive file containing a
debian/ directory plus all of its contents.

In the debian/* files, we document how to process the source package
to actually build binary packages.

For each source package that contains a library (or set of libraries,
as is your case) we need to produce two different binary packages:

libkleeruntest1, which is the binary package that a user will have to
install if she ever needs to install a package that makes use of that
library. This package only contains the REAL-NAME file and the
SONAME-to-REAL-NAME symbolic link.

libkleeruntest-dev, which is the binary package that ships all the
files needed for the use of the library from a development
perspective. That is the package where the /usr/include/*.h[pp] files
are shipped. We also ship the second LINKER-NAME-to-REAL-NAME symbolic
link. In most cases, the -dev package is not versioned: it is
typically the last version of the library which in turn depends on the
other binary package of the same version, that of course ships the
real library file.

Hope this helps.

I tried to ./configure the package, but I am missing a library on
this system and I cannot install it. If you cannot succeed with the
explanations above, send the makefile's (there are some, I saw) and
I'll try to see if I can help.

Ciao
Filippo

--
Filippo Rusconi, PhD - public crypto key 7694CF42@ pgp.mit.edu

[Marko Dimjašević]

Hi Filippo,

On Thu, 2016-02-18 at 22:58 +0100, Filippo Rusconi wrote:
> [ longish mail, bear with me ]

Thank you for this explanation! While I knew about symlinks between
shared library files, your explanation made everything very clear!

[...]

> For each source package that contains a library (or set of libraries,
> as is your case) we need to produce two different binary packages:
>
> libkleeruntest1, which is the binary package that a user will have to
> install if she ever needs to install a package that makes use of that
> library. This package only contains the REAL-NAME file and the
> SONAME-to-REAL-NAME symbolic link.
>
> libkleeruntest-dev, which is the binary package that ships all the
> files needed for the use of the library from a development
> perspective. That is the package where the /usr/include/*.h[pp] files
> are shipped. We also ship the second LINKER-NAME-to-REAL-NAME symbolic
> link. In most cases, the -dev package is not versioned: it is
> typically the last version of the library which in turn depends on the
> other binary package of the same version, that of course ships the
> real library file.
>
> Hope this helps.

Oh, even more work for me :) Would it be possible to resolve the soname
issue within one binary package as I have it now first, and then move
onto splitting the package as you outline it above?

> I tried to ./configure the package, but I am missing a library on
> this system and I cannot install it.

Are you referring to STP? It is a dependency I've been packaging as
well. A Jessie binary version is available at:

https://dimjasevic.net/marko/klee/stp_2.1.1+dfsg-1_amd64.deb

It's pretty much the same version of what I've been hoping someone would
review and upload to the New queue (the following one is for Sid):

https://mentors.debian.net/package/stp

> If you cannot succeed with the explanations above, send the makefile's
> (there are some, I saw) and I'll try to see if I can help.

I am not sure if we are thinking of the same, but all Makefile's I use
are available in the package:

https://mentors.debian.net/package/klee

The upstream file I tried to modify is Makefile.rules and in particular
the target starting at line 1115:

https://github.com/klee/klee/blob/master/Makefile.rules#L1115

What also crossed my mind is to hard-code all the names as in the
package there is only one shared library (one .so file) that gets built.


I still don't know how to fix the issue, i.e. I don't see what needs to
be changed in KLEE's build system. Also, are there debhelper tools that
would take care of creating shared library symbolic links?


--
Regards,

[Simon McVittie]

I don't know anything about this specific project, so I don't know
whether you're an upstream or downstream developer. In general SONAMEs
are something that should be managed upstream, to avoid incompatibility
between distributions. If your upstream rejects reasonable patches,
SONAMEs can be managed internally within Debian as a last resort;
but that would make binaries from Debian and not-Debian incompatible,
which is best avoided, and can also cause problems in future if upstream
later starts using SONAMEs correctly.

On Thu, 18 Feb 2016 at 22:58:48 +0100, Filippo Rusconi wrote:
> The library file proper should store a specific bit of information
> that is called a SONAME (shared object name). That SONAME should be in
> the form lib<name>.so.<major-version-number>. In our example, that would be
> libkleeRuntest.so.1 (as suggested by then file klee_1.1.0-1.dsc).

Please note that the version in the SONAME is not the "marketing version"
that you tell your users (or even your version control system), it's the
ABI version. The SONAME must change whenever the ABI changes incompatibly,
and should not change when the ABI remains compatible. If you're using
"semantic versioning", the version in the SONAME *might* match the
human-readable major version, but it also might not; if you're not using
semantic versioning, the version in the SONAME probably won't match the
major version.

For instance, choosing a couple of random libraries from my system:

- libappstream has broken ABI (several times?) before reaching version 1,
so its SONAME is libappstream.so.3 even though its version number is 0.9.1

- conversely, libassuan has never broken ABI, so its SONAME is libassuan.so.0
(meaning "compatible with the first ever version of libassuan.so") even
though its version number is 2.4.2

In some libraries you'll also see an "API version number" in the name
of the library itself. This is to allow for parallel-installation
of the *development files* for more than one version. For instance,
you can't have libjpeg 6b and libjpeg 8 installed at the same time,
because they both ship a libjpeg.so symlink; but you *can* have
Gtk 1, 2 and 3 development files at the same time, because their
libraries are called things like libgtk-3.so.0 (first ABI-version
for Gtk 3), with development symlink names like libgtk-3.so.
The classic document on this is <http://ometer.com/parallel.html>,
with an updated version recently made available at
<https://developer.gnome.org/programming-guidelines/unstable/parallel-installability.html.en>.
Parallel-installation is not at all GNOME-specific - it can also be
seen in Qt/KDE, SDL and DirectFB, for example - but it's very popular
in GNOME and GNOME-related projects, and those documents are the best
reference I've seen for how to do it and why.

S

[Vincent Danjean]

Hi,

Le 18/02/2016 22:58, Filippo Rusconi a écrit :
> [ longish mail, bear with me ]

Thank you for this long explaination. I just think there is a typo:

[...]

> How would we do that ? We need to tell the linker how to create the
> library file:
>
> compilation flags to build your libkleeRuntest shared object lib:
>
> -shared -fPIC -Wl,-soname,libkleeRuntest.so.1.1 -o libkleeRuntest.so.1.1

The argument of the linker option -soname should be the soname and not
the real name:
-shared -fPIC -Wl,-soname,libkleeRuntest.so.1 -o libkleeRuntest.so.1.1

as this is reflected by your output:
[...]
> readelf -d libkleeRuntest.so.1.1
[...]

> 0x000000000000000e (SONAME) Library soname: [libkleeRuntest.so.1]

^^^^
no libkleeRuntest.so.1.1 but the correct libkleeRuntest.so.1 soname

> 0x000000000000000c (INIT) 0x4e68

And a few remarks:
* if the upstream build system want to *link* with this library (after
creating it), it will need to either use the real name or (better) to
create the required link (the libNAME.so link)
* if the upstream build system want to *run* this library (for
example as part of a test suite), it will need to create the other
link (the libNAME.so.SOVERSION link) and to play with
LD_LIBRARY_PATH (or use LD_PRELOAD)
It is were build frameworks such as autotools/libtools helps the
developer...

Regards,
Vincent

--
Vincent Danjean GPG key ID 0xD17897FA vdan...@debian.org
GPG key fingerprint: 621E 3509 654D D77C 43F5 CA4A F6AE F2AF D178 97FA
Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
APT repo: deb http://people.debian.org/~vdanjean/debian unstable main

[Josselin Mouette]

Hi Marko,

Le jeudi 18 février 2016 à 10:43 -0700, Marko Dimjašević a écrit :
> I would need help with fixing a SONAME issue with my ITP package. It's a
> tool for development, which systematically explores as many as possible
> execution paths in a given program, potentially triggering crashes
> unknown before.

> E: klee: sharedobject-in-library-directory-missing-soname usr/lib/libkleeRuntest.so

It looks to me that this file is not meant to be a shared library. I
don’t know what it is, but there are no associated headers and nothing
that looks like it could be linked against it.

It’s not even clear you should ship it in the package, maybe it’s part
of a test suite of some sort?

In any case, it is useless to add a SONAME to such a file, because
SONAMEs are here for programs you actually link with that library. If
the file does have some use in the binary, you can move it away in
another directory (it has no place in /usr/lib).

--
.''`. Josselin Mouette
: :' :
`. `'
`-

[Marko Dimjašević]

Hi Josselin,

On Fri, 2016-02-19 at 12:51 +0100, Josselin Mouette wrote:
> It looks to me that this file is not meant to be a shared library. I
> don’t know what it is, but there are no associated headers and nothing
> that looks like it could be linked against it.
>
> It’s not even clear you should ship it in the package, maybe it’s part
> of a test suite of some sort?
>
> In any case, it is useless to add a SONAME to such a file, because
> SONAMEs are here for programs you actually link with that library. If
> the file does have some use in the binary, you can move it away in
> another directory (it has no place in /usr/lib).

As a matter of fact, it looks like it is meant to be a shared library.

The tool has a tutorial at:

https://klee.github.io/tutorials/testing-function/

And in the last section they instruct the user to execute the following
command:

$ gcc -L path-to-klee-root/Release+Asserts/lib/ get_sign.c -lkleeRuntest


The -l option links against the library in question, hence it is meant
to be used as a shared library.

Which brings us back to the original question - how to add the SONAME
attribute to the library?


--
Regards,

[Vincent Danjean]

Le 24/02/2016 08:30, Marko Dimjašević a écrit :
> Which brings us back to the original question - how to add the SONAME
> attribute to the library?

Did you try what have been said in this thread?

If yes, what did you do *exactly*?
Else, why not?

Regards,
Vincent

[Bernd Zeimetz]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi,

> E: klee: sharedobject-in-library-directory-missing-soname
> usr/lib/libkleeRuntest.so N: N: A shared object was identified
> in a library directory (a directory in N: the standard linker
> path) which doesn't have a SONAME. This is usually N: an error.

easiest fix would be not to use a directory which is in the standard
linker path.
You could use /usr/lib/klee instead....

I did not look into the packaging and build system, but adding
- -Wl,-rpath,/usr/lib/klee and moving the libraries into that directory
might work. There might be better options, though.

Cheers,

Bernd


- --
Bernd Zeimetz Debian GNU/Linux Developer
http://bzed.de http://www.debian.org
GPG Fingerprint: ECA1 E3F2 8E11 2432 D485 DD95 EB36 171A 6FF9 435F
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJW0YJcAAoJEOs2Fxpv+UNfaaIP/iIrJDou82b/ENENzvAdk5IR
7HipAAFWiCWU91SKwnR8LoMOjmeqOjphl2Kkd+5Sll0vLZvNbFREgyTsJ28CRb9K
FHnG3srpTUv4YRPp2eYQ8niNyuSsptnaBLohpoTZKon13DISZ8XDWsN8yWU2dZNi
HI7KRdtGfsi65p1zIbGvgiISbfB5vIYpi/vKDjI/A4kc6PMIryjfsSUQ8+a2kIcp
G/luNMzkpyTtbju1lWNh6skKgnCcPTG0toMFMzMkFznpUeTG0tMQh4JYwGpVlDdd
hEW2C5aBYWwR8W9AyKdvFqQInRepxIhRddV1Wn2eHg9XEa6usiwZvLthwGdTwh+U
XI13rAbvM4oExwYTSxcQU0alWDZPCBTupqDoOzJsZwaajRQLqGR4KDLP0mLGQq09
Rj7hBlHqshZDAY5yShiNl566GF2m9ANiTf5gbdHZCZXavs+Z009fc8IFe6CORTLx
pZ9TyvfUq2WKzTntKb48mvBbYq9F7i5xNnjz/NlkGpStFwYwR8kBmm7Vm8ZAinmX
9zqM9B/1BXW3Bfa4ibrmWtC5fU4uhblyvZKoYejPxql0f/zwiW95z6ymbh+IyP7A
TlIhvPX6lCnUiIpjJgj2QmJwv3ud9nbYnnPsHEC1L2jwvFKjyXXkagLfpEBVfmL1
2hmKOBfN1nb83CwgYmYG
=+axf
-----END PGP SIGNATURE-----