Re: Shared library version numbers [Was: LSB inclusion of OpenSSL]
Andy Polyakov
posed in this message. Others should feel free to correct and complement.
> Right now, we have it depend on the version number. An please tell
> what the correct format for a soname is. On some Unixen, it seems
> like the correct format is libfoo.so.{x}.{y}, where x and y has very
> specific meaning: the program that was linked against libfoo.so.{x}.{y}
> can run against libfoo.so.{x}.{y+n} for all n = 0, 1, ..., oo.
You refer to scheme when versioning is encoded into filename. As far as
I understand elder versions of some BSD flavors [most notably pre-ELF
OpenBSD] and SunOS 4 are/were using this scheme. In present OpenSSL
context the only thing one can do in this case is to either number as
.so.97, .so.98, .so.100 or use dumb numbers such as .so.1, .so.2,
.so.3... Read current OpenSSL .so versioning is effectively broken in
these systems, because run-time linker won't object linking 0.9.7
application with 0.9.8 shared object.
> On
> other Unixen, the program that was linked with a library with a
> specific soname must run against a library with the exact same
> soname.
You refer to most widely used [at least all ELF-based systems] scheme.
Upon compile time linker looks up libfoo.so, pulls an DT_SONAME or
similar field from there and puts it into application binary as is.
DT_SONAME will be looked up by run-time linker and is expected to be
present on library search path, period. DT_SONAME can contain as many
numbers as you like and it even doesn't have to be called
libfoo.so.1.2.3, it can as well be 3.2.1.foo.bar. In present OpenSSL
context it works only as long as DT_SONAME contains full version, i.e.
.so.0.9.7, .so.0.9.8, .so.1.0.0. This it what we currently have. Read
current OpenSSL .so is expected to work on these systems.
Commonly this scheme is complemented with a *convention* to tag with one
[or two] digits and let .so.x[.y] be a symbolic link to .so.x[.y].z,
where z can actually be any number, but conventionally it ascends. To
protect against descending numbers some kind of additional subversioning
is implemented, at least Linux, Tru64 and IRIX has it.
Now question to Johnny Lam [who is complaining that we don't bump
versions] and Christoph Martin [who suggests to add versioning on all
symbols]. What exactly didn't work for you? As far as I understand both
NetBSD and Debian are ELF-based so it should have worked... Does it
simply conflict with your less-than-three-numbers convention [not that
something is wrong with such convention, I'm just asking!] or is there
some other reason? I'm not saying that that we refuse to change .so
versioning in any way, all I want is to do things for right and
recognized reasons, not just upon "we have had some problems." Well, in
PAM case I can imagine problem with GLOBAL_OFFSET_TABLE being
"cross-polluted" by 0.9.7 and 0.9.8 being *both* mapped into same
application. Is it the case? Can you elaborate on which symbols were
overloaded? You can figure this out by examining dynamic name tables *in
pam modules* with objdump -T.
> Others have just one number.
Can anybody refer to an actual example of such system? That would limit
DT_SONAME [or its analogue] to single number after suffix or in any
other way at all? I'm not saying that there is no such implementation, I
just wonder which one is it... On such system the only possible solution
would [again] be to tag as 97, 98, 100 or independent numbers such as 1,
2, 3...
> Others yet place the version
> information somewhere completely different.
MacOS has analog to DT_SONAME and additionally implements internal
"minimally required version" tagging, when shared object is tagged with
"compatiblity version" and "current version" tags, each containing up to
three digit fields [first is 16 bits, second and third - 8 bits]. In
present OpenSSL context it works only as long as shared object contains
full version in its name and additionally is tagged with full version,
e.g. 0.9.7, 0.9.8, etc. One has opportunity to omit the third number
only upon 1.0 release. It should be noted that even though 0.9.8 tags
with full version, 0.9.7 tags with 0.9.0 in "compatibility version,"
which is error-prone. However! OpenSSL encodes "DT_SONAME" with full
version number, so it should work?
As for Tru64. Even though analog to DT_SONAME exists, we don't seem to
use it. On Tru64 OpenSSL .so versioning appears to be broken...
> But I'll take up the cue and see what we can do that works
> everywhere.
Then it would have to be the least common denominator: 97, 98, 100 or
independent numbers such as 1, 2, 3:-) A.
______________________________________________________________________
OpenSSL Project http://www.openssl.org
Development Mailing List opens...@openssl.org
Automated List Manager majo...@openssl.org
Stefa...@t-online.de
> Then when the dynamic linker looks for a symbol, it looks at it
> by name. It will go over all objects to see if it exists in it.
> It will use the symbol from the first library it finds it in.
>
> This means, that a symbol that libA requires, and _should_ get
> from libssl.so.0.9.7, can also exist in libssl.so.0.9.8, and will
> most likely be using that. If libssl.so.0.9.8 has a different
> ABI, this breaks.
>
> The way to make sure that libA gets the symbol with the right
> ABI, is to have all symbols have a unique name. This can be done
> with symbol versioning. It then gets named "symbol@@version".
> The runtime linker adds references to the versioned symbol, and
> dynamic linker then looks for the versioned symbols.
If you "simply" use the "-Bsymbolic" flag when building libA, doesn't
that solve the problem as well? And in a more portable way, since
vrsioned symbols don't exist on "many" platforms?
AFAIK, the idea of the flag is that the library doesn't automatically
doesn't resolve its symbols against the "global" symbols from the
main program or other libs loaded, but only against those libs that
it was linked against.
Regards,
Stefan
Kurt Roeckx
> Hi,
>
> > Then when the dynamic linker looks for a symbol, it looks at it
> > by name. It will go over all objects to see if it exists in it.
> > It will use the symbol from the first library it finds it in.
> >
> > This means, that a symbol that libA requires, and _should_ get
> > from libssl.so.0.9.7, can also exist in libssl.so.0.9.8, and will
> > most likely be using that. If libssl.so.0.9.8 has a different
> > ABI, this breaks.
> >
> > The way to make sure that libA gets the symbol with the right
> > ABI, is to have all symbols have a unique name. This can be done
> > with symbol versioning. It then gets named "symbol@@version".
> > The runtime linker adds references to the versioned symbol, and
> > dynamic linker then looks for the versioned symbols.
>
> If you "simply" use the "-Bsymbolic" flag when building libA, doesn't
> that solve the problem as well? And in a more portable way, since
> vrsioned symbols don't exist on "many" platforms?
> AFAIK, the idea of the flag is that the library doesn't automatically
> doesn't resolve its symbols against the "global" symbols from the
> main program or other libs loaded, but only against those libs that
> it was linked against.
No it doesn't. In fact, -Bsymbolic really should be avoided since
it opens alot more problems that it solves.
Kurt
Stefa...@t-online.de
> > If you "simply" use the "-Bsymbolic" flag when building libA, doesn't
> > that solve the problem as well? And in a more portable way, since
> > vrsioned symbols don't exist on "many" platforms?
> > AFAIK, the idea of the flag is that the library doesn't automatically
> > doesn't resolve its symbols against the "global" symbols from the
> > main program or other libs loaded, but only against those libs that
> > it was linked against.
>
> No it doesn't. In fact, -Bsymbolic really should be avoided since
> it opens alot more problems that it solves.
OK, I'm lost...
What does it actually do?
It sure was the only way to get my PKCS#11 module to actually link against =
the
MD5 routines in the [static] library it used instead of having it reuse the
(incompatible) symbols with the same name contained in the Nescape/Mozilla
binary. And my conclusion from that observation was that unless you do know
enough details of all the binaries that will ever use your shared object in
advance, you want to link your shared object with -Bsymbolic ...
Regards,
Stefan
Kurt Roeckx
> Hi,
>
> > > If you "simply" use the "-Bsymbolic" flag when building libA, doesn't
> > > that solve the problem as well? And in a more portable way, since
> > > vrsioned symbols don't exist on "many" platforms?
> > > AFAIK, the idea of the flag is that the library doesn't automatically
> > > doesn't resolve its symbols against the "global" symbols from the
> > > main program or other libs loaded, but only against those libs that
> > > it was linked against.
> >
> > No it doesn't. In fact, -Bsymbolic really should be avoided since
> > it opens alot more problems that it solves.
>
> OK, I'm lost...
> What does it actually do?
> MD5 routines in the [static] library it used instead of having it reuse the
> (incompatible) symbols with the same name contained in the Nescape/Mozilla
> binary. And my conclusion from that observation was that unless you do know
> enough details of all the binaries that will ever use your shared object in
> advance, you want to link your shared object with -Bsymbolic ...
It changes the order in which the dynamic linker searches for
symbols in objects files. It tries to find the symbol in the
same object that is trying to use it. Because the lookup order
is changed, you can get strange results you didn't expect.
The only thing -Bsymbolic can solve is that it would use symbols
from it's own object.
Kurt
Kurt Roeckx
>
> Now question to Johnny Lam [who is complaining that we don't bump
> versions] and Christoph Martin [who suggests to add versioning on all
> symbols]. What exactly didn't work for you? As far as I understand both
> NetBSD and Debian are ELF-based so it should have worked... Does it
> simply conflict with your less-than-three-numbers convention [not that
> something is wrong with such convention, I'm just asking!] or is there
> some other reason? I'm not saying that that we refuse to change .so
> versioning in any way, all I want is to do things for right and
> recognized reasons, not just upon "we have had some problems." Well, in
> PAM case I can imagine problem with GLOBAL_OFFSET_TABLE being
> "cross-polluted" by 0.9.7 and 0.9.8 being *both* mapped into same
> application. Is it the case? Can you elaborate on which symbols were
> overloaded? You can figure this out by examining dynamic name tables *in
> pam modules* with objdump -T.
If you want an example of things breaking because of the
transition from 0.9.7 to 0.9.8, look at:
http://bugs.debian.org/334180
In this case, libpq (from postgresql) was linked to libssl0.9.7,
and dovecot was linked to libpq and libssl0.9.8.
It gave some strange error message indicating it couldn't load
libz.so.
I really don't want to debug this, it's alot more easier to just
relink libpq against libssl0.9.8.
Andy Polyakov
>> everywhere.
>
> Then it would have to be the least common denominator: 97, 98, 100 or
The above was referring to file suffixes. It should be noted that
there're platforms, which has no notion of versioning through file
suffix, most notably Windows. The only way to get it working on Windows
and similar platforms [if any] would be to encode the version into .DLL
file name, e.g. crypto098.dll, ...099.dll, ...100.dll [note that at
least in Windows context it doesn't prevent us from calling
corresponding .lib import library for crypto.lib]. In other words least
common denominator would be embedding version number into shared object
file name, not its extension. As it's unlikely to be accepted that Unix
developers would have link with -lcrypto100, we're stuck with multiple
versioning schemes in either case:-(
On a side note in respect to problem with cross-polution of global name
table from different versions of OpenSSL. Windows would be free from
this, because import tables are private to every mapped module, meaning
that two modules, one linked with 0.9.7 and another with 0.9.8, would
not interfere with each other. Just a side note! A.
Andy Polyakov
>>"cross-polluted" by 0.9.7 and 0.9.8 being *both* mapped into same
>>application. Is it the case? Can you elaborate on which symbols were
>>overloaded? You can figure this out by examining dynamic name tables *in
>>pam modules* with objdump -T.
>
> If you want an example of things breaking because of the
> transition from 0.9.7 to 0.9.8, look at:
> http://bugs.debian.org/334180
>
> In this case, libpq (from postgresql) was linked to libssl0.9.7,
> and dovecot was linked to libpq and libssl0.9.8.
In other words it's exactly the case of "cross-pollution" of name space
by indirect link with different versions of OpenSSL .so. Thanks to
-Bsymbolic at least libcrypto.so.0.9.7 doesn't affect
libcrypro.so.0.9.8, but libcrypto.so.0.9.7 can affect libssl.so.0.9.8
[and/or vice versa].
For the record. I reckon that -Bsymbolic is more than appropriate in
OpenSSL context as it prevents overloading of inner calls by another
module. A.