SPAM [ace-users] [autotools] Compiling ACE with autotools gives different soname from "classical" compilation
thomas....@free.fr
For some reason my mails don't get to ace-users@
Can somebody please have a look at this problem ?]
Hello all,
here's the PRF:
ACE VERSION: 5.5.1
HOST MACHINE and OPERATING SYSTEM: Debian GNU/Linux testing/unstable
COMPILER NAME AND VERSION (AND PATCHLEVEL):
g++ (GCC) 4.0.4 20060422 (prerelease) (Debian 4.0.3-2)
THE $ACE_ROOT/ace/config.h FILE:
generated by autotools. I will provide it on demand.
THE $ACE_ROOT/include/makeinclude/platform_macros.GNU FILE:
N/A
CONTENTS OF $ACE_ROOT/.../config/default.features:
Unchanged
AREA/CLASS/EXAMPLE AFFECTED:
Libraries sonames are different when compiled using "classical"
method (i.e. config-XXX.h and platform_macros.GNU) and when
compiling using autotools methods. The "classical" method gives
libACE.so.5.5.1 while the autotools one gives libACE.so.5
DOES THE PROBLEM AFFECT:
COMPILATION? Somehow
LINKING? Somehow again
EXECUTION? Yes, it will
OTHER (please specify)?
I want to switch Debian packages compilation method from
"classical" to autotools based. But the new method changes
the soname to something that will break binaries on upgrade.
SYNOPSIS:
Compiling ACE with libtool or "classical" method gives different
sonames.
DESCRIPTION:
The current library naming scheme confuses two things:
* the library version
* the ABI[0] version, also called interface version. This ABI is
"captured" into a ELF property of the library: the *soname*.
More on this can be found in [1]. Basically, if two different
libraries have the same soname, then any binary linked against
one library should run with the other.
So when you compile ACE using the "classical" method, both
versions look the same, i.e. you get a libACE.so.5.5.1 file with a
libACE.so.5.5.1 soname.
But when you use the autotools method, libtool does not let you do
this confusion. It enforces inteface version compatibility
using it's own versioning system[3], and this system relies on
the numbers after the '.so' in the soname. So you get
libACE.so.5.5.1 file with a libACE.so.5 soname.
Bam ! The next ACE release, ACE 5.5.2 (or will it be 5.6.0 ?)
breaks the ABI - it's so easy to do it in C++ - and the existing
binaries stop working while the soname claim they should work. So
it's better to have a soname that "somehow" match the release
number.
You could possibly trick libtool to get the soname you want, but I
don't think that's a good idea. After all, what libtool tries to
achieve is a clean separation of implementation and interface
versions.
I believe the libtool proposal in such situations[2] should be
considered. I don't think having libACE-5.5.1.so instead of
libACE.so.5.5.1 for the library name *and* the soname would be a
big deal. Anyway, that would require changes to both compilation
methods. I would gladly provide patches.
REPEAT BY:
Compile ACE using the "classical" method. Look the soname:
$ objdump -p classical/libACE.so.5.5.1 | grep SONAME
SONAME libACE.so.5.5.1
Compile ACE using the "autotools" method. Look the soname:
$ objdump -p autootols/libACE.so.5.5.1 | grep SONAME
SONAME libACE.so.5
They don't match.
SAMPLE FIX/WORKAROUND:
Changing library and soname from libACE.so.X.Y.Z to
libACE-X.Y.Z.so should do.
Any comments ?
Thanks,
Thomas
[0] http://en.wikipedia.org/wiki/Application_Binary_Interface
[1]
http://www.usenix.org/publications/library/proceedings/als00/2000papers/papers/full_papers/browndavid/browndavid.pdf
[2] http://www.gnu.org/software/libtool/manual.html#Release-numbers
[3] http://www.gnu.org/software/libtool/manual.html#Libtool-versioning
Douglas C. Schmidt
Thanks for the PRF! J.T. and Ossama, can you please take a
look at this when you have a chance?
Thanks,
Doug
--
Dr. Douglas C. Schmidt Professor and Associate Chair
Electrical Engineering and Computer Science TEL: (615) 343-8197
Institute for Software Integrated Systems WEB: www.dre.vanderbilt.edu/~schmidt
Vanderbilt University, Nashville TN, 37203 NET: d.sc...@vanderbilt.edu
J.T. Conklin
"Douglas C. Schmidt" <sch...@cse.wustl.edu> writes:
> for the PRF! J.T. and Ossama, can you please take a
> look at this when you have a chance?
Sure, I've been thinking about this since Thomas sent it to me a few
days ago. (Btw, He's been having problems with his messages getting
to the lists even though he's subscribed, is there anyone we can point
him at to get this diagnosed/fixed?).
The big issue is ABI stability. While other projects make a large
effort to keep upwards binary compatibility between major versions,
this is not something we've worried about. ABI stability is a big
project, I don't see it happening without a big investment. Even if
we could find someone willing sponsor such an effort, I have a feeling
it may conflict with the other R&D goals of ACE/TAO to such an extent
that it is fundementally incompatible.
So if we concede that each release/beta is separate ABI, what's the
best way to manage that?
One way is to encode the ABI version in the library filenames, like
libACE-5.5.1.{a,so,...}. I'm not particularly fond of this, because
it amkes it a bit more difficult to integrate. Makefiles, Jamfiles,
SConscripts, etc. must be changed to include library version numbers.
It's even worse for thingds like configure scripts which would have
to guess about what version(s) may be installed. This is one thing
I really hate about the Boost library filename conventions.
Another way to do this is decouple the library version numbers from
ABI versions. Each time we we make release/beta with an incompatible
ABI, we increment the ABI version number. This might be a bit
confusing, at least at first, and the ABI number would increase fairly
quickly.
For my own development, I use a different install prefix which encodes
the release/snapshot number. This is because I need complete
development environments for different releases. This may be a
special case. Now that it's easier to package ACE/TAO using an
OS's package/port system, we see more and more packages that use
those packages. For that use case, separate libraries for different
versions installed in the same location is important.
So I guess that even after all that thought, I don't have a
recommendation. Hopefully Ossama has better luck.
--jtc
--
J.T. Conklin
Douglas C. Schmidt
>> Sure, I've been thinking about this since Thomas sent it to me a few
>> days ago. (Btw, He's been having problems with his messages getting
>> to the lists even though he's subscribed, is there anyone we can point
>> him at to get this diagnosed/fixed?).
His posting came through fine, which is how we knew to reply ;-).
Thomas, please see
http://www.cs.wustl.edu/~schmidt/ACE-mail.html
for instructions on how to subscribe and post.
>> The big issue is ABI stability. While other projects make a large
>> effort to keep upwards binary compatibility between major versions,
>> this is not something we've worried about. ABI stability is a big
>> project, I don't see it happening without a big investment. Even if
>> we could find someone willing sponsor such an effort, I have a feeling
>> it may conflict with the other R&D goals of ACE/TAO to such an extent
>> that it is fundementally incompatible.
I agree.
>> So if we concede that each release/beta is separate ABI, what's the
>> best way to manage that?
>>
>> One way is to encode the ABI version in the library filenames, like
>> libACE-5.5.1.{a,so,...}. I'm not particularly fond of this, because
>> it amkes it a bit more difficult to integrate. Makefiles, Jamfiles,
>> SConscripts, etc. must be changed to include library version numbers.
>> It's even worse for thingds like configure scripts which would have
>> to guess about what version(s) may be installed. This is one thing
>> I really hate about the Boost library filename conventions.
>>
>> Another way to do this is decouple the library version numbers from
>> ABI versions. Each time we we make release/beta with an incompatible
>> ABI, we increment the ABI version number. This might be a bit
>> confusing, at least at first, and the ABI number would increase fairly
>> quickly.
It makes sense to me to do for the autoconf stuff what ever we
currently do with the "classic" configuration. I think the problem
here is the inconsistency between the two approaches, not the approach
itself, right?
Thanks,
Doug
thomas....@free.fr
> Hi J.T.,
>
> >> Sure, I've been thinking about this since Thomas sent it to me a few
> >> days ago. (Btw, He's been having problems with his messages getting
> >> to the lists even though he's subscribed, is there anyone we can point
> >> him at to get this diagnosed/fixed?).
>
> His posting came through fine, which is how we knew to reply ;-).
> Thomas, please see
>
> http://www.cs.wustl.edu/~schmidt/ACE-mail.html
>
> for instructions on how to subscribe and post.
Well, this mail came through because I've used my ISP webmail instead
of using directly my ISP SMTP server. When I try to do so my mails get
lost. Possibly my ISP SMTP server (smtp dot wanadoo dot fr) is black
listed ?
> >> The big issue is ABI stability. While other projects make a large
> >> effort to keep upwards binary compatibility between major versions,
> >> this is not something we've worried about. ABI stability is a big
> >> project, I don't see it happening without a big investment. Even if
> >> we could find someone willing sponsor such an effort, I have a feeling
> >> it may conflict with the other R&D goals of ACE/TAO to such an extent
> >> that it is fundementally incompatible.
>
> I agree.
>
> >> So if we concede that each release/beta is separate ABI, what's the
> >> best way to manage that?
> >>
> >> One way is to encode the ABI version in the library filenames, like
> >> libACE-5.5.1.{a,so,...}. I'm not particularly fond of this, because
> >> it amkes it a bit more difficult to integrate. Makefiles, Jamfiles,
> >> SConscripts, etc. must be changed to include library version numbers.
> >> It's even worse for thingds like configure scripts which would have
> >> to guess about what version(s) may be installed. This is one thing
> >> I really hate about the Boost library filename conventions.
I think providing a libACE.so symlink pointing to libACE-X.Y.Z.so
would *not* require any change for integration. From what I've
understood this is what libtool -release flag does.
> >> Another way to do this is decouple the library version numbers from
> >> ABI versions. Each time we we make release/beta with an incompatible
> >> ABI, we increment the ABI version number. This might be a bit
> >> confusing, at least at first, and the ABI number would increase fairly
> >> quickly.
>
> It makes sense to me to do for the autoconf stuff what ever we
> currently do with the "classic" configuration. I think the problem
> here is the inconsistency between the two approaches, not the approach
> itself, right?
Right.
Thanks,
Thomas
Steve Huston
FYI, I'm testing some changes to change the automake/libtool build
from using -version-info to using -release. This will change the
version numbers on the libtool-generated libraries to include a
version ID that looks like the ACE+TAO release numbering
(major.minor.beta).
-Steve
--
Steve Huston, Riverace Corporation
Helping you succeed with ACE
See http://www.riverace.com/support.htm
Thomas Girard
> Hi folks,
>
> FYI, I'm testing some changes to change the automake/libtool build
> from using -version-info to using -release. This will change the
> version numbers on the libtool-generated libraries to include a
> version ID that looks like the ACE+TAO release numbering
> (major.minor.beta).
That's great news, thanks !
J.T. Conklin
> FYI, I'm testing some changes to change the automake/libtool build
> from using -version-info to using -release. This will change the
> version numbers on the libtool-generated libraries to include a
> version ID that looks like the ACE+TAO release numbering
> (major.minor.beta).
One thing that I'm worried about is dynamically loaded libraries when
multiple releases are installed under the same prefix. TAO now uses
this extensively.
I'm not sure this was supported under the old scheme, but let's be
sure that -release doesn't make it even more difficult.
--jtc
--
J.T. Conklin
Steve Huston
I'm not very experienced with TAO, but the way the MPC changes are
structured, you can still tell it to generate Makefile.am with
-version-info. So we could make ACE do -release and TAO do
-version-info if needed.
As it is, changing all the Makefile.am files in TAO to use -release is
either a hand-edit, or an MPC regenerate and a hand edit, neither of
which I'm going to try (since I can't test it) so I can leave
taodefaults.mpb alone if you like.
ACE, on the other hand, has interface changes often and no two
versions can safely be installed in the same place without some
identifier like -release.
Thomas Girard
> > One thing that I'm worried about is dynamically loaded libraries
> > when multiple releases are installed under the same prefix. TAO
> > now uses this extensively.
> >
> > I'm not sure this was supported under the old scheme, but let's be
> > sure that -release doesn't make it even more difficult.
>
> I'm not very experienced with TAO, but the way the MPC changes are
> structured, you can still tell it to generate Makefile.am with
> -version-info. So we could make ACE do -release and TAO do
> -version-info if needed.
>
> As it is, changing all the Makefile.am files in TAO to use -release is
> either a hand-edit, or an MPC regenerate and a hand edit, neither of
> which I'm going to try (since I can't test it) so I can leave
> taodefaults.mpb alone if you like.
I'll try to have a look at this possible issue this week. I'll share
the results of my investigation on this thread.
Thanks,
Thomas
Thomas Girard
ldkcld
>
> Thanks,
>
> Thomas
>
> _______________________________________________
> ace-users mailing list
> ace-...@mail.cse.wustl.edu
> http://www.cse.wustl.edu/mailman/listinfo/ace-users
Thomas Girard
Hello,
On Sun, May 14, 2006 at 08:11:22PM +0200, replying to J.T.:
> > > One thing that I'm worried about is dynamically loaded libraries
> > > when multiple releases are installed under the same prefix. TAO
> > > now uses this extensively.
> > >
> > > I'm not sure this was supported under the old scheme, but let's be
> > > sure that -release doesn't make it even more difficult.
I wrote:
> I'll try to have a look at this possible issue this week. I'll share
> the results of my investigation on this thread.
Ouch! That was more than thre months ago.
Yesterday I realized what J.T. was right, and that this is already
happening as of TAO 1.4.7 -- I need to check with 1.5.x.
Indeed, some libraries (e.g. TAO_Codeset, TAO_PI) get dlopen'ed and are
loaded if available. For instance, the library that get searched for
TAO_Codeset is libTAO_Codeset.so, *not* libTAO_Codeset.so.1.4.7.
Basically this means that distributing TAO libraries that match their
SONAME is *not* enough; the symlinks must be distributed as well for
dlopened libraries.
Therefore:
1. it is not possible to install TAO libraries from version A and
version B in the same directory.
That might seems like a strange idea, but imagine for example that
the NameService of TAO version B has bugs, so I want to use the
NameService of TAO version A. But the Telecom Log Service from
TAO version B offers way more features than the one from version A,
so I want the version B this time.
Because of the symlinks problem explained above, I couldn't have
those two binaries, NamingService and TelecomLogService rely on
libraries available from the same location (e.g. /usr/lib).
2. the Debian package libtao-orbsvcs, which contains all binaries and
libs (and will soon be split to have separate packages for binaries)
is useless if the libtao-orbscvs-dev package is not installed.
Indeed, the latter package installs symlinks.
I had a quick look and it seems the loading mechanism is in
Lib_Find.cpp. I believe this mechanism could be extented to pass extra
prefixes and suffixes. That way ldfind() could load libraries whose
name match the SONAME, may that be libTAO_Codeset.so.1.4.7 or
libTAO_Codeset-1.4.7.so.
I know from [1] that the native library path searching has changed, but
I don't know whether that solves my problem.
If it does not, do you think the solution depicted above is worth it ?
I will happily have a try at it if you agree.
Thanks,
Thomas
Ken Sedgwick
> Basically this means that distributing TAO libraries that match their
> SONAME is *not* enough; the symlinks must be distributed as well for
> dlopened libraries.
The linux RPM packaging tries to include the .so symlinks for all
libraries which are likely dynamically loaded in the runtime (non
"devel") packages.
I ended up using the following heuristic to decide which to include in
the binary packages:
1) Make a list of all the shared objects
2) run "nm" on all of them and find the ones with any symbol which has
the substring "_make_"
3) Remove two false positives (libACE.so and libTAO.so)
4) ship the symlinks for these shared objects in the runtime packages.
It's not pretty, but I believe that it generally works ...
Ken
--
Ken Sedgwick
Bonsai Software, Inc.
http://www.bonsai.com/ken/
(510) 610-4162
ken...@bonsai.com
Public Key: http://www.bonsai.com/ken/ken.asc
GPG Fingerprint: 851E 3B07 E586 0843 9434 5CC7 4033 3B9B 3F3F 9640
Thomas Girard
On Sat, Aug 05, 2006 at 11:48:41PM -0700, Ken Sedgwick wrote:
> The linux RPM packaging tries to include the .so symlinks for all
> libraries which are likely dynamically loaded in the runtime (non
> "devel") packages.
>
> I ended up using the following heuristic to decide which to include in
> the binary packages:
>
> 1) Make a list of all the shared objects
> 2) run "nm" on all of them and find the ones with any symbol which has
> the substring "_make_"
> 3) Remove two false positives (libACE.so and libTAO.so)
> 4) ship the symlinks for these shared objects in the runtime packages.
>
> It's not pretty, but I believe that it generally works ...
Thank you for the hint.
Thomas