-rpath with libtool and Debian Linux
Ben Gertzfield
debian...@lists.debian.org because my knowledge of how -rpath works
is limited.
To recap, for the Debian folks:
libtool, a tool for creating libraries and linking programs with those
libraries on multiple platforms, forces all programs it links to be
linked with the -rpath option, which hard-codes the path to the
library being linked with into the binary.
This is bad for Debian, because in all binary packaging systems,
shared libraries can and will be moved around from time to time, as
policy and major upgrades (like libc5 -> libc6) mandate.
However, Alexandre Oliva <ol...@dcc.unicamp.br> brings up an important
point: -rpath is necessary if one is installing libraries and binaries
linked to those libraries in one's home directory, or if your Unix has
no support for library search paths via environment variables like
Linux's LD_LIBRARY_PATH.
Basically, I have been asking Alexandre if it's possible to add a
--no-rpath option to libtool when calling it to tell it to not use
-rpath when linking binaries, but he refused, saying he'd have to port
that to 'hundreds of platforms'.
Can someone with more knowledge of -rpath and libtool than I explain
why Debian policy mandates avoiding -rpath?
Thanks,
Ben
--
Brought to you by the letters C and O and the number 14.
"Porcoga daisuki!"
Debian GNU/Linux maintainer of Gimp and GTK+ -- http://www.debian.org/
I'm on FurryMUCK as Che, and EFNet/Open Projects IRC as Che_Fox.
------- Start of forwarded message -------
To: Ben Gertzfield <c...@debian.org>
Cc: Manish Singh <yo...@gimp.org>, bug-l...@gnu.org,
James Troup <ja...@nocrew.org>
Subject: Re: [gimp-devel] gimp1.1 rpath hell
References: <874spgx...@nocrew.org> <ytt679v...@gilgamesh.cse.ucsc.edu> <1999012617...@yosh.gimp.org> <or90epq...@araguaia.dcc.unicamp.br> <ytt4spd...@gilgamesh.cse.ucsc.edu> <ord841p...@araguaia.dcc.unicamp.br> <yttvhht...@gilgamesh.cse.ucsc.edu>
From: Alexandre Oliva <ol...@dcc.unicamp.br>
Date: 27 Jan 1999 01:16:59 -0200
Message-ID: <or4spdp...@araguaia.dcc.unicamp.br>
Mime-Version: 1.0
On Jan 27, 1999, Ben Gertzfield <c...@debian.org> wrote:
Alexandre> Or just fix ld.so so that, if a program or library
Alexandre> depends on libc.so.5, it shouldn't even try to use
Alexandre> libc.so.6, and vice-versa.
> If we move the libraries, any program that is compiled with -rpath
> WILL NO LONGER WORK. Period.
You shouldn't move shared libraries. Period :-)
If the particular version of libX11.so was linked with depends on
libc.so.5, ld.so should use it. I don't see any need for a separate
directory for libraries, if library versioning would work correctly.
> This has happened before with Debian, as emacs was compiled with
> -rpath /usr/X11R6/lib to force the X libraries to be stored there.
If libraries are not found in the -rpath, they should be searched in
the default search directories. Aren't they?
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
------- End of forwarded message -------
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Alexandre Oliva
> This is bad for Debian, because in all binary packaging systems,
> shared libraries can and will be moved around from time to time, as
> policy and major upgrades (like libc5 -> libc6) mandate.
You might have included my suggestion to prevent having to move
libraries in the first place: creating a libc6-specific directory
right now, instead of installing libraries in /usr/lib and having to
move them into another directory when libc7 should be released.
> However, Alexandre Oliva <ol...@dcc.unicamp.br> brings up an important
> point: -rpath is necessary if one is installing libraries and binaries
> linked to those libraries in one's home directory, or if your Unix has
> no support for library search paths via environment variables like
> Linux's LD_LIBRARY_PATH.
More than that (and it was my fault to have failed to mention that
before): libtool will hard-code the installation directory of the
library into the `libdir' variable of the .la script it installs.
Therefore, if one moves the library then tries to link with the .la
file, he loses. There's also the dlopening issue: libltdl (to be
released with libtool 1.3) will dlopen a library in the directory
pointed to by `libdir' too.
In general, I feel that moving libraries around is a very bad idea,
because it will lead to failure most of the times, and that's why I
don't feel libtool should help people doing that.
> Basically, I have been asking Alexandre if it's possible to add a
> --no-rpath option to libtool when calling it to tell it to not use
> -rpath when linking binaries, but he refused, saying he'd have to port
> that to 'hundreds of platforms'.
Actually, not issuing -rpath or equivalent is quite easy to do, but
choosing *when* not to do it is the part that is hard to port. Thomas
Tanner has suggested that we could omit the -rpath switch for
libraries that are supposed to be installed in directories listed in
the default search path. However, the default search PATH might
change, and, even if it did not, it would be possible to link an
application with a library in, say, /usr/local/lib, and then find out
that at run-time it loads a library with the same name in /usr/lib,
because /usr/lib appears first in the standard search path.
The other issue is *how* to specify that -rpath should be ommitted.
Should it be a configure option, that would permanently disable any
-rpath switches? Or should it be an additional argument to the
libtool script at program-linking time? Or should it be specified at
library linking time, with one of three options: hardcode path in the
library, hardcode path in any program linked with it, or do not
hardcode path at all? Or something else? What to do on a platform
that doesn't support the requested hardcoding strategy?
The issue is very complex because we can't think just of GNU/Linux
with all its bells and whistles, because libtool is supposed to
present an homogeneous, portable interface to creating libraries.
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
J.H.M. Dassen
> You might have included my suggestion to prevent having to move libraries
> in the first place: creating a libc6-specific directory right now, instead
> of installing libraries in /usr/lib and having to move them into another
> directory when libc7 should be released.
I'm sorry, but this is IMHO completely backwards. On Linux systems, there is
nothing wrong with moving libraries around as the need arises. Having
libtool default to -rpath is what's causing problems.
I've seen one too many instances of "<foo> crashes on Debian" that turned
out to be "<foo> is a libc5 binary with an RPATH of /usr/X11R6/lib" which on
any reasonably up to date Debian system contains libc6 X libraries.
The X example also shows that the problem isn't limited to /usr/lib either.
What's next? /usr/local/lib/libc6 ?
> > However, Alexandre Oliva <ol...@dcc.unicamp.br> brings up an important
> > point: -rpath is necessary if one is installing libraries and binaries
> > linked to those libraries in one's home directory,
That is a special case. The default should be sane for regular cases.
> > or if your Unix has no support for library search paths via environment
> > variables like Linux's LD_LIBRARY_PATH.
While I appreciate concerns about supporting less fortunate operating
environments, I don't think their existence should hold us back from doing
the right thing on Linux.
> In general, I feel that moving libraries around is a very bad idea,
> because it will lead to failure most of the times, and that's why I don't
> feel libtool should help people doing that.
I see no reason why moving libraries around is a bad idea. I see defaulting
to -rpath as a bad idea, which breaks moving libraries around.
Ray
--
POPULATION EXPLOSION Unique in human experience, an event which happened
yesterday but which everyone swears won't happen until tomorrow.
- The Hipcrime Vocab by Chad C. Mulligan
Ben Gertzfield
Alexandre> You might have included my suggestion to prevent having
Alexandre> to move libraries in the first place: creating a
Alexandre> libc6-specific directory right now, instead of
Alexandre> installing libraries in /usr/lib and having to move
Alexandre> them into another directory when libc7 should be
Alexandre> released.
This is rather frustrating, because then we will need to make a
/lib/libc6, /usr/lib/libc6, /usr/X11R6/libc6, /usr/local/lib/libc6..
it never ends. :)
Alexandre> More than that (and it was my fault to have failed to
Alexandre> mention that before): libtool will hard-code the
Alexandre> installation directory of the library into the `libdir'
Alexandre> variable of the .la script it installs. Therefore, if
Alexandre> one moves the library then tries to link with the .la
Alexandre> file, he loses. There's also the dlopening issue:
Alexandre> libltdl (to be released with libtool 1.3) will dlopen a
Alexandre> library in the directory pointed to by `libdir' too.
I've never understood what the .la scripts are for. Why are they
installed into /usr/lib/, where libraries live? This is kind of
off-the-subject, but they have always confused me, and I delete
them out of any libtool-using library package I maintain.
Alexandre> In general, I feel that moving libraries around is a
Alexandre> very bad idea, because it will lead to failure most of
Alexandre> the times, and that's why I don't feel libtool should
Alexandre> help people doing that.
With Debian and Red Hat, it's totally the opposite. Moving libraries
around is what leads to upgrades being possible.
Alexandre> The issue is very complex because we can't think just
Alexandre> of GNU/Linux with all its bells and whistles, because
Alexandre> libtool is supposed to present an homogeneous, portable
Alexandre> interface to creating libraries.
Totally agreed. You are worrying just a bit too much about this,
though -- we don't need to worry about a switch that has to decide
WHEN to disable -rpath, just a switch that understands, "Okay, the
builder knows what he's talking about, no -rpath is fine with me".
Ben
--
Brought to you by the letters V and D and the number 3.
"Porcoga daisuki!"
Debian GNU/Linux maintainer of Gimp and GTK+ -- http://www.debian.org/
I'm on FurryMUCK as Che, and EFNet/Open Projects IRC as Che_Fox.
Alexandre Oliva
> On Wed, Jan 27, 1999 at 17:07:30 -0200, Alexandre Oliva wrote:
>> You might have included my suggestion to prevent having to move libraries
>> in the first place: creating a libc6-specific directory right now, instead
>> of installing libraries in /usr/lib and having to move them into another
>> directory when libc7 should be released.
> I'm sorry, but this is IMHO completely backwards. On Linux systems, there is
> nothing wrong with moving libraries around as the need arises.
Except that you risk replacing a library with an incompatible one.
That's what has caused you so much trouble. If, instead of moving the
X11 libraries to another dir and creating new, incompatible versions
under the same pathname, you had created new versions in other
directories, no unexpected crashes would have occurred.
> Having libtool default to -rpath is what's causing problems.
This is IMHO completely backwards :-)
When a program is linked with a shared library, a contract is
established between them stating that the library (or any newer but
compatible version thereof, on systems that support library
versioning) will supply the symbols that the program resolved from it,
and the program will be able to find the library at that point. If
you move the library and replace it with an incompatible one, you're
breaking the contract and the versioning mechanism, so you can't blame
the program for crashing, nor the tool that created the program.
> I've seen one too many instances of "<foo> crashes on Debian" that turned
> out to be "<foo> is a libc5 binary with an RPATH of /usr/X11R6/lib" which on
> any reasonably up to date Debian system contains libc6 X libraries.
See? You replaced one library with an incompatible one without
modifying its version number to mark it as incompatible. Isn't this
breaking the contract? How could you expect it to work?
> The X example also shows that the problem isn't limited to /usr/lib either.
> What's next? /usr/local/lib/libc6 ?
Maybe some library versioning mechanism that allows incompatible
changes to be marked as such. Maybe an environment variable or some
file in /etc containing a number that will be added to the major
version number of any library libtool creates, so that they will be
marked as incompatible.
>> > However, Alexandre Oliva <ol...@dcc.unicamp.br> brings up an important
>> > point: -rpath is necessary if one is installing libraries and binaries
>> > linked to those libraries in one's home directory,
> That is a special case. The default should be sane for regular cases.
You see the regular case as the one you use the most. I see it as the
one I use the most. I don't install any packages in /usr or
/usr/local because I find it *horrible* to have a huge /usr/local/bin
without any clear separation between packages. It's a pity that the
GNU/Linux distributions don't care about clearly separating packages
from one another...
>> > or if your Unix has no support for library search paths via environment
>> > variables like Linux's LD_LIBRARY_PATH.
> While I appreciate concerns about supporting less fortunate operating
> environments, I don't think their existence should hold us back from doing
> the right thing on Linux.
For which definition of the right thing? :-)
>> In general, I feel that moving libraries around is a very bad idea,
>> because it will lead to failure most of the times, and that's why I don't
>> feel libtool should help people doing that.
> I see no reason why moving libraries around is a bad idea. I see defaulting
> to -rpath as a bad idea, which breaks moving libraries around.
Because you break a contract every time you remove a library from the
place in which it used to live.
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
Alexandre Oliva
> I've never understood what the .la scripts are for.
They contain inter-library dependency information, the location and
the name of the actual library, and any additional run-time paths
needed for the library dependencies. libtool (1.2d) is able to link
with an installed .la file, meaning that it will select the
appropriate switches to link the library, its dependencies and its
run-time paths into the program or libtool library being linked.
.la files are also used by libltdl, a portable dlopening library that
is available in the current libtool CVS tree, and that will be
released with libtool 1.3. libltdl will use the .la file to find out
the pathname of the file to be dlopened, as well as any dependencies
that must be dlopened before the library.
It is possible that libtool 1.3 will be able to find and use .la files
even when -L and -l switches are used to refer to it (currently it
requires the pathname of the .la file).
> With Debian and Red Hat, it's totally the opposite. Moving libraries
> around is what leads to upgrades being possible.
Then why do you find so much trouble with it?
> Totally agreed. You are worrying just a bit too much about this,
> though -- we don't need to worry about a switch that has to decide
> WHEN to disable -rpath, just a switch that understands, "Okay, the
> builder knows what he's talking about, no -rpath is fine with me".
In this case, you already have the suggestion of the ld wrapper
script, right? :-)
The point is that, if we support this flag, it must be supported in a
portable way, otherwise GNU/Linux developers may feel inclined to
enable it by default in the packages they maintain, and this will
result in their packages *having* to be installed with
--prefix=/usr/local, and even then, they will only work on GNU/Linux.
I want to avoid this situation at all costs.
Jules Bean
> > Having libtool default to -rpath is what's causing problems.
>
> This is IMHO completely backwards :-)
>
> When a program is linked with a shared library, a contract is
> established between them stating that the library (or any newer but
> compatible version thereof, on systems that support library
> versioning) will supply the symbols that the program resolved from it,
> and the program will be able to find the library at that point. If
> you move the library and replace it with an incompatible one, you're
> breaking the contract and the versioning mechanism, so you can't blame
> the program for crashing, nor the tool that created the program.
The contract does not state that the library will be found in a particular
location on the filesystem hierarchy. The contract simply states that the
library will be found. Which library is used can be determined by the
linker.
Where is the need for rpath here?
The combination of library versions uniquely identifies, to a suitably
well configured linker, which library to find.
> > I've seen one too many instances of "<foo> crashes on Debian" that turned
> > out to be "<foo> is a libc5 binary with an RPATH of /usr/X11R6/lib" which on
> > any reasonably up to date Debian system contains libc6 X libraries.
>
> See? You replaced one library with an incompatible one without
> modifying its version number to mark it as incompatible. Isn't this
> breaking the contract? How could you expect it to work?
It did work. On all binaries without -rpath. It worked. What do you
mean, 'How could we expect it to work?'
> >> > However, Alexandre Oliva <ol...@dcc.unicamp.br> brings up an important
> >> > point: -rpath is necessary if one is installing libraries and binaries
> >> > linked to those libraries in one's home directory,
>
> > That is a special case. The default should be sane for regular cases.
>
> You see the regular case as the one you use the most. I see it as the
> one I use the most. I don't install any packages in /usr or
> /usr/local because I find it *horrible* to have a huge /usr/local/bin
> without any clear separation between packages. It's a pity that the
> GNU/Linux distributions don't care about clearly separating packages
> from one another...
!!!!
I'm sorry, but I'm flabbergasted.
To think that the person in charge of libtool actively disagrees with
a) The whole point of packaging systems
b) The FHS
Our distribution cares greatly about separating packages from each other.
And it does it very well. There is no need to acheive this by enforcing
such package structure on the file-system - which package a file belongs
to is a concept orthogonal to where the file lives in a sensible
filesystem.
Worried,
Jules
/----------------+-------------------------------+---------------------\
| Jelibean aka | ju...@jellybean.co.uk | 6 Evelyn Rd |
| Jules aka | ju...@debian.org | Richmond, Surrey |
| Julian Bean | jm...@hermes.cam.ac.uk | TW9 2TF *UK* |
+----------------+-------------------------------+---------------------+
| War doesn't demonstrate who's right... just who's left. |
| When privacy is outlawed... only the outlaws have privacy. |
\----------------------------------------------------------------------/
Alexandre Oliva
> On 27 Jan 1999, Alexandre Oliva wrote:
>> > Having libtool default to -rpath is what's causing problems.
>> This is IMHO completely backwards :-)
>> When a program is linked with a shared library, a contract is
>> established [...] If you move the library and replace it with an
>> incompatible one, you're breaking the contract and the versioning
>> mechanism, so you can't blame the program for crashing, nor the
>> tool that created the program.
> The contract does not state that the library will be found in a particular
> location on the filesystem hierarchy.
Correct.
> The contract simply states that the library will be found. Which
> library is used can be determined by the linker.
Except that, if you replace the library with an incompatible one, you
*are* breaking the contract.
Furthermore, there's no reason to assume that the user will *not* have
used -rpath himself, so moving libraries does have a potential for
breaking user programs, therefore it should best be avoided.
> Where is the need for rpath here?
There's no need for it, but it doesn't cause any harm unless you break
the rules, i.e., replace a library with an incompatible one that holds
an apparently compatible version number. *This* is the root of all
your problems, not that fact that a rpath is specified. If you had
not replaced libraries with incompatible versions, the dynamic linker
would not have found it in the hard-coded rpath and would look for it
in the default search path, and find it in the appropriate alternate
location.
>> See? You replaced one library with an incompatible one without
>> modifying its version number to mark it as incompatible. Isn't this
>> breaking the contract? How could you expect it to work?
> It did work. On all binaries without -rpath. It worked. What do you
> mean, 'How could we expect it to work?'
Except for this `minor' restriction. You may be silently breaking
*user* programs because they have chosed to specify -rpath where it
was not necessary. If you think people *must not* use -rpath on your
system, you'd better completely disable it, not blame the user for
making use of a (IMO nice) feature of your system.
I think a solution that is not general is not a good solution. Since
the solution of moving shared libraries around has the potential of
causing problems, if I were you, I'd work harder to try to find a
complete solution (which I happen to have already suggested) instead
of trying to blame libtool or other users for doing things that are
perfectly correct, but not in the way that would let you replace
incompatible libraries.
> Our distribution cares greatly about separating packages from each other.
Not from the point of view of the end user. When they want to find a
tool, they may `ls /usr/bin /usr/local/bin' and will be presented
*thousands* of files. I'd rather have a classification system in
which I could have links (or similar) to all programs in a common
directory, but in which I could search for programs by subject. I'd
like to have development tools such as compilers in one directory,
text writing tools such as word processors and text editors in
another, system administration utilities in another, and so on.
Additionally, I'd like to be able to keep multiple versions of a
package installed, and the only way I could do that is by installing
each version in a separate directory, and selecting which version to
use via soft-links in the directory that happens to be in my PATH.
How does the current packaging system allows me to test one version of
a package while other users of the same host are running a stable
version of that tool? Or are the GNU/Linux distributions all moving
towards the Micro$oft model of single-user workstations?
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
Jason Gunthorpe
On 27 Jan 1999, Alexandre Oliva wrote:
> > Having libtool default to -rpath is what's causing problems.
>
> This is IMHO completely backwards :-)
You know, I seem to remember that there is another rather unpleasent
side-effect of rpath - it basically completely disables library searching
and thus disables LD_LIBRARY_PATH, once you have used rpath it is not easy
for a user replace that library (for whatever reason) and I find that
highly annoying.
> > I've seen one too many instances of "<foo> crashes on Debian" that turned
> > out to be "<foo> is a libc5 binary with an RPATH of /usr/X11R6/lib" which on
> > any reasonably up to date Debian system contains libc6 X libraries.
>
> See? You replaced one library with an incompatible one without
> modifying its version number to mark it as incompatible. Isn't this
> breaking the contract? How could you expect it to work?
Not exactly, the x libraries are fully compatible and were built with the
same source - the trouble is that one is linked with libc5 and the other
with libc6. In normal cases the dymanic linker would figure this out one
way or another with rpath this functionality is disabled as it overrides
the library versioning scheme.
The linux dynamic linker will resolve things in some magical way based on
the library dependencies and the program dependencies to locate the proper
library in many cases - rpath does cripples, not enhances, this function.
On another note, you have suggested that we use /usr/lib/libc6 and other
things to isolate libraries that have been recompiled with updated
dependencies - the trouble with this is that all the distributions and all
the distributers would have to agree on a scheme for this - otherwise a
'debian' binary would not function on a RH system because they used a
different scheme and their rpaths would be incompatibly different.
Furthermore this idea of a /usr/lib/libc6 becomes entirely unmanageable
when we start having soname changes for things like libgtk - we will have
the -same- problem with all the millions of libs that link with libgtk as
we did with libc6! The linux dynamic linker has the ability to resolve
these issues on it's own by carefully relocating the old
libraries, rpath simply does not.
We must be able to turn it off for libraries used on our system!
Jason
Jules Bean
>
> > library is used can be determined by the linker.
>
> Except that, if you replace the library with an incompatible one, you
> *are* breaking the contract.
We don't replace libraries with incompatible ones.
We bring in new libraries, which are incompatible. We keep the old ones,
which are compatible. Our linker knows how to tell which is which. Who
cares where they are stored? Actually, we do care where they are stored.
My point is, that it doesn't matter if the new library is in the location
the old library used to occupy. Our linker knows which is which.
>
> Furthermore, there's no reason to assume that the user will *not* have
> used -rpath himself, so moving libraries does have a potential for
> breaking user programs, therefore it should best be avoided.
This is a valid point. It is my feeling that -rpath should not be used
for libraries in the 'standard' paths, which allows them to move (which is
a useful feature). It is reasonable to use it for libraries in unusual
places (another useful feature).
> I think a solution that is not general is not a good solution. Since
> the solution of moving shared libraries around has the potential of
> causing problems, if I were you, I'd work harder to try to find a
> complete solution (which I happen to have already suggested) instead
> of trying to blame libtool or other users for doing things that are
> perfectly correct, but not in the way that would let you replace
> incompatible libraries.
I agree with your suggested solution, which amounted to more complex
understanding and use of inter-library dependencies by the tools.
However, I don't have the expertise to implement this. And I disagree
with you (strongly) about the correct use of the existing system.
>
> > Our distribution cares greatly about separating packages from each other.
>
> Not from the point of view of the end user. When they want to find a
> tool, they may `ls /usr/bin /usr/local/bin' and will be presented
> *thousands* of files. I'd rather have a classification system in
> which I could have links (or similar) to all programs in a common
> directory, but in which I could search for programs by subject. I'd
> like to have development tools such as compilers in one directory,
> text writing tools such as word processors and text editors in
> another, system administration utilities in another, and so on.
> Additionally, I'd like to be able to keep multiple versions of a
> package installed, and the only way I could do that is by installing
> each version in a separate directory, and selecting which version to
> use via soft-links in the directory that happens to be in my PATH.
>
We do have a classification system. And we don't use the filesystem for
it. We have lists of packages, with descriptions.
We even support separate versions of software, to some extent, although
I'd agree completely that our support in this area is not what it might
be.
> How does the current packaging system allows me to test one version of
> a package while other users of the same host are running a stable
> version of that tool? Or are the GNU/Linux distributions all moving
> towards the Micro$oft model of single-user workstations?
Of course not ;-)
If you want to test one version, you simply run it with
./configure --prefix /home/me/nicepackage
or whatever. But you knew that.
Jules
/----------------+-------------------------------+---------------------\
| Jelibean aka | ju...@jellybean.co.uk | 6 Evelyn Rd |
| Jules aka | ju...@debian.org | Richmond, Surrey |
| Julian Bean | jm...@hermes.cam.ac.uk | TW9 2TF *UK* |
+----------------+-------------------------------+---------------------+
| War doesn't demonstrate who's right... just who's left. |
| When privacy is outlawed... only the outlaws have privacy. |
\----------------------------------------------------------------------/
Alexandre Oliva
> On 27 Jan 1999, Alexandre Oliva wrote:
>> Except that, if you replace the library with an incompatible one, you
>> *are* breaking the contract.
> We don't replace libraries with incompatible ones.
Oh yes, you are.
> We bring in new libraries, which are incompatible. We keep the old ones,
> which are compatible. Our linker knows how to tell which is which.
Except that it can't do that if the user happened to use a switch that
was available, and its use didn't trigger any blinking light saying
that this would cause any future upgrade he might do to silently break
his program.
If you do want to be able to freely move libraries around, -rpath must
be forbidden. If -rpath is available for users, you can't move
libraries around and expect things to work.
> Who cares where they are stored?
The user who used -rpath?
> Actually, we do care where they are stored.
Ah! And that's where the potential conflict comes from.
> My point is, that it doesn't matter if the new library is in the location
> the old library used to occupy.
It may matter for the user who used -rpath.
> Our linker knows which is which.
Not always. Or maybe you can fix your linker to do the right thing
even if the user inadvertently (incorrectly, in your terms) specified
-rpath.
> We even support separate versions of software, to some extent, although
> I'd agree completely that our support in this area is not what it might
> be.
And that's the very reason why I've never been able to adopt any of
the available packaging systems: the only way to keep multiple
working versions is to use a separate directory for each version of
each package.
>> How does the current packaging system allows me to test one version of
>> a package while other users of the same host are running a stable
>> version of that tool? Or are the GNU/Linux distributions all moving
>> towards the Micro$oft model of single-user workstations?
> Of course not ;-)
Jeez!, I was sure I had added a smiley after that last sentence.
Sorry about that...
> If you want to test one version, you simply run it with
> ./configure --prefix /home/me/nicepackage
But isn't this exactly what the packaging systems are trying to avoid,
i.e., that people have to compile systems on their own? And then, how
could I make sure that my test build works exactly as the pre-compiled
upgrade, so that I could use the packaging system for the update?
This is something that I feel that should be taken care of by the
existing GNU/Linux distributions. In fact, I've got a bunch of ideas
that I'll probably never find time to implement :-( about how to
maintain multiple versions of packages installed and allowing each
user to select which version he wants to use, either by explicit
version number or by tags such as `stable', `test', `old', etc, tags
that are determined by the system manager when he installs the
package.
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
Jules Bean
>
> If you do want to be able to freely move libraries around, -rpath must
> be forbidden. If -rpath is available for users, you can't move
> libraries around and expect things to work.
There are lots of things which users can do which might appear to work,
and then break later.
The user should understand what -rpath is for, and why he should use it,
and when he should use it. As far as I can, it is *not* useful for the
packages which are part of a 'managed' system like Debian. It *is* useful
for users custom-compiling things in custom dirs. Can you tell -rpath to
store the rpath for libmycustomthing.so and not for libc.so? (Not a
rhetorical question - I genuinely don't know the answer). If not, then
I'd suggest that it's rpath that's broken.
> > We even support separate versions of software, to some extent, although
> > I'd agree completely that our support in this area is not what it might
> > be.
>
> And that's the very reason why I've never been able to adopt any of
> the available packaging systems: the only way to keep multiple
> working versions is to use a separate directory for each version of
> each package.
In general, it is not useful to have multiple versions of the same
package. In specific cases, it can be.
I don't think that libtool is the right vehicle for you to evangelise your
dislike of packaging systems and the FHS.
> > If you want to test one version, you simply run it with
>
> > ./configure --prefix /home/me/nicepackage
>
> But isn't this exactly what the packaging systems are trying to avoid,
> i.e., that people have to compile systems on their own? And then, how
> could I make sure that my test build works exactly as the pre-compiled
> upgrade, so that I could use the packaging system for the update?
You would download the debian package. You would very slightly edit the
rules file (which is like the makefile) to change the prefix passed to
configure. You would then run dpkg-buildpackage. Or simply debian/rules
binary.
If the program wasn't debian packaged at all, you'd read some
documentation on how to create a debian package ;-)
> This is something that I feel that should be taken care of by the
> existing GNU/Linux distributions. In fact, I've got a bunch of ideas
> that I'll probably never find time to implement :-( about how to
> maintain multiple versions of packages installed and allowing each
> user to select which version he wants to use, either by explicit
> version number or by tags such as `stable', `test', `old', etc, tags
> that are determined by the system manager when he installs the
> package.
These are all worthy ideas.
The idea of a 'fluid file system' or a 'tagged file system' which allows
to mark abitrary subsets of your files with tags (like
'belongs_to_package_x' or 'part_of_debian_version_2.0') is a very worthy
idea.
The idea of extending debian's package system to cope elegantly with
concurrent versions is (probably) a good idea.
However, none of these things are currently available for Linux.
What is available, is a distribution with well-thought (not arbitrary)
structure and standards. A distribution used by many people, although not
as many as RedHat (which has similar standards in most respects anyway).
We have considered library dependencies. We have a working system. It is
reasonably elegant. It is convenient for the users.
It is not the only answer.
Nonetheless, you are refusing to support it.
Jules
/----------------+-------------------------------+---------------------\
| Jelibean aka | ju...@jellybean.co.uk | 6 Evelyn Rd |
| Jules aka | ju...@debian.org | Richmond, Surrey |
| Julian Bean | jm...@hermes.cam.ac.uk | TW9 2TF *UK* |
+----------------+-------------------------------+---------------------+
| War doesn't demonstrate who's right... just who's left. |
| When privacy is outlawed... only the outlaws have privacy. |
\----------------------------------------------------------------------/
Alexandre Oliva
> On 27 Jan 1999, Alexandre Oliva wrote:
>> > Having libtool default to -rpath is what's causing problems.
>>
>> This is IMHO completely backwards :-)
> You know, I seem to remember that there is another rather unpleasent
> side-effect of rpath - it basically completely disables library searching
> and thus disables LD_LIBRARY_PATH, once you have used rpath it is not easy
> for a user replace that library (for whatever reason) and I find that
> highly annoying.
Well, he could always move the library elsewhere and put whatever he
wants in its place... Some people might be consider it a bug in the
dynamic linker, that could look at LD_LIBRARY_PATH before -rpath. I
think -rpath really means it, so I consider the behavior correct
(albeit a bit inconvenient for us, maintainers of libtool)
>> See? You replaced one library with an incompatible one without
>> modifying its version number to mark it as incompatible. Isn't this
>> breaking the contract? How could you expect it to work?
> Not exactly, the x libraries are fully compatible
except that they don't work with programs or libraries that had been
compiled with the old version. When I talk of compatibility, I'm
talking about full binary compatibility, not just apparent
compatibility.
> In normal cases the dymanic linker would figure this out one way or
> another with rpath this functionality is disabled as it overrides
> the library versioning scheme.
> The linux dynamic linker will resolve things in some magical way based on
> the library dependencies and the program dependencies to locate the proper
> library in many cases - rpath does cripples, not enhances, this function.
Since you do support -rpath in your system, you should probably extend
your dynamic linker to work in this case too, or risk taking the blame
for silently breaking applications, if the poor user ever understands
what happened to his program.
> On another note, you have suggested that we use /usr/lib/libc6 and other
> things to isolate libraries that have been recompiled with updated
> dependencies - the trouble with this is that all the distributions and all
> the distributers would have to agree on a scheme for this - otherwise a
> 'debian' binary would not function on a RH system because they used a
> different scheme and their rpaths would be incompatibly different.
This is a good point. But since all of you face this same problem,
and all of you use the same major versions of libc, you can probably
agree on a solution that would benefit you all. Or arrange, via
inter-package dependencies, that required libraries are installed in
the proper places.
> Furthermore this idea of a /usr/lib/libc6 becomes entirely
> unmanageable when we start having soname changes for things like
> libgtk - we will have the -same- problem with all the millions of
> libs that link with libgtk as we did with libc6!
It seems to me that the main problem has to do with library
versioning. Even though the existing library versioning mechanisms
are great to describe the evolution of the API offerred by a library,
they by no means describe the dependencies of a library, so we end up
with libraries that have the same version numbers but that are not
compatible at all.
What we all are desperately looking for is a mechanism to allow the
dynamic loader to pick the right version of a shared library out of a
set of multiple builds of the same version of a library, the main
difference between them being their dependencies.
Using the version number of a dependency to modify the version number
of the dependent library, in order to ensure that incompatibilities
are automatically handled, seems perfect, except that it will only
work when there is only one dependency. :-(
If there is more than one dependency, there is no way to ensure that
the major version number is increased whenever it has to be. Or I
just can't find out how :-)
Anyway, I strongly believe that being able to tell libtool not to use
-rpath and to freely move libraries around will only give you more
trouble to worry about in the future.
> We must be able to turn it off for libraries used on our system!
I have already told you one (ugly) way to do it, but I still don't
think it is a good idea in the general case.
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
Jules Bean
>
> > In normal cases the dymanic linker would figure this out one way or
> > another with rpath this functionality is disabled as it overrides
> > the library versioning scheme.
>
> > The linux dynamic linker will resolve things in some magical way based on
> > the library dependencies and the program dependencies to locate the proper
> > library in many cases - rpath does cripples, not enhances, this function.
>
> Since you do support -rpath in your system, you should probably extend
> your dynamic linker to work in this case too, or risk taking the blame
> for silently breaking applications, if the poor user ever understands
> what happened to his program.
You may be right. I haven't examined that situation in enough detail to
explain why the designers of the dynamic linker implemented it the way
they did. Maybe they got it wrong.
Note that 'we' (Debian) write neither the kernel, nor the dynamic linker.
You have previously objected to a proposed solution on the grounds that
'you want libtool to work on all systems'. It seems to me that if you
want to make libtool work on Linux, you should find a way of disabling
rpath, since rpath is broken on Linux.
> It seems to me that the main problem has to do with library
> versioning. Even though the existing library versioning mechanisms
> are great to describe the evolution of the API offerred by a library,
> they by no means describe the dependencies of a library, so we end up
> with libraries that have the same version numbers but that are not
> compatible at all.
You're right.
That is the underlying problem.
However, our dynamic linker *does* understand the problem. And it *does*
have a solution to it. And -rpath turns it off. So we cannot afford to
use -rpath.
Jules
/----------------+-------------------------------+---------------------\
| Jelibean aka | ju...@jellybean.co.uk | 6 Evelyn Rd |
| Jules aka | ju...@debian.org | Richmond, Surrey |
| Julian Bean | jm...@hermes.cam.ac.uk | TW9 2TF *UK* |
+----------------+-------------------------------+---------------------+
| War doesn't demonstrate who's right... just who's left. |
| When privacy is outlawed... only the outlaws have privacy. |
\----------------------------------------------------------------------/
Alexandre Oliva
> On 27 Jan 1999, Alexandre Oliva wrote:
> Can you tell -rpath to store the rpath for libmycustomthing.so and
> not for libc.so?
No, but, on some systems (for example, GNU/Linux), it is possible to
hard-code the full pathname of libmycustomthing.so into the shared
library itself, so that -rpath isn't needed, and the library cannot
ever be moved.
>> And that's the very reason why I've never been able to adopt any of
>> the available packaging systems: the only way to keep multiple
>> working versions is to use a separate directory for each version of
>> each package.
> In general, it is not useful to have multiple versions of the same
> package.
You're probably talking about the single-user workstation model. I'm
talking about a networked multi-user model, in which some users need
(for reasons only they understand :-) particular versions of, say, GNU
Emacs and gcc installed.
> I don't think that libtool is the right vehicle for you to evangelise your
> dislike of packaging systems and the FHS.
But debian-devel is probably a good place to talk about these ideas.
> What is available, is a distribution with well-thought (not arbitrary)
> structure and standards. A distribution used by many people, although not
> as many as RedHat (which has similar standards in most respects anyway).
I should note that I happen to appreciate both of them.
> Nonetheless, you are refusing to support it.
I'm not refusing to support it. I'm just inclined to avoid having an
easy-to-use flag to disable explicit hard-coding of library paths
because:
1) it would be hard to make it behave correctly in a portable way (and
libtool would be useless if it were not for being portable);
2) it should not be necessary if you play by libtool rules, i.e., you
pre-declare where libraries are going to be installed and keep them
there forever (or until they're no longer needed);
3) I don't want to regret having introduced a flag that caused as much
or more trouble than -rpath; and
4) I have already suggested a (dirty and ugly, I admit) hack that is
sufficient for your needs of not using -rpath at all in Debian
packages.
I hope you understand my position now. I should also note that I
myself have already wanted flags such as -hardcode-libdir for
hardcoding the full pathname of a shared library into itself (it's
mentioned in libtool/TODO) and -no-implicit-rpath (which is what you
happen to be asking for), but I have some trouble deciding who should
be responsible for deciding which flags to use for which libraries and
programs. I feel this decision should be left for the installer of
the package, but such installer-customizations aren't easy to
introduce in the existing installation meta-tools. I'd really
appreciate if someone had a bright idea about how to do that; I'd just
like to avoid providing such dangerous flags for inadvertent use.
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
Alexandre Oliva
>> Since you do support -rpath in your system, you should probably extend
>> your dynamic linker to work in this case too, or risk taking the blame
>> for silently breaking applications, if the poor user ever understands
>> what happened to his program.
> Note that 'we' (Debian) write neither the kernel, nor the dynamic linker.
Sorry, I had assumed you had hacked the dynamic linker in order to
support the /libc5 compatibility libraries.
> You have previously objected to a proposed solution on the grounds that
> 'you want libtool to work on all systems'. It seems to me that if you
> want to make libtool work on Linux, you should find a way of disabling
> rpath, since rpath is broken on Linux.
rpath is not broken, it just won't let one replace a library (meaning
moving a library to another directory and installing a new library in
its place) with one that appears to be compatible, according to the
version numbers, but is not. That's the root of the problem, and
that's why you want so much libtool not to use -rpath.
I feel we're not going to make progress in this discussion unless
someone comes up with a bright idea about how to allow the user to
select when/how to hardcode rpaths or not. In the beginning of this
discussion, Thomas Tanner suggested that -rpath could be dropped if it
would specify a standard library search directory.
Although I see problems in this behavior, that I have exposed in other
messages, I think it is a reasonable trade-off, and I'm willing to
accept a patch for libtool that would add to AC_PROG_LIBTOOL some code
that would check whether
enable_libtool_hardcoding_of_default_search_dirs=no (but
--disable-libtool-... should remain undocumented by now) and, in this
case, pass some flag to ltconfig that causes it to create a libtool
script that drops rpaths (but not xrpaths) that specify directories
listed in sys_lib_search_path_spec.
However, I'd like to ask you to avoid distributing packages with
pre-``compiled'' libtool scripts with such hardcoding disabled,
particularly the libtool Debian package.
This arrangement should take care of your immediate needs, while we
try to find a better way to do it, or even decide to do it by default.
> However, our dynamic linker *does* understand the problem. And it *does*
> have a solution to it. And -rpath turns it off. So we cannot afford to
> use -rpath.
As I have already pointed out, the solution is not complete, otherwise
you'd not be observing any problem.
Joey Hess
>
> On 27 Jan 1999, Alexandre Oliva wrote:
>
> >
> > This is IMHO completely backwards :-)
>
> You know, I seem to remember that there is another rather unpleasent
> side-effect of rpath - it basically completely disables library searching
> and thus disables LD_LIBRARY_PATH, once you have used rpath it is not easy
> for a user replace that library (for whatever reason) and I find that
> highly annoying.
Yes, for example if a X program is linked with rpath, and you want to use
something like xaw39 (or xaw95 or nextaw) instead of the usual xaw widgets
with it, you can't. This has it's uses if you know the program won't work
with those replacement libraries, but most programs do so it's a horrid
default.
--
see shy jo
Marcus Brinkmann
> > I don't think that libtool is the right vehicle for you to evangelise your
> > dislike of packaging systems and the FHS.
>
> But debian-devel is probably a good place to talk about these ideas.
Please start another thread under a different subject for this. It is an
important but mostly unrelated topic.
> > Nonetheless, you are refusing to support it.
>
> I'm not refusing to support it. I'm just inclined to avoid having an
> easy-to-use flag to disable explicit hard-coding of library paths
> because:
>
> 1) it would be hard to make it behave correctly in a portable way (and
> libtool would be useless if it were not for being portable);
So why not move the discussion in a direction where we can talk about
portability, instead trying to convince each other that the own ideas are
the best. We are faced here with two different ideas about library
dependencies (handling them at compile/installation time on your side,
or handling them at run time on our side), so we better talk about a way how
to address both.
IMHO, both have their positive and their negative. Now let's try to get
constructive.
> 2) it should not be necessary if you play by libtool rules, i.e., you
> pre-declare where libraries are going to be installed and keep them
> there forever (or until they're no longer needed);
See above. Libtools rules are not necessarily the only "true" rules. Let's
try to get support for diversity in library handling.
> 3) I don't want to regret having introduced a flag that caused as much
> or more trouble than -rpath; and
I don't understand this comment. Which "trouble" with "--rpath" do you mean?
Until now, I only heard from you that rpath is the ideal solution. Maybe we
can get a better understanding of each other when we outline the negative of
the own solution. I'll try a beginning:
No rpath makes it harder for the linker to find the library. In fact, on
some systems, it may make it impossible, so rpath is essential on some
systems (though not GNU/Linux or GNU/Hurd). On other systems, the linker
needs some smart way to resolve dependencies and search for libraries.
No rpath makes it harder for a user to determine exactly which system
libraries he links with. (With rpath, though, it only works when the system
administrator never changes the library file at this place, too).
This is in a hurry. Could you do me tha favour and list some negative points
of using rpath, too?
> 4) I have already suggested a (dirty and ugly, I admit) hack that is
> sufficient for your needs of not using -rpath at all in Debian
> packages.
We can find our own hacks (and do so since a long time). Now we are
interested in a compatible, portable and general solution. As the libtool
maintainer, you should be the ideal person to talk about such a solution.
I think we understood by now that a simple disable flag does not satisfy
your high ambitions wrt to portability. Let's move on to better solutions.
> I hope you understand my position now. I should also note that I
> myself have already wanted flags such as -hardcode-libdir for
> hardcoding the full pathname of a shared library into itself (it's
> mentioned in libtool/TODO) and -no-implicit-rpath (which is what you
> happen to be asking for), but I have some trouble deciding who should
> be responsible for deciding which flags to use for which libraries and
> programs.
Maybe you should not be the one to decide at all? Offer flexible solutions,
where the last person can override the (possible) default given by others.
This means, the package can provide a default, which can be overridden at
compilation time. Finally, the installer can override both.
> I feel this decision should be left for the installer of
> the package, but such installer-customizations aren't easy to
> introduce in the existing installation meta-tools.
Implement what works and provide a framework for things which should be made
to work. This means, implement those flags for compilation, and
leave installation possible for later.
Marcus
--
"Rhubarb is no Egyptian god." Debian GNU/Linux finger brinkmd@
Marcus Brinkmann http://www.debian.org master.debian.org
Marcus.B...@ruhr-uni-bochum.de for public PGP Key
http://homepage.ruhr-uni-bochum.de/Marcus.Brinkmann/ PGP Key ID 36E7CD09
Thomas Tanner
> More than that (and it was my fault to have failed to mention that
> before): libtool will hard-code the installation directory of the
> library into the `libdir' variable of the .la script it installs.
> Therefore, if one moves the library then tries to link with the .la
> file, he loses. There's also the dlopening issue: libltdl (to be
> released with libtool 1.3) will dlopen a library in the directory
> pointed to by `libdir' too.
Well, that's actually not true. libltdl will try to dlopen the library
in the directory where the .la file was found, too.
> In general, I feel that moving libraries around is a very bad idea,
> because it will lead to failure most of the times, and that's why I
> don't feel libtool should help people doing that.
Moving libraries should be avoided, but libtool shouldn't
make it impossible.
> Actually, not issuing -rpath or equivalent is quite easy to do, but
> choosing *when* not to do it is the part that is hard to port. Thomas
> Tanner has suggested that we could omit the -rpath switch for
> libraries that are supposed to be installed in directories listed in
> the default search path. However, the default search PATH might
> change, and, even if it did not, it would be possible to link an
> application with a library in, say, /usr/local/lib, and then find out
> that at run-time it loads a library with the same name in /usr/lib,
> because /usr/lib appears first in the standard search path.
I still support my proposal.
We have the same problem with the -L option and we
can't do anything against it.
For example, if you link with "-L/usr/local/lib -lfoo -L/usr/lib -lbar"
the linker might link the library libbar in /usr/local/lib instead
of the one in /usr/lib.
AFAIK there's no clean and portable way to avoid this.
So _the user_ has to ensure that this won't happen.
If it's impossible for -L, then it's not necessary for -rpath.
--
Thomas Tanner ----------------------------------------
email: tanner@(gmx.de|ggi-project.org|gnu.org|mucl.de)
web: http://home.pages.de/~tanner
visit: www.ggi-project.org www.gnustep.org
Jules Bean
> On Jan 27, 1999, Jules Bean <jm...@hermes.cam.ac.uk> wrote:
>
> > On 27 Jan 1999, Alexandre Oliva wrote:
>
[watch indenting carefully : I wrote this next bit, of course]
> > In general, it is not useful to have multiple versions of the same
> > package.
>
> You're probably talking about the single-user workstation model. I'm
> talking about a networked multi-user model, in which some users need
> (for reasons only they understand :-) particular versions of, say, GNU
> Emacs and gcc installed.
In general, such a situation only arises because of a bug in the software.
That is why I say that, in general, it is not useful.
> > Nonetheless, you are refusing to support it.
>
> I'm not refusing to support it. I'm just inclined to avoid having an
> easy-to-use flag to disable explicit hard-coding of library paths
> because:
>
> 1) it would be hard to make it behave correctly in a portable way (and
> libtool would be useless if it were not for being portable);
Special case-it for linux, if you will. Libtool has plenty of special
cases as it is.
>
> 2) it should not be necessary if you play by libtool rules, i.e., you
> pre-declare where libraries are going to be installed and keep them
> there forever (or until they're no longer needed);
>
We don't want to play by libtool rules. We don't see that as a sensible
restriction.
[more information to follow in a separate follow-up]
Jules
/----------------+-------------------------------+---------------------\
| Jelibean aka | ju...@jellybean.co.uk | 6 Evelyn Rd |
| Jules aka | ju...@debian.org | Richmond, Surrey |
| Julian Bean | jm...@hermes.cam.ac.uk | TW9 2TF *UK* |
+----------------+-------------------------------+---------------------+
| War doesn't demonstrate who's right... just who's left. |
| When privacy is outlawed... only the outlaws have privacy. |
\----------------------------------------------------------------------/
Ulrich Drepper
> rpath is broken. You said as much yourself. rpath is broken because it
> *overrides* all other sorts of library searching.
I think people here do not know about $ORIGIN. This allows to define
relative rpaths. E.g., a package with a program foo and a library
libbar.so where foo is installed in $PATH/bin and libbar.so is defined
in $PATH/lib should use
-rpath \$ORIGIN/../lib
The $ORIGIN is defined relative to the location of the object
containing the reference.
This is available in Solaris 7 (maybe 2.6?) and Linux w/ glibc 2.1.
--
---------------. drepper at gnu.org ,-. 1325 Chesapeake Terrace
Ulrich Drepper \ ,-------------------' \ Sunnyvale, CA 94089 USA
Cygnus Solutions `--' drepper at cygnus.com `------------------------
Jules Bean
> On Jan 27, 1999, Jules Bean <jm...@hermes.cam.ac.uk> wrote:
>
> >> Since you do support -rpath in your system, you should probably extend
> >> your dynamic linker to work in this case too, or risk taking the blame
> >> for silently breaking applications, if the poor user ever understands
> >> what happened to his program.
>
> > Note that 'we' (Debian) write neither the kernel, nor the dynamic linker.
>
> Sorry, I had assumed you had hacked the dynamic linker in order to
> support the /libc5 compatibility libraries.
Absolutely not.
This is one of my main points, in fact.
Our dynamic linker is already very good at solving versioning problems.
It doesn't need us to guarantee the locations of libraries to work - it
can get it all correct anyway.
Therefore, we chose to solve that particular problem (the libc5-6
transition) by moving libraries around, knowing that our linker was up to
the job.
You are now saying that libtool, by giving no way of avoiding rpath, will
not support this scheme.
>
> > You have previously objected to a proposed solution on the grounds that
> > 'you want libtool to work on all systems'. It seems to me that if you
> > want to make libtool work on Linux, you should find a way of disabling
> > rpath, since rpath is broken on Linux.
>
> rpath is not broken, it just won't let one replace a library (meaning
> moving a library to another directory and installing a new library in
> its place) with one that appears to be compatible, according to the
> version numbers, but is not. That's the root of the problem, and
> that's why you want so much libtool not to use -rpath.
>
rpath is broken. You said as much yourself. rpath is broken because it
*overrides* all other sorts of library searching.
For all I know, there may be a design justification for this. My
understanding of rpath was that it was an 'option of last resort' when you
couldn't make it work any other way (and hence, the overriding of all
other search options is justified).
However, using rpath as default is bad for our setup. Since you think it
is right to use rpath as default, I therefore claim it is broken.
> I feel we're not going to make progress in this discussion unless
> someone comes up with a bright idea about how to allow the user to
> select when/how to hardcode rpaths or not. In the beginning of this
> discussion, Thomas Tanner suggested that -rpath could be dropped if it
> would specify a standard library search directory.
Easy.
Allow libtool to be run with the option
libtool --no-rpath.
This would then be run in all debian-maintained versions of
libtoolized packages, and they'd work.
Furthermore, when talking to software developers, I'd recommend they use
this option by default, disabling it with a
./configure --use-libtool-rpath
if the user really needs the functionality.
> > However, our dynamic linker *does* understand the problem. And it *does*
> > have a solution to it. And -rpath turns it off. So we cannot afford to
> > use -rpath.
>
> As I have already pointed out, the solution is not complete, otherwise
> you'd not be observing any problem.
What do you mean the solution is not complete?
It works. It works well.
People developing only for linux, and not using libtool, wouldn't even
*think* of using rpath since it is a) unnecessary and b) a bad thing.
However, people don't develop only for linux (this is a very good thing).
And because various OSes have wildly different rules for dynamic
libraries, libtool was born to help developers keep their makefiles sane.
And libtool is now deciding not to support the linux model.
Jules
/----------------+-------------------------------+---------------------\
| Jelibean aka | ju...@jellybean.co.uk | 6 Evelyn Rd |
| Jules aka | ju...@debian.org | Richmond, Surrey |
| Julian Bean | jm...@hermes.cam.ac.uk | TW9 2TF *UK* |
+----------------+-------------------------------+---------------------+
| War doesn't demonstrate who's right... just who's left. |
| When privacy is outlawed... only the outlaws have privacy. |
\----------------------------------------------------------------------/
Jason Gunthorpe
On 27 Jan 1999, Alexandre Oliva wrote:
> > You know, I seem to remember that there is another rather unpleasent
> > side-effect of rpath - it basically completely disables library searching
> > and thus disables LD_LIBRARY_PATH, once you have used rpath it is not easy
> > for a user replace that library (for whatever reason) and I find that
> > highly annoying.
>
> Well, he could always move the library elsewhere and put whatever he
> wants in its place... Some people might be consider it a bug in the
> dynamic linker, that could look at LD_LIBRARY_PATH before -rpath. I
> think -rpath really means it, so I consider the behavior correct
> (albeit a bit inconvenient for us, maintainers of libtool)
Actually you want to know why I remember this? I used libtool a while back
and I installed a copy of my program in /usr/bin and /usr/lib and wanted
to us a new local copy of my libtool program. Of course libtool had used
-rpath to make sure that my local binary used /usr/lib (as it was intended
to be installed there someday) and then used LD_LIBRARY_PATH in the
wrapper script to try to override this. Needless to say it did not work
and it took me a bit of figuring to determine why my changes had no
effect. Even in an all libtool environment rpath causes pain.
Actually you might have something there with the search order. What if the
linker were to examine it's library list before examining rpath, rpath
would be like a compiled in default search path instead of a compiled in
search path override. The ld-linux people probably would not go for that
though.
Incidently from what I've read I think that description of rpath more
closely matches what you want to use it for than does it's current
behavior.
> > The linux dynamic linker will resolve things in some magical way based on
> > the library dependencies and the program dependencies to locate the proper
> > library in many cases - rpath does cripples, not enhances, this function.
>
> Since you do support -rpath in your system, you should probably extend
> your dynamic linker to work in this case too, or risk taking the blame
> for silently breaking applications, if the poor user ever understands
> what happened to his program.
Well then you change the meaning of rpath, if you accept rpath means 'use
this library not matter what the consequences' then it is arguable the
user should expect that from rpath.
> This is a good point. But since all of you face this same problem,
> and all of you use the same major versions of libc, you can probably
> agree on a solution that would benefit you all. Or arrange, via
Lol! There has been little luck in that area, the only way to do it would
really be to convince all the upstream maintianers to do that and then
force those changes on the distributions. In any event as we agree below
this is not an adaquate solution.
> inter-package dependencies, that required libraries are installed in
> the proper places.
No, we are talking about 3rd party binary applications such as netscape -
arguably they are a compelling reason why we use the scheme we do.
> It seems to me that the main problem has to do with library
> versioning. Even though the existing library versioning mechanisms
> are great to describe the evolution of the API offerred by a library,
> they by no means describe the dependencies of a library, so we end up
> with libraries that have the same version numbers but that are not
> compatible at all.
Bingo. We have a dynamic linker that can complete the second half of the
puzzle if all available major versions of a library are in the search
path, it will pick the one with the matching dependencies, we as a
distribution vendor take steps to make sure that any compatibility
libraries remain within the linkers search path.
> What we all are desperately looking for is a mechanism to allow the
> dynamic loader to pick the right version of a shared library out of a
> set of multiple builds of the same version of a library, the main
> difference between them being their dependencies.
I belive we already have this capablility - the only problem is that rpath
disables it. If rpath did not break the normal library searching
mechanisms then nobody would care that libtool uses it. [See above]
> If there is more than one dependency, there is no way to ensure that
> the major version number is increased whenever it has to be. Or I
> just can't find out how :-)
No, you will never be able to work this out because you have to get the
upstream author to do it, assume for instance that we changed our xlib6
soname to be libX11.so.6-C6-ICE6 then anything that is linked on our
system will not work on a redhat one unless they did the same and then
people who compile out of the tar ball would be in trouble and so on.
Permuting the soname is not a good enough solution because it cannot be
applied universally and consistently.
> > We must be able to turn it off for libraries used on our system!
>
> I have already told you one (ugly) way to do it, but I still don't
> think it is a good idea in the general case.
Didn't we decide that all of the available alternatives that you have
suggested are not a feasable solution (does this mail help make it clear
why)?
Now correct me if I am wrong but the story goes like this:
- rpath is good because it allows a binary to have a shared library
in a non-standard place without requiring the user to use
LD_LIBRARYPATH or the sysadmin to add that place to the search
path
- rpath is bad because it disables LD_LIBRARYPATH
- rpath is bad because it disables the linkers automatic versioning
mechanism
- rpath is bad because it prevents you from moving shared libraries
around freely.
Jason
Ben Gertzfield
Ben> With Debian and Red Hat, it's totally the opposite. Moving
Ben> libraries around is what leads to upgrades being possible.
Alexandre> Then why do you find so much trouble with it?
Because of -rpath. :)
That's the only reason we've run into trouble with it.
--
Brought to you by the letters D and P and the number 9.
"What's green and sits in the corner? ... A naughty frog!"
Debian GNU/Linux maintainer of Gimp and GTK+ -- http://www.debian.org/
I'm on FurryMUCK as Che, and EFNet/Open Projects IRC as Che_Fox.
Bernard Dautrevaux
> De: Ulrich Drepper [SMTP:dre...@cygnus.com]
> Date: jeudi 28 janvier 1999 00:54
> À: Jules Bean
> Cc: Alexandre Oliva; Debian Developers; bug-l...@gnu.org
> Objet: Re: -rpath with libtool and Debian Linux
>
> Jules Bean <jm...@hermes.cam.ac.uk> writes:
>
> > rpath is broken. You said as much yourself. rpath is broken
> because it
> > *overrides* all other sorts of library searching.
>
> relative rpaths. E.g., a package with a program foo and a library
> libbar.so where foo is installed in $PATH/bin and libbar.so is defined
> in $PATH/lib should use
>
> -rpath \$ORIGIN/../lib
>
> The $ORIGIN is defined relative to the location of the object
> containing the reference.
>
> This is available in Solaris 7 (maybe 2.6?) and Linux w/ glibc 2.1.
>
shared library and a set of programs using it; then relative paths are
OK. By this does not solve the problem of finding independently
installed libraries or system ones... There -rpath will force to use the
absolute path of these libraries on the development system and if
installed as a binary package, these may be in slightly differing places
(I'm sure there is system libs that are in /usr/lib in some Linux
distribs but in /lib or /usr/local/lib in others...)
AFAIK this is the subject of this whole thread about -rpath: how could
we create binary distributions that WORK... (other than statically
linking all executables of course...)
Regards,
Bernard
------------------------------------------------------------------------
------
Bernard Dautrevaux
Microprocess Ingéniérie
97 bis, rue de Colombes
92400 COURBEVOIE
FRANCE
Tel: +33 (0) 1 47 68 80 80
Fax: +33 (0) 1 47 88 97 85
e-mail: dautr...@microprocess.com
b.daut...@usa.net
------------------------------------------------------------------------
------
Bernard Dautrevaux
> De: Alexandre Oliva [SMTP:ol...@dcc.unicamp.br]
> Date: mercredi 27 janvier 1999 20:53
> À: J.H.M. Dassen
> Cc: debian...@lists.debian.org; yo...@gimp.org;
> bug-l...@gnu.org; lib...@packages.debian.org
> Objet: Re: -rpath with libtool and Debian Linux
>
>
> > On Wed, Jan 27, 1999 at 17:07:30 -0200, Alexandre Oliva wrote:
> >> You might have included my suggestion to prevent having to move
> libraries
> >> in the first place: creating a libc6-specific directory right now,
> instead
> >> of installing libraries in /usr/lib and having to move them into
> another
> >> directory when libc7 should be released.
>
> > I'm sorry, but this is IMHO completely backwards. On Linux systems,
> there is
> > nothing wrong with moving libraries around as the need arises.
>
> Except that you risk replacing a library with an incompatible one.
> That's what has caused you so much trouble. If, instead of moving the
>
> X11 libraries to another dir and creating new, incompatible versions
> under the same pathname, you had created new versions in other
> directories, no unexpected crashes would have occurred.
>
>
> This is IMHO completely backwards :-)
>
> compatible version thereof, on systems that support library
> versioning) will supply the symbols that the program resolved from it,
> you move the library and replace it with an incompatible one, you're
> breaking the contract and the versioning mechanism, so you can't blame
> the program for crashing, nor the tool that created the program.
>
using the compiler, to the system on which you install. The problem is
with the exact contract passed, and interpretation of this contract.
You say the contract is "I want to find THERE the lib that does THIS.x
and THAT.x"; I think (and that's at least true for Linux) the contract
the compiler and linker has signed was twofold; it says:
1) "I will give you the library that makes THIS.x and THAT.x"
2) "The prefered library I want you to use to obtain THIS and THAT
is there and makes THIS.x and THAT.x"
Now you trick it with -rpath in finding both the ".x" and THERE and all
the problem comes from there...
An analogy: When I wand to get hot water in restrooms, I just look at
the tap, and turn the red one; I do not INSIST on opening the left one,
risking to get cold water...
> > I've seen one too many instances of "<foo> crashes on Debian" that
> turned
> > out to be "<foo> is a libc5 binary with an RPATH of /usr/X11R6/lib"
> which on
> > any reasonably up to date Debian system contains libc6 X libraries.
>
> See? You replaced one library with an incompatible one without
> modifying its version number to mark it as incompatible. Isn't this
> breaking the contract? How could you expect it to work?
>
Moving the library was just fulfilling the 2nd part of the contract:
"You will find here the prefered library to use for THIS and THAT".
> > The X example also shows that the problem isn't limited to /usr/lib
> either.
> > What's next? /usr/local/lib/libc6 ?
>
> Maybe some library versioning mechanism that allows incompatible
> changes to be marked as such. Maybe an environment variable or some
> file in /etc containing a number that will be added to the major
> version number of any library libtool creates, so that they will be
> marked as incompatible.
>
Under Linux at least the incompatibilities we are talking here ARE
managed by the dynamic linker, IFF we do not trick it saying him (using
-rpath) "Do not be smart, just load the library from there". YOU are
breaking the Linux contract...
It's possible the way you are using -rpath correspond to the "contract"
of some OSes, but surely not LINUX :-(; and if there is different
contracts to be adhered to depending on the OS, it seems that exaactly
why we use autoconf/automake/libtool: allow us NOT to bother about OS
diferences but rather be able to ask libtool: "Do what teh OS expect you
to do to fulfill its contracts"
So my opinion, at least on Linux, would be not to use -rpath, except if
we need to access shared libraries that WE install together with the
executables; and then install it in some package-dependant directory
where we have no risk to get ANY other lib, pass ONLY this dir to the
exec using -rpath, and then all should be OK.
Now for the one that want to compile and install a set of interdependant
packages in it's home directory on some OS where he is not able to pass
some kind of an LD_LIBRARY_PATH to the exec, then it may use some option
to each compiles asking that -rpath be passed for all links, and warn
him he should not try to distribute or move the resultant installed
packages, and may need to reinstall them if changing its system
configuration.
Regards,
Bernard
> >> > However, Alexandre Oliva <ol...@dcc.unicamp.br> brings up an
> important
> >> > point: -rpath is necessary if one is installing libraries and
> binaries
> >> > linked to those libraries in one's home directory,
>
> > That is a special case. The default should be sane for regular
> cases.
>
> You see the regular case as the one you use the most. I see it as the
>
> one I use the most. I don't install any packages in /usr or
> /usr/local because I find it *horrible* to have a huge /usr/local/bin
> without any clear separation between packages. It's a pity that the
> GNU/Linux distributions don't care about clearly separating packages
> from one another...
>
> >> > or if your Unix has no support for library search paths via
> environment
> >> > variables like Linux's LD_LIBRARY_PATH.
>
> > While I appreciate concerns about supporting less fortunate
> operating
> > environments, I don't think their existence should hold us back from
> doing
> > the right thing on Linux.
>
> For which definition of the right thing? :-)
>
> >> In general, I feel that moving libraries around is a very bad idea,
> >> because it will lead to failure most of the times, and that's why I
> don't
> >> feel libtool should help people doing that.
>
> > I see no reason why moving libraries around is a bad idea. I see
> defaulting
> > to -rpath as a bad idea, which breaks moving libraries around.
>
> Because you break a contract every time you remove a library from the
> place in which it used to live.
>
> --
> Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
> oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
> Universidade Estadual de Campinas, SP, Brasil
>
Bernard Dautrevaux
> À: Jules Bean
> Cc: J.H.M. Dassen; debian...@lists.debian.org; yo...@gimp.org;
>
> > On 27 Jan 1999, Alexandre Oliva wrote:
>
> > We even support separate versions of software, to some extent,
> although
> > I'd agree completely that our support in this area is not what it
> might
> > be.
>
> And that's the very reason why I've never been able to adopt any of
> the available packaging systems: the only way to keep multiple
> working versions is to use a separate directory for each version of
> each package.
>
Good point. I'm not either using any "standard" packaging system like
rpm or deb, just due to this. I use a (quite primitive :-)) homegrown
packaging system taht allows my users to install each realease in a
different location, anywhere they want on their disk. Period. (I think
difficult to be more opened...)
My problem is that if in my package I create a shared library that will
be used by the exec in this package, and I use libtool for this, then
the use of -rpath for this cause the user to have to install exactly
where I build it (that is in some highly non-standard place due to our
complex development environment), or else the -rpath will point in some
unexistent place and (as Linux will ignore LD_LIBRARY_PATH) I can't
override it...
Not to talk about having to cope with you, Linux packagers, moving
system libraries from release to release :-) But I'm confident your
tools are coherent and that if you move libraries, your linker will find
them in the proper place.
So I think -rpath may be useful INSIDE packages, IFF we use relative
pathnames (that \$ORIGIN I saw in a message yesterday I think), but
should only be used for standard libraries if the OS distributor advised
us to, and NOT USED IF THE CONTRACT I PROPOSE US say NOT TO USE IT!
The purpose of libtool is to help us build portable code, whose built
rules are adapted to the build platform, isn't-it :-), so let's do it:
if Linux distribution maintainers advise us not to use -rpath on their
distribution, we must trust them (and blame them if the solution they
recommend do not work 8-(, but then THEY have to correct their
implementation! :-))
> >> How does the current packaging system allows me to test one version
> of
> >> a package while other users of the same host are running a stable
> >> version of that tool? Or are the GNU/Linux distributions all
> moving
> >> towards the Micro$oft model of single-user workstations?
>
> > Of course not ;-)
>
> Jeez!, I was sure I had added a smiley after that last sentence.
> Sorry about that...
>
> > If you want to test one version, you simply run it with
>
> > ./configure --prefix /home/me/nicepackage
>
> But isn't this exactly what the packaging systems are trying to avoid,
> i.e., that people have to compile systems on their own? And then, how
> could I make sure that my test build works exactly as the pre-compiled
>
> upgrade, so that I could use the packaging system for the update?
>
>
> This is something that I feel that should be taken care of by the
> existing GNU/Linux distributions. In fact, I've got a bunch of ideas
> that I'll probably never find time to implement :-( about how to
> maintain multiple versions of packages installed and allowing each
> user to select which version he wants to use, either by explicit
> version number or by tags such as `stable', `test', `old', etc, tags
> that are determined by the system manager when he installs the
> package.
>
I think this is slightly off topic, or we may have to start a whole new
mailing list to discuss packaging systems (I don't know but I'm quite
sure there exist mailing lists or newsgroups on RPM, DEB, etc. :-)) Here
we are talking about how libtool could help isolate us from the
peculiarities of different OSes, and I think this is enough work :-)
Regards,
Bernard
Alexandre Oliva
>> 1) it would be hard to make it behave correctly in a portable way (and
>> libtool would be useless if it were not for being portable);
> Special case-it for linux, if you will. Libtool has plenty of special
> cases as it is.
Not in the interface it presents to its users. Internally, it's
obviously full of special cases to support all the crazyness of OS
developers and their wonderful dynamic linkers.
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
Alexandre Oliva
> On Wed, Jan 27, 1999 at 08:22:09PM -0200, Alexandre Oliva wrote:
>> 3) I don't want to regret having introduced a flag that caused as much
>> or more trouble than -rpath; and
> I don't understand this comment. Which "trouble" with "--rpath" do you mean?
The exact problem the Debian developers have been complaining about.
The more I think about the problem, the more I see that the problem
they're facing is an incomplete hack of ld.so, in that it modifies the
library search path under certain circumstances, but not in all
circumstances that would need it. I.e., if ld.so would replace
/usr/lib with /usr/lib/libc5:/usr/lib whenever it found that the
application was linked with libc5, even if /usr/lib had been
hard-coded in the program's rpath, everything would be working
beautifully.
The fact that libtool always hard-codes /usr/lib in a program that is
linked with a libtool library that is (going to be) installed in
/usr/lib is a side issue, because users may do it on their own, and,
IMHO, they're not to be blamed because of doing that.
> Until now, I only heard from you that rpath is the ideal solution.
Maybe you haven't read (or even received) one of my messages in the
beginning of this discussion, in which I stated that even rpath is
wrong in certain circumstances. For example, if a program is linked
with libfoo, that lives in /foo, and libbar, that lives in /bar, but
there's a compatible (WRT to version numbers, not necessarily WRT
version of libc) version of libbar in /foo, the linker will pick the
one in /foo, not the one in libbar.
In fact, Thomas Tanner's suggestion of dropping -rpath when it would
hard-code a standard library just makes this problem more apparent: if
a (l)user installs a library in /usr/local/lib, but there's a library
with the same name in /usr/lib, the version in /usr/lib will be used
at run-time. This may have very bad consequences, such as the
never-ending problem of (l)users installing gcc 2.[78].* in /usr and
egcs 1.0.* in /usr/local, each one with its particular (and
incompatible) version of libstdc++ (and libg++, for some).
The only way to avoid this potential problem is to hard-code the full
library path in the soname of the library itself, but then, this is
not portable, and it is not desirable because then you can't move a
library at all (read it again: I'm not talking about replacing a
library with an apparently compatible but actually incompatible
version of it :-)
> No rpath makes it harder for a user to determine exactly which system
> libraries he links with. (With rpath, though, it only works when the system
> administrator never changes the library file at this place, too).
It is not a problem to move a library, as long as it can be found in
other hard-coded rpath, in the default search path or in
LD_LIBRARY_PATH. A problem only arises if an apparently compatible
but actually incompatible library is found first, which is exactly the
problem that the Debian developers are facing.
>> 4) I have already suggested a (dirty and ugly, I admit) hack that is
>> sufficient for your needs of not using -rpath at all in Debian
>> packages.
> We can find our own hacks (and do so since a long time). Now we are
> interested in a compatible, portable and general solution. As the libtool
> maintainer, you should be the ideal person to talk about such a solution.
> I think we understood by now that a simple disable flag does not satisfy
> your high ambitions wrt to portability. Let's move on to better solutions.
And, in fact, such a flag, that I said I was willing to accept,
wouldn't actually help you much, because applications are distributed
with their own versions of libtool, and you'd have to modify them all,
or use your own hacked version of libtool to build all applications
(via make LIBTOOL=/hacked/for/debian/libtool), so there'd not be much
point in introducing this change only in libtool 1.3 and newer... :-(
>> I hope you understand my position now. I should also note that I
>> myself have already wanted flags such as -hardcode-libdir for
>> hardcoding the full pathname of a shared library into itself (it's
>> mentioned in libtool/TODO) and -no-implicit-rpath (which is what you
>> happen to be asking for), but I have some trouble deciding who should
>> be responsible for deciding which flags to use for which libraries and
>> programs.
> Maybe you should not be the one to decide at all?
I'm certainly not.
> Offer flexible solutions, where the last person can override the
> (possible) default given by others.
That's the hard part. Overriding may have to occur in a per-library
and/or per-program basis, so I haven't been able to come up with a
satisfactory solution.
> This means, the package can provide a default, which can be overridden at
> compilation time. Finally, the installer can override both.
That's exactly what I'm looking for. But I wouldn't like to install a
quick hack now that would later reveal to be a step in the wrong
direction. That's why I'm being so conservative about all this issue.
Alexandre Oliva
> Therefore, we chose to solve that particular problem (the libc5-6
> transition) by moving libraries around, knowing that our linker was up to
> the job.
It is now clear that it is not. :-(
> rpath is broken. You said as much yourself. rpath is broken because it
> *overrides* all other sorts of library searching.
But isn't overriding library searching exactly what the ld.so hack is
doing? Why is one of those blessed for doing that, while the other is
crucified for guilt of all the sins of the world? :-)
>> > However, our dynamic linker *does* understand the problem. And it *does*
>> > have a solution to it. And -rpath turns it off. So we cannot afford to
>> > use -rpath.
>> As I have already pointed out, the solution is not complete, otherwise
>> you'd not be observing any problem.
> What do you mean the solution is not complete?
It does not replace /usr/lib with /usr/lib/libc5:/usr/lib in the
rpath. That's what is causing you trouble, not the fact that /usr/lib
is hard-coded in the program.
> It works. It works well.
If it worked well you wouldn't be complaining about a problem that is
caused by its incomplete behavior, but that could be avoided by
modifying other pieces of software that are doing their jobs
correctly.
Alexandre Oliva
> Jules Bean <jm...@hermes.cam.ac.uk> writes:
>> rpath is broken. You said as much yourself. rpath is broken because it
>> *overrides* all other sorts of library searching.
> I think people here do not know about $ORIGIN. This allows to define
> relative rpaths. E.g., a package with a program foo and a library
> libbar.so where foo is installed in $PATH/bin and libbar.so is defined
> in $PATH/lib should use
> -rpath \$ORIGIN/../lib
> The $ORIGIN is defined relative to the location of the object
> containing the reference.
This is a great feature, but I think it is hardly usable by libtool,
since, in order to use it, libtool would have to know at program link
time where the program is going to be installed (it currently doesn't
need this information), and it would have to check whether the libtool
libraries that the program is linked with are going to be installed in
paths that are easily accessible via \$ORIGIN relative pathnames, and
that no soft-linking (say, /bin -> /usr/bin but not /lib -> /usr/lib)
is going to break it, and probably many other potential problems that
I can't foresee.
But I agree, it's a nice feature, and we may be able to adopt it in
the future, on platforms that support it.
Alexandre Oliva
> Actually you want to know why I remember this? I used libtool a while back
> and I installed a copy of my program in /usr/bin and /usr/lib and wanted
> to us a new local copy of my libtool program. Of course libtool had used
> -rpath to make sure that my local binary used /usr/lib (as it was intended
> to be installed there someday) and then used LD_LIBRARY_PATH in the
> wrapper script to try to override this. Needless to say it did not work
> and it took me a bit of figuring to determine why my changes had no
> effect. Even in an all libtool environment rpath causes pain.
This is a known bug in the current libtool, and we're working on a
fix.
>> I have already told you one (ugly) way to do it, but I still don't
>> think it is a good idea in the general case.
> Didn't we decide that all of the available alternatives that you have
> suggested are not a feasable solution (does this mail help make it clear
> why)?
You may have missed the ugly one I was referring to, that I suggested
in the very beginning of this discussion, that would work even for
packages that were distributed with older versions of libtool:
configure the packages to use a gcc or ld wrapper that removes
switches such as -rpath /usr/lib from the command line then call the
appropriate program.
This will have the extra benefit of fixing other packages that don't
use libtool, but happen to specify -rpath on their own.
> - rpath is good because it allows a binary to have a shared library
> in a non-standard place without requiring the user to use
> LD_LIBRARYPATH or the sysadmin to add that place to the search
> path
> - rpath is bad because it disables LD_LIBRARYPATH
It does not disable it, it just precedes LD_LIBRARY_PATH. AFAIK, the
purpose of LD_LIBRARY_PATH is to help a program find a library that
was moved, and it does fulfil this purpose as long as you don't
install another (in)compatible library in place of the moved library.
> - rpath is bad because it disables the linkers automatic versioning
> mechanism
Does it? You mean, that hack in ld.so that adds /usr/lib/libc5 to the
library search path in certain circumstances? The hack is incomplete,
you just have to fix it.
> - rpath is bad because it prevents you from moving shared libraries
> around freely.
It does not. It just prevents you from arbitrarily replacing a
library with an (in)compatible version of it. Since you shouldn't do
that, libtool is not the piece of software to be blamed for using
-rpath.
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
Alexandre Oliva
> You say the contract is "I want to find THERE the lib that does THIS.x
> and THAT.x"; I think (and that's at least true for Linux) the contract
> the compiler and linker has signed was twofold; it says:
> 1) "I will give you the library that makes THIS.x and THAT.x"
> 2) "The prefered library I want you to use to obtain THIS and THAT
> is there and makes THIS.x and THAT.x"
> Now you trick it with -rpath in finding both the ".x" and THERE and all
> the problem comes from there...
> An analogy: When I wand to get hot water in restrooms, I just look at
> the tap, and turn the red one; I do not INSIST on opening the left one,
> risking to get cold water...
Good analogy. What's happening here is that Debian is placing the red
lable on the cold water tap. I.e., they're replacing a library with
an incompatible version of it, and getting in trouble because some
programs are now getting cold water where they expected hot water.
> Under Linux at least the incompatibilities we are talking here ARE
> managed by the dynamic linker, IFF we do not trick it saying him (using
> -rpath) "Do not be smart, just load the library from there". YOU are
> breaking the Linux contract...
ld.so is trying to outsmart everybody, but it is not smart enough to
do it. When you moved libc5-compatible libraries from /usr/lib to
/usr/lib/libc5, you established a rule that, if any program was linked
with libc5, it should look for libraries in /usr/lib/libc5 first,
right? Then why doesn't ld.so follow this rule, by replacing /usr/lib
with /usr/lib/libc5 when it finds this directory in the rpath too?
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
Richard Braakman
> ld.so is trying to outsmart everybody, but it is not smart enough to
> do it. When you moved libc5-compatible libraries from /usr/lib to
> /usr/lib/libc5, you established a rule that, if any program was linked
> with libc5, it should look for libraries in /usr/lib/libc5 first,
> right? Then why doesn't ld.so follow this rule, by replacing /usr/lib
> with /usr/lib/libc5 when it finds this directory in the rpath too?
No, that's not how it works. To the best of my understanding, it works
by adding a "libc5 or libc6" field to its cache. When it looks for
a cached library, and it finds two entries, it picks the one with the
correct libc. It always searches all of its directories.
It allows -rpath and LD_LIBRARY_PATH to override this behaviour.
I think that that is correct -- these _are_ overrides. They're
to be used when the default behaviour gets things wrong.
I think the dynamic linker could be further changed to always ignore a
library that would introduce a mixed libc5/libc6 linkage. That would
give the correct behaviour even with these overrides.
However, that only solves the _previous_ problem, not any future ones.
A general solution would require that "soname" be split into a library
name and a major version, so that the dynamic linker can detect
incompatible versions of the same library. That would be a major change.
Richard Braakman
Alexandre Oliva
> Alexandre Oliva wrote:
>> ld.so is trying to outsmart everybody, but it is not smart enough to
>> do it. When you moved libc5-compatible libraries from /usr/lib to
>> /usr/lib/libc5, you established a rule that, if any program was linked
>> with libc5, it should look for libraries in /usr/lib/libc5 first,
>> right? Then why doesn't ld.so follow this rule, by replacing /usr/lib
>> with /usr/lib/libc5 when it finds this directory in the rpath too?
> No, that's not how it works. To the best of my understanding, it works
> by adding a "libc5 or libc6" field to its cache. When it looks for
> a cached library, and it finds two entries, it picks the one with the
> correct libc. It always searches all of its directories.
> It allows -rpath and LD_LIBRARY_PATH to override this behaviour.
> I think that that is correct -- these _are_ overrides. They're
> to be used when the default behaviour gets things wrong.
Since -rpath is specified at program link time, and libc5 is supposed
to be used only by old programs, it would be correct for ld.so to use
/usr/lib/libc5 instead of /usr/lib if it finds that the program was
linked with libc5. This would make the transition as transparent as
possible, even for users that, inadvertently or not, have decided to
use -rpath /usr/lib for their programs.
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
Hamish Moffatt
> Does it? You mean, that hack in ld.so that adds /usr/lib/libc5 to the
> library search path in certain circumstances? The hack is incomplete,
> you just have to fix it.
Have you checked our ld.so source? The only mentioned of "libc5-compat"
is documentation.
Hamish
--
Hamish Moffatt VK3TYD ham...@debian.org, ham...@rising.com.au
Latest Debian packages at ftp://ftp.rising.com.au/pub/hamish. PGP#EFA6B9D5
CCs of replies from mailing lists are welcome. http://hamish.home.ml.org
Hamish Moffatt
> On Jan 27, 1999, Jules Bean <jm...@hermes.cam.ac.uk> wrote:
>
> > Therefore, we chose to solve that particular problem (the libc5-6
> > transition) by moving libraries around, knowing that our linker was up to
> > the job.
>
> It is now clear that it is not. :-(
It IS, as long as you don't use rpath. We don't use rpath for vendor-supplied
parts of the system. The user is obviously free to use them for locally
compiled stuff, and AFAIK it will behave as advertised. Yes, when Debian
moves those libraries in the future, those programs will break. The user
shouldn't really use rpath. But there are plenty of other ways for a user
to hose their system; we really can't stop them doing it.
> If it worked well you wouldn't be complaining about a problem that is
> caused by its incomplete behavior, but that could be avoided by
> modifying other pieces of software that are doing their jobs
> correctly.
Modifying libtool to remove -rpath fixes the problem at our end.
The documentation for our package checker (lintian) includes two
ways to do this easily.
Bernard Dautrevaux
> > I think that is correct -- these _are_ overrides. They're
> > to be used when the default behavior gets things wrong.
>
> Since -rpath is specified at program link time, and libc5 is supposed
> to be used only by old programs, it would be correct for ld.so to use
> /usr/lib/libc5 instead of /usr/lib if it finds that the program was
> linked with libc5. This would make the transition as transparent as
> possible, even for users that, inadvertently or not, have decided to
> use -rpath /usr/lib for their programs.
>
That sounds VERY sensible, although it's breaking the whole semantics of
-rpath... Anyway I'm now a bit confused: please could someone more
familiar than me with the *exact* way ld.so resolves library clarify
it's search rules:
If -rpath introduce some non-standard library (like -rpath
/home/foo/bar/lib) what happens? What I understood a while ago was the
following:
If the program needs some libfoobar.something, and it finds
libfoobar.so.5 in /home/foo/bar/lib, it will use it. period.
If the program needs some libfoobar.something, and it does not find it
in /home/foo/bar/lib, it looks in ld.so.cache, and select the one that
fit depending on dependants (like libc5 Vs libc6)
So far so good for me... (I expect the user to be a bit careful about
dependencies between packages it explicitly install on its system, that
is he do not install executable foo linked against libc5 that needs
library bar, that was build against libc6).
BUT it seems that if I also give -rpath /usr/lib, then ld.so will no
more check dependencies and then will load the libfoobar.so.whatever he
find in /usr/lib even if dependencies are broken
If that's true, I must admit that ld.so should at least warn about
dependencies clashes; but nevertheless libtool is meant to be able to
use non perfect systems, so libtool must avoid using -rpath in this
case.
Please, to try to clear up this thread, could someone familiar with
ld.so say exactly what happens on Linux (and possibly on other platforms
also).
Regards,
Bernard
--------------------------------------------
Bernard Dautrevaux
Microprocess Ingéniérie
97 bis, rue de Colombes
92400 COURBEVOIE
FRANCE
Tel: +33 (0) 1 47 68 80 80
Fax: +33 (0) 1 47 88 97 85
e-mail: dautr...@microprocess.com
b.daut...@usa.net
--------------------------------------------
Jason Gunthorpe
On 29 Jan 1999, Alexandre Oliva wrote:
> > Didn't we decide that all of the available alternatives that you have
> > suggested are not a feasable solution (does this mail help make it clear
> > why)?
>
> You may have missed the ugly one I was referring to, that I suggested
> in the very beginning of this discussion, that would work even for
> packages that were distributed with older versions of libtool:
> configure the packages to use a gcc or ld wrapper that removes
> switches such as -rpath /usr/lib from the command line then call the
> appropriate program.
So you agree that we should be able to choose to disable rpath but you
feel that we should do extra work to advoid it for libtool because it does
not fit your beliefs of how shared libraries work? What about other dists
that do the same thing? We are not the only linux dist to use this scheme!
> This will have the extra benefit of fixing other packages that don't
> use libtool, but happen to specify -rpath on their own.
Actually virtually every other package we would just hand edit the
makefiles, libtool kinda makes that impossible.
> > - rpath is good because it allows a biary to have a shared library
> > in a non-standard place without requiring the user to use
> > LD_LIBRARYPATH or the sysadmin to add that place to the search
> > path
> > - rpath is bad because it disables LD_LIBRARYPATH
>
> It does not disable it, it just precedes LD_LIBRARY_PATH. AFAIK, the
> purpose of LD_LIBRARY_PATH is to help a program find a library that
> was moved, and it does fulfil this purpose as long as you don't
> install another (in)compatible library in place of the moved library.
It prevents you from using LD_LIBRARY_PATH to superceed the default
location of libraries, which is partially what it does - rpath prevents
library searching and thus kills this functionality.
> > - rpath is bad because it disables the linkers automatic versioning >
> mechanism
>
> Does it? You mean, that hack in ld.so that adds /usr/lib/libc5 to the
> library search path in certain circumstances? The hack is incomplete,
> you just have to fix it.
See the other messages - it is not a hack and it is not horribly
incomplete.
Jason
Gordon Matzigkeit
[Creeeeeaaak... Gordon pops out of the grave reserved for former
libtool maintainers to make some comments.]
>>>>> Alexandre Oliva writes:
>> I don't understand this comment. Which "trouble" with "--rpath" do
>> you mean?
AO> The exact problem the Debian developers have been complaining
AO> about. The more I think about the problem, the more I see that
AO> the problem they're facing is an incomplete hack of ld.so, in
AO> that it modifies the library search path under certain
AO> circumstances, but not in all circumstances that would need it.
Exactly.
>> This means, the package can provide a default, which can be
>> overridden at compilation time. Finally, the installer can
>> override both.
AO> That's exactly what I'm looking for. But I wouldn't like to
AO> install a quick hack now that would later reveal to be a step in
AO> the wrong direction. That's why I'm being so conservative about
AO> all this issue.
For the record, Alexandre's conservativeness is well-justified.
Debian maintainers should feel free to patch individual Debian
packages, but fixing this problem upstream is a lot harder than it
appears at first glance.
The best solution I can come up with is to *always* change a library's
soname when its dependencies change. I believe it was Joel Klecker
who mentioned something about `libapi' patches for egcs that were
supposed to implement this automatically.
Joel, can you comment on this (or somebody else who knows the details)?
--
Gordon Matzigkeit <go...@fig.org> //\ I'm a FIG (http://www.fig.org/)
Lovers of freedom, unite! \// I use GNU (http://www.gnu.org/)
[Unfortunately, www.fig.org is broken. Please stay tuned for details.]
Steve Dunham
> On Fri, Jan 29, 1999 at 07:27:28AM -0200, Alexandre Oliva wrote:
> > Does it? You mean, that hack in ld.so that adds /usr/lib/libc5 to the
> > library search path in certain circumstances? The hack is incomplete,
> > you just have to fix it.
> Have you checked our ld.so source? The only mentioned of "libc5-compat"
> is documentation.
It's a magic hack. Somehow (according to Alexandre) it magically adds
/usr/lib/libc5 to the path, even though "libc5" occurs nowhere in the
binary. (go figure.)
Maybe we should just agree that libtool is broken, that it won't be
fixed upstream, and just fix the Debian version? This would mean
that we would have to rerun autoconf &co when we build packages -
but it beats arguing with this guy till the end of time.
Steve
dun...@cse.msu.edu
Joel Klecker
>The best solution I can come up with is to *always* change a library's
>soname when its dependencies change. I believe it was Joel Klecker
>who mentioned something about `libapi' patches for egcs that were
>supposed to implement this automatically.
>
>Joel, can you comment on this (or somebody else who knows the details)?
That patch merely applies to the soname of the libstdc++ library that
is part of egcs, it imparts no other functionality.
--
Joel Klecker (aka Espy) <URL:http://web.espy.org/>
<URL:mailto:j...@espy.org> <URL:mailto:es...@debian.org>
Debian GNU/Linux PowerPC -- <URL:http://www.debian.org/ports/powerpc/>
Alexandre Oliva
>> > Didn't we decide that all of the available alternatives that you have
>> > suggested are not a feasable solution (does this mail help make it clear
>> > why)?
> On 29 Jan 1999, Alexandre Oliva wrote:
>> You may have missed the ugly one I was referring to, that I suggested
>> in the very beginning of this discussion, that would work even for
>> packages that were distributed with older versions of libtool:
>> configure the packages to use a gcc or ld wrapper that removes
>> switches such as -rpath /usr/lib from the command line then call the
>> appropriate program.
> So you agree that we should be able to choose to disable rpath but you
> feel that we should do extra work to advoid it for libtool because it does
> not fit your beliefs of how shared libraries work? What about other dists
> that do the same thing? We are not the only linux dist to use this scheme!
I agree that libtool may, some day, allow users to do that in a
portable fashion. But I still don't see how modifying the current
version of libtool would help you fix the problem of backward
compatibility for already compiled programs or for packages
distributed with older versions of libtool. Therefore, the fix you
really need is not in libtool, is in the dynamic loader.
>> This will have the extra benefit of fixing other packages that don't
>> use libtool, but happen to specify -rpath on their own.
> Actually virtually every other package we would just hand edit the
> makefiles, libtool kinda makes that impossible.
Yep. Now you have to edit a single libtool script, instead of all the
Makefiles of the package. Ain't tha progress?
>> > - rpath is bad because it disables LD_LIBRARYPATH
> It prevents you from using LD_LIBRARY_PATH to superceed the default
> location of libraries, which is partially what it does
Not according to the Posix specification.
> rpath prevents library searching and thus kills this functionality.
It doesn't prevent library searching, it just takes precedence over
it. If the library is not found in the rpath specified at link-time,
the library is searched in other directories, such as the ones
specified in LD_LIBRARY_PATH.
>> > - rpath is bad because it disables the linkers automatic versioning >
>> mechanism
>> Does it? You mean, that hack in ld.so that adds /usr/lib/libc5 to the
>> library search path in certain circumstances? The hack is incomplete,
>> you just have to fix it.
> See the other messages - it is not a hack and it is not horribly
> incomplete.
It doesn't work for applications that have chosen to hard-code
/usr/lib or /usr/lib/X11R6 in their code, for whatever reason,
therefore I can only see two possible conclusions:
1) your choice to move libraries around was a bad idea, because it
causes certain applications to break
2) the code in the dynamic loader that chooses the `right' version of
a library is incomplete, in the sense that it doesn't choose the
`right' version when shared library paths are hard-coded in the
application
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
Alexandre Oliva
> On Fri, Jan 29, 1999 at 07:11:54AM -0200, Alexandre Oliva wrote:
>> On Jan 27, 1999, Jules Bean <jm...@hermes.cam.ac.uk> wrote:
>>
>> > Therefore, we chose to solve that particular problem (the libc5-6
>> > transition) by moving libraries around, knowing that our linker was up to
>> > the job.
>>
>> It is now clear that it is not. :-(
> It IS, as long as you don't use rpath.
And libtool works perfectly well, as long as you don't replace
libraries with incompatible versions. What makes you think the
maintainers of libtool should introduce potential problems and break
backward compatibility just to fix a (IMHO) bad choice you have made,
even knowing that modifying libtool at this point would not contribute
in *any* way to fix the problem you currently have?
> The user is obviously free to use them for locally compiled stuff,
> and AFAIK it will behave as advertised. Yes, when Debian moves those
> libraries in the future, those programs will break. The user
> shouldn't really use rpath.
Maybe you should ask distributors of Debian to print this advice in
any CD-ROM containing Debian distributions they sell.
> But there are plenty of other ways for a user to hose their system;
> we really can't stop them doing it.
That's not the point. By replacing libraries with (in)compatible
versions, you're actively working to hose users' systems.
> Modifying libtool to remove -rpath fixes the problem at our end.
Nope, because the current problem has to do with pre-installed
programs. Modifying the libtool script will, by no means, fix this
problem.
Alexandre Oliva
> On Fri, Jan 29, 1999 at 07:27:28AM -0200, Alexandre Oliva wrote:
>> Does it? You mean, that hack in ld.so that adds /usr/lib/libc5 to the
>> library search path in certain circumstances? The hack is incomplete,
>> you just have to fix it.
> Have you checked our ld.so source? The only mentioned of "libc5-compat"
> is documentation.
Nope, I'm just believing what the people that have complained have
told me about it.
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
Marcus Brinkmann
On Fri, Jan 29, 1999 at 03:41:46PM -0600, Gordon Matzigkeit wrote:
>
> >> I don't understand this comment. Which "trouble" with "--rpath" do
> >> you mean?
>
> AO> The exact problem the Debian developers have been complaining
> AO> about. The more I think about the problem, the more I see that
> AO> the problem they're facing is an incomplete hack of ld.so, in
> AO> that it modifies the library search path under certain
> AO> circumstances, but not in all circumstances that would need it.
>
> Exactly.
Mmh. I think I see the point now, and I have to agree.
So, the problem we are facing is twofold:
* How can Debian hack around the rpath problem, so user can use third party
software which is libc5+rpath.
* Is there a better way to do a library transition? I think it is very
obvious, that the only correct behaviour is to change the
library/soname of all involeved libraries when doing a transition.
So we had to move either from libfoo.2 to libfoog.2, libfoo.libc6.2,
libc6/libfoo.2 or whatever, until a new library with a new, incompatible,
soname is released. Changing the name does not look correct, so we had
to change soname or path.
I am sorry if this sounds quite confusing, but I hope you get the idea
(Gordon, this is exactly what we discussed on the Hurd list, right?).
Fortunately, libc6 will use symbol versioning, so we will not experience
problems at this scale again (hopefully).
Furthermore, it would be great if upstream author, compiler and installer
could all influence the linking conditions (rpath or not etc), which seems
to be a good goal. This could fix our problem, too, but only as a hack. The
theoretical solution is outlined above.
There is still the issue of xaw wrappers, can you please comment on this?
Shouldn't there be a way to override rpath? Currently,LD_LIBRARY_PATH does
override rpath, right? But then, if I want to link a program which was
compiled for xaw with xaw3d, to get a better look+feel, I would need a way
to override the rpath inside the binary. Maybe another environment variable
is needed for this, or should the priority changed?
Because, I really think the sysadmin/user should always have the last word
on this issue. Currently, rpath overrides everything, which is set by the
distributor of the binary.
> >> This means, the package can provide a default, which can be
> >> overridden at compilation time. Finally, the installer can
> >> override both.
I should add here that ideally, a user/sysadmin should always be able to
override all three.
> AO> That's exactly what I'm looking for. But I wouldn't like to
> AO> install a quick hack now that would later reveal to be a step in
> AO> the wrong direction. That's why I'm being so conservative about
> AO> all this issue.
>
> For the record, Alexandre's conservativeness is well-justified.
I still don't see if libtools default, rpath, is correct, but I see now what
Debian did wrong. I also see that if Debian would have done it correctly,
the use of rpath would be a non-issue.
> The best solution I can come up with is to *always* change a library's
> soname when its dependencies change.
Ah, here you say it what I cludged together above with my limited
understanding of the issues :)
Thanks,
Marcus
--
"Rhubarb is no Egyptian god." Debian GNU/Linux finger brinkmd@
Marcus Brinkmann http://www.debian.org master.debian.org
Marcus.B...@ruhr-uni-bochum.de for public PGP Key
http://homepage.ruhr-uni-bochum.de/Marcus.Brinkmann/ PGP Key ID 36E7CD09
Marcus Brinkmann
> On Jan 29, 1999, Jason Gunthorpe <j...@ualberta.ca> wrote:
>
> > rpath prevents library searching and thus kills this functionality.
>
> It doesn't prevent library searching, it just takes precedence over
> it. If the library is not found in the rpath specified at link-time,
> the library is searched in other directories, such as the ones
> specified in LD_LIBRARY_PATH.
I think a way to override even rpath would be great.
> It doesn't work for applications that have chosen to hard-code
> /usr/lib or /usr/lib/X11R6 in their code, for whatever reason,
> therefore I can only see two possible conclusions:
>
> 1) your choice to move libraries around was a bad idea, because it
> causes certain applications to break
>
> 2) the code in the dynamic loader that chooses the `right' version of
> a library is incomplete, in the sense that it doesn't choose the
> `right' version when shared library paths are hard-coded in the
> application
Why should the application choose to hard code the PATH in the binary?
AFAICS, there is no apparent reason for it. What has the path to do with the
library? I think the only thing that should be hard coded is the exact
soname and library name. Maybe I am missing something?
Ian Lance Taylor
From: Marcus Brinkmann <Marcus.B...@ruhr-uni-bochum.de>
Why should the application choose to hard code the PATH in the binary?
AFAICS, there is no apparent reason for it. What has the path to do with the
library? I think the only thing that should be hard coded is the exact
soname and library name. Maybe I am missing something?
Suppose you have your own set of shared libraries, in your own
directory. Suppose you want to let other people use your programs
linked against your own shared libraries. You can tell everyone who
uses your programs to set LD_LIBRARY_PATH, or you can simply use
-rpath so that your programs can always find your shared libraries.
In general, it's convenient to store the path in the executable any
time a shared library is installed in a directory which the dynamic
linker does not search by default.
Incidentally, I don't know what you mean by saying both soname and
library name. There is only one name recorded for a shared library:
the soname.
Ian
Marcus Brinkmann
>
> Suppose you have your own set of shared libraries, in your own
> directory. Suppose you want to let other people use your programs
> linked against your own shared libraries. You can tell everyone who
> uses your programs to set LD_LIBRARY_PATH, or you can simply use
> -rpath so that your programs can always find your shared libraries.
Okay.
> In general, it's convenient to store the path in the executable any
> time a shared library is installed in a directory which the dynamic
> linker does not search by default.
Yes, I should have narrowed my question to system libraries. Unfortunately,
system libraries are most likely to cause the problems, for example if
people hard code /usr/X11R6/lib/ for xaw library and you want to use
xaw3d...
> Incidentally, I don't know what you mean by saying both soname and
> library name. There is only one name recorded for a shared library:
> the soname.
Ignorance I think. I thought soname is only the number, and a lib is stored
in libfoo.x.y.z, where foo is the library name and x.y.z the soname. If I
got it wrong, I apologize.
Marcus
--
"Rhubarb is no Egyptian god." Debian GNU/Linux finger brinkmd@
Marcus Brinkmann http://www.debian.org master.debian.org
Marcus.B...@ruhr-uni-bochum.de for public PGP Key
http://homepage.ruhr-uni-bochum.de/Marcus.Brinkmann/ PGP Key ID 36E7CD09
Ian Lance Taylor
From: Marcus Brinkmann <Marcus.B...@ruhr-uni-bochum.de>
* Is there a better way to do a library transition? I think it is very
obvious, that the only correct behaviour is to change the
library/soname of all involeved libraries when doing a transition.
So we had to move either from libfoo.2 to libfoog.2, libfoo.libc6.2,
libc6/libfoo.2 or whatever, until a new library with a new, incompatible,
soname is released. Changing the name does not look correct, so we had
to change soname or path.
When you make an incompatible change to a shared library, change the
soname. That's just a matter of choosing a name for the new library
and tweaking a symlink. There is no reason to do anything else.
What do you mean when you say ``changing the name does not look
correct?''
Don't confuse the name of the library found by the runtime linker
(libc.so) with the soname used by the dynamic linker (libc.so.6). The
runtime linker will normally find a symlink to the soname. When you
build a shared library, use the -h/--soname option to set the soname
of the shared library.
Shouldn't there be a way to override rpath? Currently,LD_LIBRARY_PATH does
override rpath, right?
No, LD_LIBRARY_PATH does not override rpath. The rpath is searched
first, and then the LD_LIBRARY_PATH is searched. I think you may have
agreed with that later in your message.
Ian
Marcus Brinkmann
>
> * Is there a better way to do a library transition? I think it is very
> obvious, that the only correct behaviour is to change the
> library/soname of all involeved libraries when doing a transition.
> So we had to move either from libfoo.2 to libfoog.2, libfoo.libc6.2,
> libc6/libfoo.2 or whatever, until a new library with a new, incompatible,
> soname is released. Changing the name does not look correct, so we had
> to change soname or path.
>
> When you make an incompatible change to a shared library, change the
> soname. That's just a matter of choosing a name for the new library
> and tweaking a symlink. There is no reason to do anything else.
Yes, this is what I meant. Debian should have changed the sonames of the
libc6 libraries when the library exists for libc5, too.
> What do you mean when you say ``changing the name does not look
> correct?''
Well, you _could_ rename the library, and recompile applications using the
new name... for a transition period... OTOH I realized that this would be
very ugly and require changes to Makefiles etc... so it "does not look
correct" == "is a stupid and brain dead idea".
> Shouldn't there be a way to override rpath? Currently,LD_LIBRARY_PATH does
> override rpath, right?
>
> No, LD_LIBRARY_PATH does not override rpath. The rpath is searched
> first, and then the LD_LIBRARY_PATH is searched. I think you may have
> agreed with that later in your message.
Sorry about the typo. I meant to say "does not override rpath".
Thanks,
Marcus
--
"Rhubarb is no Egyptian god." Debian GNU/Linux finger brinkmd@
Marcus Brinkmann http://www.debian.org master.debian.org
Marcus.B...@ruhr-uni-bochum.de for public PGP Key
http://homepage.ruhr-uni-bochum.de/Marcus.Brinkmann/ PGP Key ID 36E7CD09
Manish Singh
> Shouldn't there be a way to override rpath? Currently,LD_LIBRARY_PATH does
> override rpath, right?
>
> No, LD_LIBRARY_PATH does not override rpath. The rpath is searched
> first, and then the LD_LIBRARY_PATH is searched. I think you may have
> agreed with that later in your message.
This is another irksome thing about libtool and -rpath. Test programs,
even if they are marked noinst, use -rpath, and when they are run using
the installed version instead of the newly built one. Which is annoying,
since the whole point of test programs is to make sure the library is
working *before* you install it.
So maybe we should have an explicit -no-rpath switch to libtool to fix
this.
-Yosh
Ian Lance Taylor
From: Marcus Brinkmann <Marcus.B...@ruhr-uni-bochum.de>
> In general, it's convenient to store the path in the executable any
> time a shared library is installed in a directory which the dynamic
> linker does not search by default.
Yes, I should have narrowed my question to system libraries. Unfortunately,
system libraries are most likely to cause the problems, for example if
people hard code /usr/X11R6/lib/ for xaw library and you want to use
xaw3d...
It's hard to distinguish system libraries from non-system libraries,
except by the distinction quoted above: a non-system library is a
library installed in a directory which the dynamic linker does not
search by default.
Unfortunately, the GNU/Linux dynamic linker reportedly uses a rather
complex algorithm, in which it makes decisions based on the libc
version number which libraries are linked against, which would seem to
make it hard to determine just which directories the dynamic linker
searches by default.
In the normal case I think one can assume that the dynamic linker will
search any directory listed in /etc/ld.so.conf, and it would be OK to
omit a -rpath argument for any shared library installed in one of the
directories listed in that file.
Note that although you can set up a case in which the xaw library is
found in /usr/X11R6/lib, it's not normal. Normally the program will
be linked against libxaw.so.N, and will have a specified search path,
the rpath, to find that file.
> Incidentally, I don't know what you mean by saying both soname and
> library name. There is only one name recorded for a shared library:
> the soname.
Ignorance I think. I thought soname is only the number, and a lib is stored
in libfoo.x.y.z, where foo is the library name and x.y.z the soname. If I
got it wrong, I apologize.
When I, and I think most others, use the word soname, I refer to the
DT_SONAME tag in a shared library which appears in a DT_NEEDED tag in
the executable. The soname is set in a shared library using the
-h/--soname option, and is copied into the executable by the program
linker. In this case the soname is the full name of the file:
libfoo.x.y.z.
Ian
Alexandre Oliva
> In the normal case I think one can assume that the dynamic linker will
> search any directory listed in /etc/ld.so.conf, and it would be OK to
> omit a -rpath argument for any shared library installed in one of the
> directories listed in that file.
This seems to be a reasonable assumption, as long as no directory is
ever removed from /etc/ld.so.conf. But then, there's also the problem
of finding the wrong version of a library, if it is found before the
correct one in the search path. Anyway, as Thomas Tanner argued,
hard-coding rpath won't always solve this problem either (although it
would solve it in some cases), so I'd welcome a patch that would cause
libtool to:
1) use information from /etc/ld.so.conf, instead of having a
hard-coded list of directories, to set sys_lib_search_path_spec
2) refrain from hard-coding directories listed in
sys_lib_search_path_spec (but not in $shlibpath_var) in executables
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
Raja R Harinath
> On Fri, Jan 29, 1999 at 03:41:46PM -0600, Gordon Matzigkeit wrote:
> > >> I don't understand this comment. Which "trouble" with "--rpath" do
> > >> you mean?
> >
> > AO> The exact problem the Debian developers have been complaining
> > AO> about. The more I think about the problem, the more I see that
> > AO> the problem they're facing is an incomplete hack of ld.so, in
> > AO> that it modifies the library search path under certain
> > AO> circumstances, but not in all circumstances that would need it.
> >
> > Exactly.
>
> Mmh. I think I see the point now, and I have to agree.
> So, the problem we are facing is twofold:
>
> * How can Debian hack around the rpath problem, so user can use third party
> software which is libc5+rpath.
>
> obvious, that the only correct behaviour is to change the
> library/soname of all involeved libraries when doing a transition.
> So we had to move either from libfoo.2 to libfoog.2, libfoo.libc6.2,
> libc6/libfoo.2 or whatever, until a new library with a new, incompatible,
> soname is released. Changing the name does not look correct, so we had
> to change soname or path.
Caveat: I'm a novice at these issues.
IMHO (with 20/20 hindsight), it would have been much nicer if the
libc5->libc6 transition had used a different mechanism -- something
akin to $_RLD_ROOT of IRIX.
The idea of _RLD_ROOT is that if that env. variable is set, it is
prepended to every runpath in the binary + every default path.
E.g. if _RLD_ROOT="/mnt1:/mnt2:", and you have a binary with -rpath
set to "/usr/foo/lib" and, the default library search path is
"/lib:/usr/lib". The set of paths searched by the linker are, in
order:
/mnt1/usr/foo/lib
/mnt2/usr/foo/lib
/usr/foo/lib
$LD_LIBRARY_PATH
/mnt1/lib
/mnt2/lib
/lib
/mnt1/usr/lib
/mnt2/usr/lib
/usr/lib
(I may have got the specifics wrong, but this should be the general
idea.)
The "better" way for libc5/6 hack would have been to modify
ld-linux.so.1 (the libc5 ELF loader) to act _as if_ the _RLD_ROOT
env. var. was set to `/usr/compat-glibc1:'. This way, the libc5
libraries could have been moved into to the /usr/compat-glibc1 tree
maintaining the same tree structure, and would automatically have
worked with any `-rpath's.
E.g., xlib6 (the libc5 compatible X library) could have put its
libraries in
/usr/compat-glibc1/usr/X11R6/lib
(if it originally put it in `/usr/X11R6/lib') and libc5 could have put
its library either in /lib or in /usr/compat-glibc1/lib.
Moving from `bo' to `hamm' for libc5 libraries could just have been a
matter of dpkg-repack or some similar tool.
Of course, I'm not conversant with all the issues, and could be
completely wrong.
- Hari
--
Raja R Harinath ------------------------------ hari...@cs.umn.edu
"When all else fails, read the instructions." -- Cahn's Axiom
"Our policy is, when in doubt, do the right thing." -- Roy L Ash
Alexandre Oliva
>> No, LD_LIBRARY_PATH does not override rpath. The rpath is searched
>> first, and then the LD_LIBRARY_PATH is searched. I think you may have
>> agreed with that later in your message.
> This is another irksome thing about libtool and -rpath. Test programs,
> even if they are marked noinst, use -rpath, and when they are run using
> the installed version instead of the newly built one. Which is annoying,
> since the whole point of test programs is to make sure the library is
> working *before* you install it.
Yep, this is a known bug in libtool, and a partial fix for it, by
Edouard Parmelan, is already installed in the CVS tree. Hopefully,
someone will soon be able to complete his work, based on the short
description of what is missing he recently provided.
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
Jim Pick
Ian Lance Taylor <i...@cygnus.com> writes:
> Date: Sat, 30 Jan 1999 23:30:43 +0100
> From: Marcus Brinkmann <Marcus.B...@ruhr-uni-bochum.de>
>
> Why should the application choose to hard code the PATH in the binary?
> AFAICS, there is no apparent reason for it. What has the path to do with the
> library? I think the only thing that should be hard coded is the exact
> soname and library name. Maybe I am missing something?
>
> Suppose you have your own set of shared libraries, in your own
> directory. Suppose you want to let other people use your programs
> linked against your own shared libraries. You can tell everyone who
> uses your programs to set LD_LIBRARY_PATH, or you can simply use
> -rpath so that your programs can always find your shared libraries.
Yes, I completely agree.
For example, I want to use -rpath for the Debian snapshots stuff,
which is going to get installed under /opt/snapshots. Using -rpath is
very nice, because the libraries can then use the snapshot version of
gtk, glib, or gnome-libs (also installed under /opt/snapshots) instead
of the standard Debian libs. This way, the snapshot packages can be
kept completely independent from the regular Debian packages.
So -rpath is still very useful in some circumstances - where you don't
want the users to have to set LD_LIBRARY_PATH.
Cheers,
- Jim
John Goerzen
> You might have included my suggestion to prevent having to move
> libraries in the first place: creating a libc6-specific directory
> right now, instead of installing libraries in /usr/lib and having to
> move them into another directory when libc7 should be released.
So, in other words, you would have Debian violate FSSTND, FHS, and decades
or historical precedent simply because you are unwilling to fix brokenness
in libtool? I simply find this level of arrogance almost beyond
comprehension. To say that Debian and every other distribution that tries
to adhere to these standards (most do) is broken for trying to do so is
ludicrous.
> > point: -rpath is necessary if one is installing libraries and binaries
> > linked to those libraries in one's home directory, or if your Unix has
> > no support for library search paths via environment variables like
> > Linux's LD_LIBRARY_PATH.
But our Unix does have that support (no rpath necessary!)
> before): libtool will hard-code the installation directory of the
> library into the `libdir' variable of the .la script it installs.
> Therefore, if one moves the library then tries to link with the .la
> file, he loses. There's also the dlopening issue: libltdl (to be
How does one link with a script?
> released with libtool 1.3) will dlopen a library in the directory
> pointed to by `libdir' too.
Then how about fixing it?
> In general, I feel that moving libraries around is a very bad idea,
> because it will lead to failure most of the times, and that's why I
> don't feel libtool should help people doing that.
How come it only leads to failure when libtool is involved then?
> Actually, not issuing -rpath or equivalent is quite easy to do, but
> choosing *when* not to do it is the part that is hard to port. Thomas
Look, we're not picky. You put an option in there that Configure can pass
to it at compile time, that's fine.
> hardcode path at all? Or something else? What to do on a platform
> that doesn't support the requested hardcoding strategy?
Obviously you use what you have.
> The issue is very complex because we can't think just of GNU/Linux
> with all its bells and whistles, because libtool is supposed to
> present an homogeneous, portable interface to creating libraries.
Ohhh, I get it. We're supposed to ignore the functionality that we have
because others are still running on PDP-11s, right?
Of all the nerve, this has to take the cake. I have no problem with
maintaining compatibility, but when it comes to something like this, where
compatibility CAN be maintained while still allowing us to take advantage of
our features, to insist that we must ignore our features is outlandish.
John
John Goerzen
> > I'm sorry, but this is IMHO completely backwards. On Linux systems, there is
> > nothing wrong with moving libraries around as the need arises.
>
> Except that you risk replacing a library with an incompatible one.
> That's what has caused you so much trouble. If, instead of moving the
> X11 libraries to another dir and creating new, incompatible versions
> under the same pathname, you had created new versions in other
> directories, no unexpected crashes would have occurred.
OK, let's assume for a moment that we cripple Debian by ignoring the FHS in
this instance. Not all Linux distributions will make this choice. So
somebody on some other distribution compiles things with the pathname
hard-coded in. On his system, it is /usr/X11R6/lib for libraries. But his
program will not work on Debian, because we would have listened to you and
moved our current libraries to a nonstandard location.
If people use -rpath like you reccommend, the consequences are that
virtually all commercial Linux apps would be unusable on Debian. That
virtually all Linux binaries would be unusable on Debian.
I frankly am not going to accept this, and I fail to see why you think that
we ought to cripple our distribution so badly simply because you are
unwilling to add a simple command-line option to libtool to fix this
brokenness.
> and the program will be able to find the library at that point. If
> you move the library and replace it with an incompatible one, you're
> breaking the contract and the versioning mechanism, so you can't blame
> the program for crashing, nor the tool that created the program.
You're missing the point (I think; to which versioning mechanism are you
referring?). It's the same version of the library, designed to be linked
with different versions of other libraries.
We can have libncurses3.4 designed to be linked with libc5 or one for libc6.
> > The X example also shows that the problem isn't limited to /usr/lib either.
> > What's next? /usr/local/lib/libc6 ?
>
> Maybe some library versioning mechanism that allows incompatible
> changes to be marked as such. Maybe an environment variable or some
Ahh, so first you say that we must not take advantage of our unique
features, and now you suggest we add more? Will you condemn us for adding
these features now?
> You see the regular case as the one you use the most. I see it as the
> one I use the most. I don't install any packages in /usr or
> /usr/local because I find it *horrible* to have a huge /usr/local/bin
> without any clear separation between packages. It's a pity that the
> GNU/Linux distributions don't care about clearly separating packages
> from one another...
Five points about that:
#1. Your comments show a fundamental lack of understanding about filesystem
layout in Unix, the FSSTND, and the FHS.
#2. Debian does not install packages in /usr/local/bin.
#3. Debian does not install all binaries in /usr/bin.
#4. Ever heard of 'which'? 'locate'?
#5. You can easily ask a Debian system for information about which
files a package provides, or which package provides a given file.
> Because you break a contract every time you remove a library from the
> place in which it used to live.
The 'contract' never should have cared about its location in the first
place. The OS, through mechanisms like /etc/ld.so.conf, HAS THE RIGHT to
move it. By assuming that it does not, YOU are breaking that contract.
John Goerzen
Alexandre,
One would hope that you would learn a bit about Unix before criticizing it.
> > Our distribution cares greatly about separating packages from each other.
>
> Not from the point of view of the end user. When they want to find a
> tool, they may `ls /usr/bin /usr/local/bin' and will be presented
> *thousands* of files. I'd rather have a classification system in
If they want to find a tool, this is not the correct place anyway, because
it may not be installed.
Debian does categorize files by package affiliation. Packages are then
categorized by priority, license, general purpose (editors, games, etc), and
other factors (some at the users' discretion).
> which I could have links (or similar) to all programs in a common
> directory, but in which I could search for programs by subject. I'd
Debian already has such a mechanism, but one that is not limited to
already-installed software.
> like to have development tools such as compilers in one directory,
> text writing tools such as word processors and text editors in
> another, system administration utilities in another, and so on.
Ahh. So everyone's supposed to have a 16k PATH variable, eh?
> How does the current packaging system allows me to test one version of
> a package while other users of the same host are running a stable
> version of that tool? Or are the GNU/Linux distributions all moving
> towards the Micro$oft model of single-user workstations?
So, you tell me. How can you run one version of the kernel while others run
a different one? How about sendmail? You expect the system to somehow
magically know that an incoming SMTP connect will be delivering you mail,
and fire up your own "special" version? wu-ftpd? Apache? Frankly, that is
not practical on a multi-user system. You are the one asking us to move
towards the M$ model.
>
> --
> Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
> oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
> Universidade Estadual de Campinas, SP, Brasil
John Goerzen
> If you do want to be able to freely move libraries around, -rpath must
> be forbidden. If -rpath is available for users, you can't move
> libraries around and expect things to work.
So, because the user is clueless and uses a bad option, we should cripple
the system?
By that logic, we ought to remove the following commands:
* cat
(user could run cat /dev/zero > /dev/hda, messing up their HD in the near
future)
* dd
(user could run dd if=/dev/zero of=/dev/hda)
* fdisk
(user could mess up their partition table in the future)
* echo
(user could echo stuff to /dev/hda)
* bash
(has a built-in echo)
* gcc
(offers -rpath, which can mess up their executables in the future.
I simply don't follow. People should not just try stuff because "Gee, I
wonder what this will do?!" It is not our fault if sombody is curious and
incorrectly uses -rpath, just as it is not our fault if somebody is curious
and inadvertantly cat's /dev/zero over their hard disk.
> But isn't this exactly what the packaging systems are trying to avoid,
> i.e., that people have to compile systems on their own? And then, how
> could I make sure that my test build works exactly as the pre-compiled
> upgrade, so that I could use the packaging system for the update?
Oh, so you're saying that the admin controls the default versions for
people, then? Gee, sounds like what we have now.
John Goerzen
> > An analogy: When I wand to get hot water in restrooms, I just look at
> > the tap, and turn the red one; I do not INSIST on opening the left one,
> > risking to get cold water...
>
> Good analogy. What's happening here is that Debian is placing the red
> lable on the cold water tap. I.e., they're replacing a library with
> an incompatible version of it, and getting in trouble because some
> programs are now getting cold water where they expected hot water.
No, we're labeling it properly, you're just getting trouble because you have
purposely blinded yourself and are unable to read the label.
> right? Then why doesn't ld.so follow this rule, by replacing /usr/lib
> with /usr/lib/libc5 when it finds this directory in the rpath too?
Because rpath shouldn't be overridden, perhaps? Sorta defeats the purpose
of it, doesn't it? Makes us non-compliant. Introduces a bug!
Ian Lance Taylor
From: John Goerzen <jgoe...@complete.org>
On Wed, Jan 27, 1999 at 05:07:30PM -0200, Alexandre Oliva wrote:
> You might have included my suggestion to prevent having to move
> libraries in the first place: creating a libc6-specific directory
> right now, instead of installing libraries in /usr/lib and having to
> move them into another directory when libc7 should be released.
So, in other words, you would have Debian violate FSSTND, FHS, and decades
or historical precedent simply because you are unwilling to fix brokenness
in libtool? I simply find this level of arrogance almost beyond
comprehension. To say that Debian and every other distribution that tries
to adhere to these standards (most do) is broken for trying to do so is
ludicrous.
Wow, just when I thought we'd reached some degree of consensus.
``Decades of historical precedent?'' ``Arrogance almost beyond
comprehension?'' Are you interested in solving the problem, or are
you just talking?
Of all the nerve, this has to take the cake. I have no problem with
maintaining compatibility, but when it comes to something like this, where
compatibility CAN be maintained while still allowing us to take advantage of
our features, to insist that we must ignore our features is outlandish.
As far as I can tell, compatibility was broken with any program that
linked with -rpath and a system directory. libtool merely happens to
be one example.
Ian
Ian Lance Taylor
From: John Goerzen <jgoe...@complete.org>
On Wed, Jan 27, 1999 at 05:53:12PM -0200, Alexandre Oliva wrote:
> > I'm sorry, but this is IMHO completely backwards. On Linux systems, there is
> > nothing wrong with moving libraries around as the need arises.
>
> Except that you risk replacing a library with an incompatible one.
> That's what has caused you so much trouble. If, instead of moving the
> X11 libraries to another dir and creating new, incompatible versions
> under the same pathname, you had created new versions in other
> directories, no unexpected crashes would have occurred.
OK, let's assume for a moment that we cripple Debian by ignoring the FHS in
this instance. Not all Linux distributions will make this choice. So
somebody on some other distribution compiles things with the pathname
hard-coded in. On his system, it is /usr/X11R6/lib for libraries. But his
program will not work on Debian, because we would have listened to you and
moved our current libraries to a nonstandard location.
But if that person happened to be building on a libc5 system with the
hardcoded path, then Debian already broke his program. I agree that
moving libraries around is not a good idea--I think it's better to
change the soname as necessary. If Debian (and other distributions)
had not moved any libraries, nobody would ever have noticed this
problem.
> and the program will be able to find the library at that point. If
> you move the library and replace it with an incompatible one, you're
> breaking the contract and the versioning mechanism, so you can't blame
> the program for crashing, nor the tool that created the program.
You're missing the point (I think; to which versioning mechanism are you
referring?). It's the same version of the library, designed to be linked
with different versions of other libraries.
We can have libncurses3.4 designed to be linked with libc5 or one for libc6.
Those are different versions of the library. They have different
requirements. From the perspective of the dynamic linker, they can
not be considered to be the same version.
> Because you break a contract every time you remove a library from the
> place in which it used to live.
The 'contract' never should have cared about its location in the first
place. The OS, through mechanisms like /etc/ld.so.conf, HAS THE RIGHT to
move it. By assuming that it does not, YOU are breaking that contract.
I disagree. If the OS is gong to move the library, it is responsible
for making sure that old programs linked using -rpath continue to
work, one way or another. You are effectively saying that -rpath is
prohibited, which I do not think is reasonable.
Ian Lance Taylor
From: John Goerzen <jgoe...@complete.org>
> How does the current packaging system allows me to test one version of
> a package while other users of the same host are running a stable
> version of that tool? Or are the GNU/Linux distributions all moving
> towards the Micro$oft model of single-user workstations?
So, you tell me. How can you run one version of the kernel while others run
a different one? How about sendmail? You expect the system to somehow
magically know that an incoming SMTP connect will be delivering you mail,
and fire up your own "special" version? wu-ftpd? Apache? Frankly, that is
not practical on a multi-user system. You are the one asking us to move
towards the M$ model.
This is really a sidetrack, but it's pretty each to test specific
programs, although it's probably impossible on Linux to test a
specific kernel. To test sendmail, ftpd, or Apache, just run them on
a different port number, and arrange to connect to that port for
testing purposes. I did stuff like this all the time when I was
packaging and testing Kerberos releases.
John Goerzen
> OK, let's assume for a moment that we cripple Debian by ignoring the FHS in
> this instance. Not all Linux distributions will make this choice. So
> somebody on some other distribution compiles things with the pathname
> hard-coded in. On his system, it is /usr/X11R6/lib for libraries. But his
> program will not work on Debian, because we would have listened to you and
> moved our current libraries to a nonstandard location.
>
> But if that person happened to be building on a libc5 system with the
> hardcoded path, then Debian already broke his program. I agree that
No, HE broke his program. When you use -rpath, you are presumably aware of
the consequences of your actions, both beneficial and harmful. If you use
it, you have to be aware that this can happen.
> > and the program will be able to find the library at that point. If
> > you move the library and replace it with an incompatible one, you're
> > breaking the contract and the versioning mechanism, so you can't blame
> > the program for crashing, nor the tool that created the program.
>
> You're missing the point (I think; to which versioning mechanism are you
> referring?). It's the same version of the library, designed to be linked
> with different versions of other libraries.
>
> We can have libncurses3.4 designed to be linked with libc5 or one for libc6.
>
> Those are different versions of the library. They have different
> requirements. From the perspective of the dynamic linker, they can
> not be considered to be the same version.
That's what I was thinking (hence my confusion about which versioning
mechanism he's talking about). He was saying that we replaced a library
with a different and incompatible one, but of the same version.
>
> > Because you break a contract every time you remove a library from the
> > place in which it used to live.
>
> The 'contract' never should have cared about its location in the first
> place. The OS, through mechanisms like /etc/ld.so.conf, HAS THE RIGHT to
> move it. By assuming that it does not, YOU are breaking that contract.
>
> I disagree. If the OS is gong to move the library, it is responsible
> for making sure that old programs linked using -rpath continue to
> work, one way or another. You are effectively saying that -rpath is
> prohibited, which I do not think is reasonable.
No! -rpath is saying that it wants the program to IGNORE what the OS says
about where the library is. We don't care about programs that use -rpath;
they presumably know what they're doing. The program is, if our
distribution has programs compiled with -rpath, we can have trouble. (I
would say that almost no user would do that if it weren't the default.)
John Goerzen
etc.) What I'm talking about is a test (live). A simulation only goes so
far.
On Tue, Feb 02, 1999 at 06:57:07PM -0500, Ian Lance Taylor wrote:
> > How does the current packaging system allows me to test one version of
> > a package while other users of the same host are running a stable
> > version of that tool? Or are the GNU/Linux distributions all moving
> > towards the Micro$oft model of single-user workstations?
>
> So, you tell me. How can you run one version of the kernel while others run
> a different one? How about sendmail? You expect the system to somehow
> magically know that an incoming SMTP connect will be delivering you mail,
> and fire up your own "special" version? wu-ftpd? Apache? Frankly, that is
> not practical on a multi-user system. You are the one asking us to move
> towards the M$ model.
>
> This is really a sidetrack, but it's pretty each to test specific
> programs, although it's probably impossible on Linux to test a
> specific kernel. To test sendmail, ftpd, or Apache, just run them on
> a different port number, and arrange to connect to that port for
> testing purposes. I did stuff like this all the time when I was
> packaging and testing Kerberos releases.
>
John Goerzen
> So, in other words, you would have Debian violate FSSTND, FHS, and decades
>
> Wow, just when I thought we'd reached some degree of consensus.
>
> ``Decades of historical precedent?'' ``Arrogance almost beyond
> comprehension?'' Are you interested in solving the problem, or are
> you just talking?
I, and others, have already proposed a simple solution: add a command-line
option to disable rpath. A *better* solution would be to disable it where
it is not needed.
> Of all the nerve, this has to take the cake. I have no problem with
> maintaining compatibility, but when it comes to something like this, where
> compatibility CAN be maintained while still allowing us to take advantage of
> our features, to insist that we must ignore our features is outlandish.
>
> As far as I can tell, compatibility was broken with any program that
> linked with -rpath and a system directory. libtool merely happens to
> be one example.
Those programs broke compatibility. By using rpath, they were specifically
asking to NOT be compatible. This could be desirable in some cases, but
only in special cases, and not in general.
Ian Lance Taylor
From: John Goerzen <jgoe...@complete.org>
On Tue, Feb 02, 1999 at 06:47:26PM -0500, Ian Lance Taylor wrote:
> OK, let's assume for a moment that we cripple Debian by ignoring the FHS in
> this instance. Not all Linux distributions will make this choice. So
> somebody on some other distribution compiles things with the pathname
> hard-coded in. On his system, it is /usr/X11R6/lib for libraries. But his
> program will not work on Debian, because we would have listened to you and
> moved our current libraries to a nonstandard location.
>
> But if that person happened to be building on a libc5 system with the
> hardcoded path, then Debian already broke his program. I agree that
No, HE broke his program. When you use -rpath, you are presumably aware of
the consequences of your actions, both beneficial and harmful. If you use
it, you have to be aware that this can happen.
This argument would be more convincing if -rpath weren't required on
most Unix systems. Most Unixes do not have the complex ldconfig
mechanisms provided by Linux. Therefore, it is natural for people
writing cross-platform tools to add a -rpath argument to find, e.g.,
the X libraries.
-rpath is not some weird bizarro option. It's quite commonly used by
people who understand the difference between the program linker search
path and the dynamic linker search path.
-rpath works reasonably in the presence of upgrades, provided the
upgrade does not move any shared libraries. This is typical on other
Unix systems. Linux had a special one-time problem in the switch from
libc5 to glibc. Unfortunately, that switch was not handled as cleanly
as it could have been.
> > and the program will be able to find the library at that point. If
> > you move the library and replace it with an incompatible one, you're
> > breaking the contract and the versioning mechanism, so you can't blame
> > the program for crashing, nor the tool that created the program.
>
> You're missing the point (I think; to which versioning mechanism are you
> referring?). It's the same version of the library, designed to be linked
> with different versions of other libraries.
>
> We can have libncurses3.4 designed to be linked with libc5 or one for libc6.
>
> Those are different versions of the library. They have different
> requirements. From the perspective of the dynamic linker, they can
> not be considered to be the same version.
That's what I was thinking (hence my confusion about which versioning
mechanism he's talking about). He was saying that we replaced a library
with a different and incompatible one, but of the same version.
I don't see that in the quoted paragraph, although the appropriate
quote may have been dropped somewhere. He is referring to the
versioning mechanism, which is the soname. When the library changed
in an incompatible fashion, the soname should have changed.
Unfortunately, that didn't happen.
> > Because you break a contract every time you remove a library from the
> > place in which it used to live.
>
> The 'contract' never should have cared about its location in the first
> place. The OS, through mechanisms like /etc/ld.so.conf, HAS THE RIGHT to
> move it. By assuming that it does not, YOU are breaking that contract.
>
> I disagree. If the OS is gong to move the library, it is responsible
> for making sure that old programs linked using -rpath continue to
> work, one way or another. You are effectively saying that -rpath is
> prohibited, which I do not think is reasonable.
No! -rpath is saying that it wants the program to IGNORE what the OS says
about where the library is. We don't care about programs that use -rpath;
they presumably know what they're doing. The program is, if our
distribution has programs compiled with -rpath, we can have trouble. (I
would say that almost no user would do that if it weren't the default.)
I disagree. You should care about programs which use -rpath, and you
should strive to make them continue to work when possible.
In any case, I think everybody agrees that programs linked using
-rpath on a glibc system will consistently work on all foreseeable
future glibc systems.
Ian Lance Taylor
From: John Goerzen <jgoe...@complete.org>
What you're describing is a simulation (not in live cases, with fake data,
etc.) What I'm talking about is a test (live). A simulation only goes so
far.
Alexandre asked about testing one version of a package while others
are running a stable version. That is entirely possible. I'm not
talking about a simulation, in which at least one program is faked.
I'm talking about a real but limited test, using the real programs on
both ends of the link. It's true that that also only goes so far, but
it's still a pretty good idea, and it's a better test than a
simulation.
On Tue, Feb 02, 1999 at 06:57:07PM -0500, Ian Lance Taylor wrote:
> > How does the current packaging system allows me to test one version of
> > a package while other users of the same host are running a stable
> > version of that tool? Or are the GNU/Linux distributions all moving
> > towards the Micro$oft model of single-user workstations?
>
> So, you tell me. How can you run one version of the kernel while others run
> a different one? How about sendmail? You expect the system to somehow
> magically know that an incoming SMTP connect will be delivering you mail,
> and fire up your own "special" version? wu-ftpd? Apache? Frankly, that is
> not practical on a multi-user system. You are the one asking us to move
> towards the M$ model.
>
> This is really a sidetrack, but it's pretty each to test specific
> programs, although it's probably impossible on Linux to test a
> specific kernel. To test sendmail, ftpd, or Apache, just run them on
> a different port number, and arrange to connect to that port for
> testing purposes. I did stuff like this all the time when I was
> packaging and testing Kerberos releases.
Ian Lance Taylor
From: John Goerzen <jgoe...@complete.org>
On Tue, Feb 02, 1999 at 06:40:31PM -0500, Ian Lance Taylor wrote:
> So, in other words, you would have Debian violate FSSTND, FHS, and decades
>
> Wow, just when I thought we'd reached some degree of consensus.
>
> ``Decades of historical precedent?'' ``Arrogance almost beyond
> comprehension?'' Are you interested in solving the problem, or are
> you just talking?
I, and others, have already proposed a simple solution: add a command-line
option to disable rpath. A *better* solution would be to disable it where
it is not needed.
Good. Alexandre has already said that he will accept a patch to
disable -rpath for any directory listed in /etc/ld.so.conf. I
encourage you and/or others who care to write this patch. Then the
problem will be solved, without requiring further hyperbole.
Another simple solution would be to tweak the Debian dynamic linker in
the same way that the RedHat dynamic linker was tweaked, as already
discussed.
Ian
John Goerzen
> I, and others, have already proposed a simple solution: add a command-line
> option to disable rpath. A *better* solution would be to disable it where
> it is not needed.
>
> Good. Alexandre has already said that he will accept a patch to
> disable -rpath for any directory listed in /etc/ld.so.conf. I
> encourage you and/or others who care to write this patch. Then the
> problem will be solved, without requiring further hyperbole.
>
> Another simple solution would be to tweak the Debian dynamic linker in
> the same way that the RedHat dynamic linker was tweaked, as already
> discussed.
I apparently missed both messages to which you refer; however, commenting
upon the first: I'm unsure what you mean here. -rpath is decided at compile
time, ld.so.conf at runtime. ls.so.conf itself may change, and at compile
time it may not have a complete picture.
I don't know what sort of tweaks RedHat made. Can you enlighten me/us?
John
Ian Lance Taylor
From: John Goerzen <jgoe...@complete.org>
> Good. Alexandre has already said that he will accept a patch to
> disable -rpath for any directory listed in /etc/ld.so.conf. I
> encourage you and/or others who care to write this patch. Then the
> problem will be solved, without requiring further hyperbole.
>
> Another simple solution would be to tweak the Debian dynamic linker in
> the same way that the RedHat dynamic linker was tweaked, as already
> discussed.
I apparently missed both messages to which you refer; however, commenting
upon the first: I'm unsure what you mean here. -rpath is decided at compile
time, ld.so.conf at runtime. ls.so.conf itself may change, and at compile
time it may not have a complete picture.
I don't know what sort of tweaks RedHat made. Can you enlighten me/us?
Both were discussed on some set of mailing lists, but I don't know
precisely which ones. I thought at least one Debian list was on the
CC line at that point.
You're quite right that ld.so.conf may not be the same when a program
is linked and when it is run. However, it is appropriate for libtool
to use -rpath if the shared library is installed in a user's
directory; otherwise, the user would have to set LD_LIBRARY_PATH,
which is inconvenient, particularly if many people expect to run the
resulting program.
Therefore, it's not OK to eliminate -rpath entirely from libtool.
However, it's probably OK to avoid using -rpath for any system
directory, one which the dynamic linker will search.
The use of /etc/ld.so.cache, rather than a simple directory search,
means that there is no actual list of system directories on a Linux
system other than /lib and /usr/lib. However, /etc/ld.so.conf is a
reasonable approximation, one which will probably not mislead an
ordinary program.
It would also be possible to add an option to libtool to avoid the use
of -rpath. However, no package maintainer would ever use such an
option, since it would never be correct without knowing the --prefix
option used for configure. Only an organization like Debian would use
it. Alexandre sent out a patch which disabled -rpath, which Debian
could use just as easily as a special option. I don't have a copy of
that patch.
The RedHat patch to the libc5 dynamic linker has been sent out twice.
I've appended it again. This patch changes the handling of DT_RPATH
in order to make it more likely that a program linked on a libc5
system will work on a glibc system. It breaks the ABI for a libc5
program, but then the ABI was already broken anyhow.
Ian
--- ld.so-1.9.5/d-link/readelflib1.c.ewt Mon Nov 17 10:04:15 1997
+++ ld.so-1.9.5/d-link/readelflib1.c Mon Nov 17 10:23:15 1997
@@ -179,38 +179,10 @@
goto goof;
}
- /*
- * The ABI specifies that RPATH is searched before LD_*_PATH or
- * the default path of /usr/lib.
- * Check in rpath directories
- */
- for (tpnt = _dl_loaded_modules; tpnt; tpnt = tpnt->next) {
- if (tpnt->libtype == elf_executable) {
- pnt1 = (char *)tpnt->dynamic_info[DT_RPATH];
- if(pnt1) {
- pnt1 += (unsigned int) tpnt->loadaddr + tpnt->dynamic_info[DT_STRTAB];
- while(*pnt1){
- pnt2 = mylibname;
- while(*pnt1 && *pnt1 != ':') {
- if (pnt2 - mylibname < 1024)
- *pnt2++ = *pnt1++;
- else
- pnt1++;
- }
- if (pnt2 - mylibname >= 1024)
- break;
- if(pnt2[-1] != '/') *pnt2++ = '/';
- pnt = libname;
- while(*pnt) *pnt2++ = *pnt++;
- *pnt2++ = 0;
- tpnt1 = _dl_load_elf_shared_library(mylibname, 0);
- if(tpnt1) return tpnt1;
- if(*pnt1 == ':') pnt1++;
- }
- }
- }
- }
-
+ /* EWT - change things around a bit... The RPATH is almost definitely
+ wrong for libc 5 apps as things got moved around so much. Rather
+ then checking it first, we'll check it last. While this could
+ cause major breakages, it probably won't. */
/* Check in LD_{ELF_}LIBRARY_PATH, if specified and allowed */
pnt1 = _dl_library_path;
@@ -259,6 +231,38 @@
}
}
#endif
+
+ /*
+ * The ABI specifies that RPATH is searched before LD_*_PATH or
+ * the default path of /usr/lib.
+ * Check in rpath directories
+ */
+ for (tpnt = _dl_loaded_modules; tpnt; tpnt = tpnt->next) {
+ if (tpnt->libtype == elf_executable) {
+ pnt1 = (char *)tpnt->dynamic_info[DT_RPATH];
+ if(pnt1) {
+ pnt1 += (unsigned int) tpnt->loadaddr + tpnt->dynamic_info[DT_STRTAB];
+ while(*pnt1){
+ pnt2 = mylibname;
+ while(*pnt1 && *pnt1 != ':') {
+ if (pnt2 - mylibname < 1024)
+ *pnt2++ = *pnt1++;
+ else
+ pnt1++;
+ }
+ if (pnt2 - mylibname >= 1024)
+ break;
+ if(pnt2[-1] != '/') *pnt2++ = '/';
+ pnt = libname;
+ while(*pnt) *pnt2++ = *pnt++;
+ *pnt2++ = 0;
+ tpnt1 = _dl_load_elf_shared_library(mylibname, 0);
+ if(tpnt1) return tpnt1;
+ if(*pnt1 == ':') pnt1++;
+ }
+ }
+ }
+ }
/* Check in /usr/lib */
#ifdef IBCS_COMPATIBLE
John Goerzen
> You're quite right that ld.so.conf may not be the same when a program
> is linked and when it is run. However, it is appropriate for libtool
> to use -rpath if the shared library is installed in a user's
> directory; otherwise, the user would have to set LD_LIBRARY_PATH,
> which is inconvenient, particularly if many people expect to run the
> resulting program.
This is not correct either. User home directories are never guaranteed to
remain at the same place, and in some situations (ISPs, academia, etc.) may
actually frequently move. Hence the reason that people are encouraged to
use $HOME instead of hard-coding their home directory.
Not only that, but I just make a $HOME/lib directory, set that in
LD_LIBRARY_PATH, and I'm set. That's not so hard.
> Therefore, it's not OK to eliminate -rpath entirely from libtool.
> However, it's probably OK to avoid using -rpath for any system
> directory, one which the dynamic linker will search.
I did not advocate eliminating it completely, esp. in light of the
(apparent) fact that it's necessary on other platforms.
> The use of /etc/ld.so.cache, rather than a simple directory search,
I think you mean ld.so.conf here (cache is basically the result of
ld.so.conf plus the list of installed libraries).
> means that there is no actual list of system directories on a Linux
> system other than /lib and /usr/lib. However, /etc/ld.so.conf is a
> reasonable approximation, one which will probably not mislead an
> ordinary program.
> It would also be possible to add an option to libtool to avoid the use
> of -rpath. However, no package maintainer would ever use such an
I think we're having a bit of difficulty communicating here. Do you mean
Debian package maintainer or...?
> option, since it would never be correct without knowing the --prefix
> option used for configure. Only an organization like Debian would use
> it. Alexandre sent out a patch which disabled -rpath, which Debian
> could use just as easily as a special option. I don't have a copy of
> that patch.
I would maintain that -rpath should be disabled by default on Linux. Adding
an option to disable it is not the correct solution, but it does solve the
immediate problem at hand, so I would not complain about that certainly, but
it leaves the underlying issue unfixed.
Ian Lance Taylor
From: John Goerzen <jgoe...@complete.org>
On Tue, Feb 02, 1999 at 08:41:06PM -0500, Ian Lance Taylor wrote:
> You're quite right that ld.so.conf may not be the same when a program
> is linked and when it is run. However, it is appropriate for libtool
> to use -rpath if the shared library is installed in a user's
> directory; otherwise, the user would have to set LD_LIBRARY_PATH,
> which is inconvenient, particularly if many people expect to run the
> resulting program.
This is not correct either. User home directories are never guaranteed to
remain at the same place, and in some situations (ISPs, academia, etc.) may
actually frequently move. Hence the reason that people are encouraged to
use $HOME instead of hard-coding their home directory.
This argument sounds like a quibble to me. My home directory hasn't
moved since I installed this system eight years ago. If you really
can't stomach the home directory here, then instead consider
/usr/local/lib, or /opt, or any place which a sysadmin might set up
for the permanent installation of user controlled binaries.
Not only that, but I just make a $HOME/lib directory, set that in
LD_LIBRARY_PATH, and I'm set. That's not so hard.
It's a fair bit harder when you consider my final point about many
people expecting to run the resulting program.
> The use of /etc/ld.so.cache, rather than a simple directory search,
I think you mean ld.so.conf here (cache is basically the result of
ld.so.conf plus the list of installed libraries).
No, I mean /etc/ld.so.cache. /etc/ld.so.conf is just a list of
directories. If the dynamic linker just searched that list of
directories, the Linux library search algorithm would be much easier
to understand. In reality, ldconfig builds /etc/ld.so.cache with
flags indicating the type of each shared library and which version of
libc was built against. The dynamic linker then uses those flags when
deciding which library to pull in.
Without some such scheme, moving the shared libraries without changing
the sonames would not have worked, because the dynamic linker would
simply have found the glibc version of a shared library even for a
libc5 program, since the glibc shared library is first in the link
order (or, if it isn't, then the glibc program would fail instead,
finding the libc5 version of the shared library).
> means that there is no actual list of system directories on a Linux
> system other than /lib and /usr/lib. However, /etc/ld.so.conf is a
> reasonable approximation, one which will probably not mislead an
> ordinary program.
> It would also be possible to add an option to libtool to avoid the use
> of -rpath. However, no package maintainer would ever use such an
I think we're having a bit of difficulty communicating here. Do you mean
Debian package maintainer or...?
No, I mean the person who writes the software package in the first
place, as in ``I am the GNU binutils maintainer.''
> option, since it would never be correct without knowing the --prefix
> option used for configure. Only an organization like Debian would use
> it. Alexandre sent out a patch which disabled -rpath, which Debian
> could use just as easily as a special option. I don't have a copy of
> that patch.
I would maintain that -rpath should be disabled by default on Linux. Adding
an option to disable it is not the correct solution, but it does solve the
immediate problem at hand, so I would not complain about that certainly, but
it leaves the underlying issue unfixed.
I disagree. There are uses for -rpath beyond using it for system
libraries. These uses are just as valid on Linux as they are on any
other system.
Ian
Alexandre Oliva
> From: John Goerzen <jgoe...@complete.org>
> No, HE broke his program. When you use -rpath, you are
> presumably aware of the consequences of your actions, both
> beneficial and harmful. If you use it, you have to be aware that
> this can happen.
How could a user that linked his program before the Debian Developers
decided to move the libc5 libraries be aware of the consequences of
his actions?
On Feb 2, 1999, Ian Lance Taylor <i...@cygnus.com> wrote:
> -rpath works reasonably in the presence of upgrades, provided the
> upgrade does not move any shared libraries.
Moving shared libraries has never been the real problem. Replacing
shared libraries with incompatible ones was.
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
John Goerzen
> > presumably aware of the consequences of your actions, both
> > beneficial and harmful. If you use it, you have to be aware that
> > this can happen.
>
> How could a user that linked his program before the Debian Developers
> decided to move the libc5 libraries be aware of the consequences of
> his actions?
You're making the thing too specific. The point is that libraries can
change; we're not the only ones that have done that (Linux isn't the only OS
that has done that either). Forcing the paths to be in the executable when
they do not need to be should logically be thought of as bad due this this
possibility.
Ian Lance Taylor
From: John Goerzen <jgoe...@complete.org>
You're making the thing too specific. The point is that libraries can
change; we're not the only ones that have done that (Linux isn't the only OS
that has done that either). Forcing the paths to be in the executable when
they do not need to be should logically be thought of as bad due this this
possibility.
Actually, I'm curious as to whether there really is another example of
an OS which changed shared libraries without changing the soname (as
Alexandre pointed out, that was really what caused the problem). I
don't know of any example of that.
Ian
Jules Bean
> Date: Tue, 2 Feb 1999 22:27:15 -0600
> From: John Goerzen <jgoe...@complete.org>
>
> You're making the thing too specific. The point is that libraries can
> change; we're not the only ones that have done that (Linux isn't the only OS
> that has done that either). Forcing the paths to be in the executable when
> they do not need to be should logically be thought of as bad due this this
> possibility.
>
> Actually, I'm curious as to whether there really is another example of
> an OS which changed shared libraries without changing the soname (as
> Alexandre pointed out, that was really what caused the problem). I
> don't know of any example of that.
And, as we've pointed out, changing the soname is not a practicable
answer, until we have a cleverer automatic soname system.
Just look at the mess of libraries embeddeded in GNOME.
(Does symbol versioning fix this? I don't know...)
Whilst I'm glad that we've come to some agreement that its not necessary
to use -rpath for system directories, for some appropriate definition of
system directories (which is a decision which, for Debian, can be made at
the distribution level, so we'll get it right), I am quite baffled that
you still think it's right to enable it as default...
I really don't see why it's clever to hardcode paths... And I do see why
it's clever to have a dynamic library which goes looking for its libs, and
knows where to go... This is in fact largely orthogonal to the
'incompatible lib without soname change' argument.
Jules
/----------------+-------------------------------+---------------------\
| Jelibean aka | ju...@jellybean.co.uk | 6 Evelyn Rd |
| Jules aka | ju...@debian.org | Richmond, Surrey |
| Julian Bean | jm...@hermes.cam.ac.uk | TW9 2TF *UK* |
+----------------+-------------------------------+---------------------+
| War doesn't demonstrate who's right... just who's left. |
| When privacy is outlawed... only the outlaws have privacy. |
\----------------------------------------------------------------------/
Alexandre Oliva
> On Tue, 2 Feb 1999, Ian Lance Taylor wrote:
>> Actually, I'm curious as to whether there really is another example of
>> an OS which changed shared libraries without changing the soname (as
>> Alexandre pointed out, that was really what caused the problem). I
>> don't know of any example of that.
> And, as we've pointed out, changing the soname is not a practicable
> answer, until we have a cleverer automatic soname system.
libtool might be able to help here, by providing a flag to arrange
that each library is sonamed after its dependencies, or that an
arbitrary string is appended to the soname. I'm not sure this can be
made portable, but then, libtool could reject or silently ignore such
flags if one tries to use them in a system that does not support
them.
> I really don't see why it's clever to hardcode paths...
I think the point is that they can't hurt (or couldn't, before the bad
transition from libc5 to glibc), and selectively dropping them
requires additional code, which nobody has ever implemented.
--
Alexandre Oliva http://www.dcc.unicamp.br/~oliva aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org}
Universidade Estadual de Campinas, SP, Brasil
Bernard Dautrevaux
> From: John Goerzen [mailto:jgoe...@complete.org]
> Sent: Wednesday, February 03, 1999 2:15 AM
> To: Ian Lance Taylor
> Cc: ol...@dcc.unicamp.br; c...@debian.org;
> debian...@lists.debian.org;
> yo...@gimp.org; ja...@nocrew.org; bug-l...@gnu.org;
> lib...@packages.debian.org
> Subject: Re: -rpath with libtool and Debian Linux
>
>
> On Tue, Feb 02, 1999 at 07:34:32PM -0500, Ian Lance Taylor wrote:
>
> > I, and others, have already proposed a simple solution:
> add a command-line
> > option to disable rpath. A *better* solution would be
> to disable it where
> > it is not needed.
> >
> > disable -rpath for any directory listed in /etc/ld.so.conf. I
> > encourage you and/or others who care to write this patch. Then the
> > problem will be solved, without requiring further hyperbole.
> >
> > Another simple solution would be to tweak the Debian
> dynamic linker in
> > the same way that the RedHat dynamic linker was tweaked, as already
> > discussed.
>
> I apparently missed both messages to which you refer;
> however, commenting
> upon the first: I'm unsure what you mean here. -rpath is
> decided at compile
> time, ld.so.conf at runtime. ls.so.conf itself may change,
> and at compile
> time it may not have a complete picture.
Right, but wrong also... What we want is that when compiling we do not
hardcode in the executable any directory that is searched by default, so
that at runtime, if ld.so.conf was changed in the meantime (while the
libraries were moved accordingly), we find the right library in its new
place...
>
> I don't know what sort of tweaks RedHat made. Can you
> enlighten me/us?
They looked in LD_LIBRARY_PATH before -rpath I think...
Regards,
Bernard
--------------------------------------------
Bernard Dautrevaux
Microprocess Ingéniérie
97 bis, rue de Colombes
92400 COURBEVOIE
FRANCE
Tel: +33 (0) 1 47 68 80 80
Fax: +33 (0) 1 47 88 97 85
e-mail: dautr...@microprocess.com
b.daut...@usa.net
--------------------------------------------
Bernard Dautrevaux
> From: Alexandre Oliva [mailto:ol...@dcc.unicamp.br]
> Sent: Wednesday, February 03, 1999 12:01 PM
> To: Jules Bean
> Cc: Ian Lance Taylor; jgoe...@complete.org; jda...@wi.leidenuniv.nl;
> debian...@lists.debian.org; yo...@gimp.org; bug-l...@gnu.org;
> lib...@packages.debian.org
> Subject: Re: -rpath with libtool and Debian Linux
>
>
>
> > On Tue, 2 Feb 1999, Ian Lance Taylor wrote:
>
>
> > I really don't see why it's clever to hardcode paths...
>
> I think the point is that they can't hurt (or couldn't, before the bad
> transition from libc5 to glibc), and selectively dropping them
> requires additional code, which nobody has ever implemented.
>
To be right you have to say: They shouldn't hurt. The problem is that
they *do* hurt... Anyway I think we've reached a consesus to not use
-rpath on system directories (that is directories that get searched by
default) and that seems perfectly reasonable.
I can *not* expect the linker to be able to resolve problems with
libraries he do *not* know and I do *not* ask for a system that will
allow a program built on some specially tailored system (that is one
where I've added a lot of new "system" directories) to work on standard
systems. After all if I mess up my system, it's my fault...
Marcus Brinkmann
> On Wed, Feb 03, 1999 at 02:22:14AM -0200, Alexandre Oliva wrote:
>
> > > presumably aware of the consequences of your actions, both
> > > beneficial and harmful. If you use it, you have to be aware that
> > > this can happen.
> >
> > How could a user that linked his program before the Debian Developers
> > decided to move the libc5 libraries be aware of the consequences of
> > his actions?
>
> change; we're not the only ones that have done that (Linux isn't the only OS
> that has done that either). Forcing the paths to be in the executable when
> they do not need to be should logically be thought of as bad due this this
> possibility.
You are wrong, IMHO. Libraries with the same soname must be compatible. Debian
broke this with the libc5->libc6 transition. It was our mistake (and teh
mistake of every Linux dist), and it is our job to fix that, not the job of
the libtool maintainer (who can't anyway).
Which other OS has done this mistake before?
Marcus
--
"Rhubarb is no Egyptian god." Debian GNU/Linux finger brinkmd@
Marcus Brinkmann http://www.debian.org master.debian.org
Marcus.B...@ruhr-uni-bochum.de for public PGP Key
http://homepage.ruhr-uni-bochum.de/Marcus.Brinkmann/ PGP Key ID 36E7CD09
Ian Lance Taylor
From: Jules Bean <jm...@hermes.cam.ac.uk>
And, as we've pointed out, changing the soname is not a practicable
answer, until we have a cleverer automatic soname system.
One possibility would be a soname rewriting program. It's a moot
point now, though.
(Does symbol versioning fix this? I don't know...)
Yes, it does. Symbol versioning permits changing specific symbols in
a library as their interface changes, such that old programs get the
old symbol and new programs get the new symbol. It requires a fair
bit of discipline on the part of the library maintainer, but at least
in principle nobody else has to worry about it.
Whilst I'm glad that we've come to some agreement that its not necessary
to use -rpath for system directories, for some appropriate definition of
system directories (which is a decision which, for Debian, can be made at
the distribution level, so we'll get it right), I am quite baffled that
you still think it's right to enable it as default...
I really don't see why it's clever to hardcode paths... And I do see why
it's clever to have a dynamic library which goes looking for its libs, and
knows where to go... This is in fact largely orthogonal to the
'incompatible lib without soname change' argument.
Remember that the path set by -rpath is only a suggestion. It's not
an error if a library is not found at runtime at the path where it was
at link time. -rpath just adds to the search path used to find shared
libraries.
It's clever to hardcode paths to any shared libraries because it makes
it more likely that your program will be able to run without the
person running it having to set LD_LIBRARY_PATH.
Ian
James Troup
John Goerzen <jgoe...@complete.org> writes:
> One would hope that you [Alexandre] would learn a bit about Unix
> before criticizing it.
For God's sakes, please don't do this kind of thing. Whatever one
might think about libtool, it's Evilness and -rpath, that kind of
unjustified ad hominem against an upstream maintainer is completely
out of line and just makes Debian look very bad :-(
It should be clear to anyone that the author/maintainer (I can't
remember which) of a complex tool like libtool clearly _does_ know
quite a bit about UNIX and accusing him of not doing so in such a
gratuitous fashion doesn't help your arguments any.
--
James
Joey Hess
> This argument sounds like a quibble to me. My home directory hasn't
> moved since I installed this system eight years ago. If you really
> can't stomach the home directory here, then instead consider
> /usr/local/lib, or /opt, or any place which a sysadmin might set up
> for the permanent installation of user controlled binaries.
>
> Not only that, but I just make a $HOME/lib directory, set that in
> LD_LIBRARY_PATH, and I'm set. That's not so hard.
>
> It's a fair bit harder when you consider my final point about many
> people expecting to run the resulting program.
But if the library is in /usr/local or /opt, then the sysadmin is setting
this up and they merely have to add the directory to /etc/ld.so.conf and run
ldconfig.
> I disagree. There are uses for -rpath beyond using it for system
> libraries. These uses are just as valid on Linux as they are on any
> other system.
No-one's arguing there arn't uses, it's simply not a good default.
--
see shy jo