How to use a different ld-linux.so?
Paul D. Smith
I have a need to use a different runtime linker (ld-linux.so) for some
of my applications. I _don't_ want to change the executables themselves
(changing the path to the runtime linker in the ELF image for example).
I know that I can just run my application as an argument to ld-linux.so,
and that actually does work very well. But, I am left with two
problems:
* Some of my applications fork/exec other programs, and those other
programs also need to use the other runtime linker. Just as I don't
want to change the ELF image, I certainly don't want to have to
recompile the programs to exec the runtime linker!
* Debugging: I can't debug because I can't find a way to convince GDB
to invoke the program-to-be-debugged using an alternative runtime
linker.
I was wondering if anyone has any ideas about how to do this? I checked
the man pages and didn't see anything (I was looking for something like
an environment variable that ld-linux.so might obey to re-exec a
different version, or some such thing--not likely, I know).
If there's nothing like that available another idea is to use LD_PRELOAD
to install a private version of the execve() system call, which would
set the alternative ld-linux.so as the first argument. Any thoughts on
this approach? Do you think it would solve the GDB problem as well?
I'm going to proceed to work on this idea but I wanted to see if anyone
had other thoughts about it.
PS. In case anyone's curious, I'm trying to run some programs natively
on my desktop system which were actually compiled for a different
system, and the native and target systems have different versions of
glibc installed. I can't change either of the versions of glibc,
but I do have the complete root filesystem of the target box in a
subdirectory of my native box and I can use LD_LIBRARY_PATH, etc. to
use them... I just need a way to get the runtime environment to
initialize properly.
--
-------------------------------------------------------------------------------
Paul D. Smith <psm...@nortelnetworks.com> HASMAT--HA Software Mthds & Tools
"Please remain calm...I may be mad, but I am a professional." --Mad Scientist
-------------------------------------------------------------------------------
These are my opinions---Nortel Networks takes no responsibility for them.
Kasper Dupont
--
Kasper Dupont -- der bruger for meget tid paa usenet.
For sending spam use kas...@kd.lir.dk and ab...@kd.lir.dk
I'd rather be a hammer than a nail.
John Reiser
> of my applications. I _don't_ want to change the executables themselves
> (changing the path to the runtime linker in the ELF image for example).
Why not? http://www.BitWagon.com/rtldi/rtldi.html
> I know that I can just run my application as an argument to ld-linux.so,
> and that actually does work very well. But, I am left with two
> problems:
>
> * Some of my applications fork/exec other programs, and those other
> programs also need to use the other runtime linker. Just as I don't
> want to change the ELF image, I certainly don't want to have to
> recompile the programs to exec the runtime linker!
fork() is never a problem, but it is necessary to intercept execve()
if you wish to invoke a different executable or to change its environment.
> * Debugging: I can't debug because I can't find a way to convince GDB
> to invoke the program-to-be-debugged using an alternative runtime
> linker.
$ gdb /lib/ld-linux.so.2
(gdb) run --library-path /lib /bin/date
If you want to set breakpoints then put one near the end of _dl_start_user
where it invokes _dl_init_internal. Inspect /proc/<pid>/maps to find out
what has been mapped, then use the add-symbol-file command of gdb.
Find the offset from "objdump --section-headers a.elf | grep text"
then add the base from the map of the address space.
> ... another idea is to use LD_PRELOAD
> to install a private version of the execve() system call, which would
> set the alternative ld-linux.so as the first argument.
By using LD_PRELOAD one can supersede the visible uses of execve() [the ones
that use global symbol linkage to invoke execve()], but there can be "bare"
system calls that do not use symbols.
Paul D. Smith
kd> I think chroot is your best option.
Unfortunately, chroot is not an option in this particular situation for
various reasons, including that it requires root privileges to invoke,
and that once inside the chroot area I don't have access to the other
tools I need. Importing all those other tools to work inside the chroot
environment is not practical.
I should point out that a lot of work with this environment is done
inside UML. I need this specialized invocation for those situations
where invoking UML is not feasible... and for many of the same reasons
UML is not feasible, chroot is not feasible.
Paul D. Smith
>> I have a need to use a different runtime linker (ld-linux.so) for some
>> of my applications. I _don't_ want to change the executables themselves
>> (changing the path to the runtime linker in the ELF image for example).
jr> Why not? http://www.BitWagon.com/rtldi/rtldi.html
The main reason is because I don't have easy write access to the
binaries. Also, those same binaries are being used inside a UML
instance, where they want the "normal" path to the runtime linker... I
don't want to have to edit the ELF image to swap them back and forth.
The rtldi tool looks interesting, but I'm not sure it will work in our
environment: we really need a way of choosing at runtime, not link time,
what the pathname to the linker will be.
I'm curious, since you need to specify which runtime linker you want at
link time anyway, what's the advantage of rtldi over just specifying the
alternative linker directly with -Wl,--dynamic-linker at link time?
IOW, in your example on the web page, why not just say:
... -Wl,--dynamic-linker=/2.2.4-24/ld-linux.so.2 ...
instead of invoking /2.2.4-24/rtldi ? I see that the latter sets up the
library path properly so you don't have to set LD_LIBRARY_PATH, but are
there other differences as well?
>> * Debugging: I can't debug because I can't find a way to convince GDB
>> to invoke the program-to-be-debugged using an alternative runtime
>> linker.
jr> $ gdb /lib/ld-linux.so.2
jr> (gdb) run --library-path /lib /bin/date
jr> If you want to set breakpoints then put one near the end of
jr> _dl_start_user where it invokes _dl_init_internal. Inspect
jr> /proc/<pid>/maps to find out what has been mapped, then use the
jr> add-symbol-file command of gdb. Find the offset from "objdump
jr> --section-headers a.elf | grep text" then add the base from the
jr> map of the address space.
Oy! I need something simpler than that. This is going to be a commonly
used environment. We could look into doing that through a GDB init
script or something I guess.
>> ... another idea is to use LD_PRELOAD to install a private version
>> of the execve() system call, which would set the alternative
>> ld-linux.so as the first argument.
jr> By using LD_PRELOAD one can supersede the visible uses of execve()
jr> [the ones that use global symbol linkage to invoke execve()], but
jr> there can be "bare" system calls that do not use symbols.
Yes, true. I wonder how prevalent those are. Certainly none of our
code uses a bare system call.
I tried this using a simple replacement that just printed an extra
string than ran the native execve(), and that at least worked:
$ /bin/sh -c '/bin/echo hi'
hi
$ LD_PRELOAD=libexecshim.so.1.0 /bin/sh -c '/bin/echo hi'
woot woot!
hi
John Reiser
> environment: we really need a way of choosing at runtime, not link time,
> what the pathname to the linker will be.
rtldi chooses which ld-linux.so.2 at runtime, after the main executable
has been invoked by execve(). rtldi just happens to base its choice on
the contents of the PT_INTERP, something that was known at the time of
execve(); but it is obvious that a modified rtldi could consult time(),
read(), etc., [or even getenv()] to make its choice "more dynamically."
> I'm curious, since you need to specify which runtime linker you want at
> link time anyway, what's the advantage of rtldi over just specifying the
> alternative linker directly with -Wl,--dynamic-linker at link time
> IOW, in your example on the web page, why not just say:
>
> ... -Wl,--dynamic-linker=/2.2.4-24/ld-linux.so.2 ...
>
> instead of invoking /2.2.4-24/rtldi ?
If you have an old a.elf main executable program that was linked
to run with libc.so.6, and you want to run that a.elf on a machine
whose libc.so.6 differs from the one which a.elf was linked against,
and you want a reasonable guarantee that it will work, then it may
be necessary to use something like rtldi to choose the matching
ld-linux.so.2 and libc.so.6 and other libraries as of the time of
static binding of a.elf. In theory, libc.so.6 is backwards compatibile
(a given a.elf should run the same under any subsequent libc.so.6),
but in practice there have been too many bugs in {ld-linux.so.2,
libc.so.6, ...}. For example, many programs linked for libc.so.6
under RedHat Linux 7.2 and earlier won't run under glibc-2.3.2.
The most effective way to run such a program may be to install the
old glibc-x.y.z[.rpm] in a different location, binary edit the PT_INTERP
string of the a.elf, and use rtldi. This makes the old [edited] a.elf
simultaneously interoperable with other programs from different
libc.so.6 generations, and only the "oddball" program has to know.
> I see that the latter sets up the
> library path properly so you don't have to set LD_LIBRARY_PATH, but are
> there other differences as well?
ld-linux.so.2, libc.so.6, libdl.so, libnss*.so, and _many_ other shared
libraries of a given released version, are a matched set. They must be
used together, all from the same release, or not at all. Mixing and
matching pieces from different glibc-x.y.z need not work. In order to
use a consistent set that differs from the default set, you must invoke
the ld-linux.so.2 using "--library-path PATH" to specify the location
of the rest of the set. Using --library-path supersedes LD_LIBRARY_PATH
for one execve() only, and does not interfere with chilren, execve()
with no fork(), or other processes.
>
> >> * Debugging: I can't debug because I can't find a way to convince GDB
> >> to invoke the program-to-be-debugged using an alternative runtime
> >> linker.
>
> jr> $ gdb /lib/ld-linux.so.2
> jr> (gdb) run --library-path /lib /bin/date
> jr> If you want to set breakpoints then put one near the end of
> jr> _dl_start_user where it invokes _dl_init_internal. Inspect
> jr> /proc/<pid>/maps to find out what has been mapped, then use the
> jr> add-symbol-file command of gdb. Find the offset from "objdump
> jr> --section-headers a.elf | grep text" then add the base from the
> jr> map of the address space.
>
> Oy! I need something simpler than that. This is going to be a commonly
> used environment. We could look into doing that through a GDB init
> script or something I guess.
Unless you modify execve() or ld-linux.so.2, or allow chroot, then
this is as close as you can get.
Paul D. Smith
>> The rtldi tool looks interesting, but I'm not sure it will work in our
>> environment: we really need a way of choosing at runtime, not link time,
>> what the pathname to the linker will be.
jr> rtldi chooses which ld-linux.so.2 at runtime, after the main
jr> executable has been invoked by execve(). rtldi just happens to
jr> base its choice on the contents of the PT_INTERP, something that
jr> was known at the time of execve(); but it is obvious that a
jr> modified rtldi could consult time(), read(), etc., [or even
jr> getenv()] to make its choice "more dynamically."
OK, I'll check out the code. I was at first thrown off by the
requirement that the "real" ld-linux path must be no longer than the one
in the executable but now I see that this applies only to old
executables you're trying to patch without rebuilding.
Thanks John!
John Reiser
help choose libraries dynamically. In search paths used to locate
DT_NEEDED shared libraries, ld-linux.so.2 expands the strings
${ORIGIN}, ${PLATFORM}, and ${LIB} using the shell environment
variables. This is undocumented (at least, I cannot find any),
but unlikely to disappear; it's for "Solaris compatibility".
The source is function _dl_dst_substitute() in file elf/dl-load.c.
"dst" apparently means "dynamic string token".
Paul D. Smith
jr> If re-linking is allowed, then ld-linux.so.2 has a feature that can
jr> help choose libraries dynamically. In search paths used to locate
jr> DT_NEEDED shared libraries, ld-linux.so.2 expands the strings
jr> ${ORIGIN}, ${PLATFORM}, and ${LIB} using the shell environment
jr> variables. This is undocumented (at least, I cannot find any),
jr> but unlikely to disappear; it's for "Solaris compatibility".
jr> The source is function _dl_dst_substitute() in file elf/dl-load.c.
jr> "dst" apparently means "dynamic string token".
That's interesting... I'll keep this in mind.
In my situation, though, I actually need a different runtime linker
itself. My situation is that the code to be tested has a _NEWER_
version of glibc than what exists on the desktop. For various annoying
but difficult to change reasons I can't upgrade the desktop glibc
libraries to match the ones under test.
So, backward compatibility doesn't help me: I need forward compatibility,
which of course is not there.