dlopen, poor man's implementation
Jorn van Engelen
Xorg-server depends on a dlopen implementation to load the video and
input drivers. Till five years ago xorg-server had it's own
implementation to dynamically load these drivers, now it depends on
dlopen and family.
Now that ELF format is supported, a mmap which maps files is needed for
a proper dlopen implementation. I was wondering whether a poor man's
dlopen, which does not use mmap, is possible.
This is my idea for implementing a dlopen without mmap:
1. For dlopen: do a fopen on the file, then copy the file into memory
with fread, then fclose the file.
2. For dlsym: search the in-memory copy of the file for the symbol, then
return a pointer to that memory location. The symbol is, for example, a
function.
3. For dlclose: free the memory which was used to hold a copy of the
file in memory.
Now I do have a couple of concerns with my idea:
1. Is the memory executable? I plan to use malloc to allocate the memory
which holds a copy of the file. (the pointer to a symbol could be a
pointer to a function, so the memory has to be executable).
Also, will the program be able to jump to inside the malloc'ed memory
location?
2. Will jumps inside functions in the loaded file go ok?
3. Will functions inside the loaded file be able to read the strings
they use, e.g.: when they use printf("foo");. Perhaps these strings are
in an entirely different section in the file, and not inside the function.
What are your thoughts? Could an implementation like this work?
With kind regards,
Jorn van Engelen
Antoine LECA
> Now that ELF format is supported, a mmap which maps files is needed for
> a proper dlopen implementation.
Sorry, I do not see the connection with ELF (I do see the connection
between mmap and dlopen, though; but neither are particularly connected
with ELF in my mind.)
> I was wondering whether a poor man's dlopen, which does not use mmap,
> is possible.
At least, it could be something to try while the official team is
working hard on the regular mmap stuff. As you'll see, a lot of work
here has to be done to support shared libraries, independently of
whether it uses mmap or not.
> This is my idea for implementing a dlopen without mmap:
Now I should say, "for the ELF format!" Because ELF makes a number of
important points to be more definite in this area.
> 1. For dlopen: do a fopen on the file, then copy the file into memory
> with fread, then fclose the file.
You should keep in mind that the ELF file will be loaded at a different
address than the one it is designed for; this is not a problem for PIC
(.so binaries like shared libraries), but it should be kept in mind for
regular executables. On the other hand, shared objects need a number of
other things to be done, like initialization of the GOT, the resolution
of the fix ups relocations.
> 2. For dlsym: search the in-memory copy of the file for the symbol, then
> return a pointer to that memory location. The symbol is, for example, a
> function.
I believe a regular implementation for dlsym (like the one which should
go inside the dynamic loader) should do that work smoothly here.
In NetBSD it resides in src/libexec/ld_elf.so/rtld.c
> Now I do have a couple of concerns with my idea:
>
> 1. Is the memory executable?
Sorry, this one I do not know for sure.
> I plan to use malloc to allocate the memory which holds a copy of the
file.
Perhaps you should consider memory-only mmap here (which is normally
used for sharing memory between processes): it should not show
significant differences w.r.t malloc for your code, and might ease a lot
the reuse of both you using external, existing code (like rtld), and
later others reusing your code to implement regular dlopen.
> Also, will the program be able to jump to inside the malloc'ed memory
> location?
For i386 ELF, yes, because there is a single memory address space (MINIX
a.out traditionally has two spaces, T for code and D for data, which is
also used for stack, heap, malloc etc.)
> 2. Will jumps inside functions in the loaded file go ok?
Only for PIC code; and after you made sure the reloations (fix ups) for
GOT etc. has been processed.
> 3. Will functions inside the loaded file be able to read the strings
> they use, e.g.: when they use printf("foo");. Perhaps these strings are
> in an entirely different section in the file, and not inside the function.
This kind of stuff is exactly what the PIC mechanism deals with.
Antoine
Jorn van Engelen
> Jorn van Engelen wrote:
>> Now that ELF format is supported, a mmap which maps files is needed for
>> a proper dlopen implementation.
>
> Sorry, I do not see the connection with ELF (I do see the connection
> between mmap and dlopen, though; but neither are particularly connected
> with ELF in my mind.)
dlopen and mmap indeed have no particular connection with ELF. But as
you noted in another thread, ELF makes it easier to have dynamically
linked shared libraries than a.out.
>> This is my idea for implementing a dlopen without mmap:
>
> Now I should say, "for the ELF format!" Because ELF makes a number of
> important points to be more definite in this area.
Completely agree, making a dlopen implementation for the ELF format!
>> 1. For dlopen: do a fopen on the file, then copy the file into memory
>> with fread, then fclose the file.
>
> You should keep in mind that the ELF file will be loaded at a different
> address than the one it is designed for; this is not a problem for PIC
> (.so binaries like shared libraries), but it should be kept in mind for
> regular executables.
Hadn't thought about that yet. So regular executables don't need PIC
because the MMU translates the addresses.
> On the other hand, shared objects need a number of
> other things to be done, like initialization of the GOT, the resolution
> of the fix ups relocations.
I've been reading up on GOT (global offset table). To make sure I
understand correctly: when the code in the shared object accesses data
in the regular executable the following happens:
1. It will dereference a pointer to that data, this pointer is in the GOT.
2. It knows where the GOT is because it is located at a fixed offset
from the code in the shared object.
3. For 1. to work, the loader (dlopen) needs to fill the GOT with the
correct pointers.
Similar to this is the PLT (procedure linkage table), which takes care
of routines instead of data. The PLT can be extended to have 'lazy'
behavior (where the GOT gets populated in advance, an entry in the PLT
will only get populated when called).
Not only dynamically linked shared libraries have GOT and PLT, but the
regular executable also has these tables. I assume these are used to
lookup data and routines in the shared objects.
Please correct me where I'm wrong!
>> 2. For dlsym: search the in-memory copy of the file for the symbol, then
>> return a pointer to that memory location. The symbol is, for example, a
>> function.
>
> I believe a regular implementation for dlsym (like the one which should
> go inside the dynamic loader) should do that work smoothly here.
> In NetBSD it resides in src/libexec/ld_elf.so/rtld.c
Thanks for the pointer! It'll save some time.
>> I plan to use malloc to allocate the memory which holds a copy of the
> file.
>
> Perhaps you should consider memory-only mmap here (which is normally
> used for sharing memory between processes): it should not show
> significant differences w.r.t malloc for your code, and might ease a lot
> the reuse of both you using external, existing code (like rtld), and
> later others reusing your code to implement regular dlopen.
Good idea.
>
> Antoine
>
Thanks you very much for the advice and comments so far!
Tomas Hruby
> >address than the one it is designed for; this is not a problem for PIC
> >(.so binaries like shared libraries), but it should be kept in mind for
> >regular executables.
>
> Hadn't thought about that yet. So regular executables don't need PIC
> because the MMU translates the addresses.
Normal executable does not need PIC because it is loaded at the right
(virtual) address. E.g. for a.out it is loaded from address 0. In case
of ELF, the file indicates at what virtual address it should be
loaded.
In case of dynamically loaded object files the desired memory may be
already occupied and thus it needs PIC to load correctly anywhere.
T.
Antoine LECA
> On 04/11/2011 09:40 AM, Antoine LECA wrote:
>>> 1. For dlopen: do a fopen on the file, then copy the file into memory
>>> with fread, then fclose the file.
>>
>> You should keep in mind that the ELF file will be loaded at a different
>> address than the one it is designed for; this is not a problem for PIC
>> (.so binaries like shared libraries), but it should be kept in mind for
>> regular executables.
>
> Hadn't thought about that yet. So regular executables don't need PIC
> because the MMU translates the addresses.
Regular executable do not need to use -kPIC (cf. Tomas' post). The point
is, you cannot use your idea right out of the box for further deal with
them, since the system won't let you map them at the desired address
(because it is where the system already loaded the executable itself);
perhaps what the system had loaded itself of the binary is enough for
your needs, so no problem; but if it is not enough... :-(
> > On the other hand, shared objects need a number of
> > other things to be done, like initialization of the GOT, the resolution
> > of the fix ups relocations.
>
> I've been reading up on GOT (global offset table). To make sure I
> understand correctly: when the code in the shared object accesses data
> in the regular executable the following happens:
>
> 1. It will dereference a pointer to that data, this pointer is in the GOT.
>
> 2. It knows where the GOT is because it is located at a fixed offset
> from the code in the shared object.
>
> 3. For 1. to work, the loader (dlopen) needs to fill the GOT with the
> correct pointers.
Correct.
> Similar to this is the PLT (procedure linkage table), which takes care
> of routines instead of data.
Yes, there is also the mechanism to make the PLT works smoothly,
including R_386_JMPTAB relocations and the updating of the PLT entries;
I did include it under the "other things" above. My point was to show
that file-based mmap() is just a part of the dynamic linker process.
> Not only dynamically linked shared libraries have GOT and PLT, but the
> regular executable also has these tables.
Yes; and this is exactly why I advised about possible problems above.
Antoine