Shared Libraries: what is the linux equivalent of "dllimport" and "dllexport"
Noam K
I am trying to write a shared library which exports only some of its
functions.
In windows this is easily done by creating a DLL and using
__declspec(import) or (export).
Any ideas how this is done in Linux? are there different solutions for a.out
and ELF libraries?
Thanks,
Noam
Petter Reinholdtsen
> Any ideas how this is done in Linux? are there different solutions
> for a.out and ELF libraries?
You make all the functions not to be exported 'static'. All global
symbols are exported. I don't know any way to change this.
--
##> Petter Reinholdtsen <## | pe...@td.org.uit.no
Kaz Kylheku
>[Noam K]
>> Any ideas how this is done in Linux? are there different solutions
>> for a.out and ELF libraries?
>
>You make all the functions not to be exported 'static'. All global
>symbols are exported. I don't know any way to change this.
Then why bother posting? You could be reading the GNU info on
``ld'' to find out how this is done.
The following might be useful:
`--retain-symbols-file FILENAME'
Retain *only* the symbols listed in the file FILENAME, discarding
all others. FILENAME is simply a flat file, with one symbol name
per line. This option is especially useful in environments (such
as VxWorks) where a large global symbol table is accumulated
gradually, to conserve run-time memory.
`--retain-symbols-file' does *not* discard undefined symbols, or
symbols needed for relocations.
You may only specify `--retain-symbols-file' once in the command
line. It overrides `-s' and `-S'.
Jonas Utterstrom
k...@ashi.FootPrints.net (Kaz Kylheku) wrote:
> On 21 Aug 1999 11:11:14 GMT, Petter Reinholdtsen <pe...@td.org.uit.no>
wrote:
> >[Noam K]
> >> Any ideas how this is done in Linux? are there different solutions
> >> for a.out and ELF libraries?
> >
> >You make all the functions not to be exported 'static'. All global
> >symbols are exported. I don't know any way to change this.
>
> Then why bother posting? You could be reading the GNU info on
> ``ld'' to find out how this is done.
>
Better to say that you don't know than posting incorrect answers. And
using static functions is one way to limit the exported functions from a
shared library. But a very limited way.
> The following might be useful:
>
> `--retain-symbols-file FILENAME'
> Retain *only* the symbols listed in the file FILENAME, discarding
> all others. FILENAME is simply a flat file, with one symbol name
> per line. This option is especially useful in environments (such
> as VxWorks) where a large global symbol table is accumulated
> gradually, to conserve run-time memory.
This is useful only when you are performing static linking. What he
wants is to avoid exporting internal functions and objects in his SHARED
library. See my answer in the other thread for more info on this.
/Jonas U
Sent via Deja.com http://www.deja.com/
Share what you know. Learn what you don't.
Kaz Kylheku
>Better to say that you don't know than posting incorrect answers. And
>using static functions is one way to limit the exported functions from a
>shared library. But a very limited way.
You are right, I screwed this one up! Apologies for the misleading info.
The --retain-symbols-file option doesn't affect the retention of dynamic
symbols. These are still there, and can be viewed with ``nm -D''.
Noam K
My problem is still not solved however:
1. 'static' does not work since my source code is in more than one c file.
(As far as I know one file is scope of the static declaration so calls in
other c files that make my library will not see the functions as well)
2. using ld with`--retain-symbols-file' does only creates symbols defined in
a specified file. However, since `--retain-symbols-file' does *not*
discard undefined symbols, or symbols needed for relocations; if anyone
uses one of the 'hidden functions' (those not specified in the symbols file)
the linker will still find them...
Any more ideas?
Noam
Josh Stern
I don't know if you described the complete situation and its constraints -
however one thought that occurs to me is that if your code is either C++
or C that compiles as C++, then you may be able to make use of the
flexible namespace facilities in C++ to wrap portions of the code
with conflicting symbols in different namespaces.
- Josh
Philip Armstrong
Noam K <k...@netvision.net.il> wrote:
>Hi,
>
>I am trying to write a shared library which exports only some of its
>functions.
>
>In windows this is easily done by creating a DLL and using
>__declspec(import) or (export).
>
>and ELF libraries?
you might be able to remove symbols with judicious use of ar, nm and
possibly strip. Putting all your internal functions in a specific ELF
section so you can dump the symbols in one go might be the way to go.
I'm not sure you really want to do this though. It depends how those
functions are being called internally.
I have a question though. Why do you want to do this? The usual
pattern as far sa I can see is to put your public functions in the
header files for the library, and leave the private ones out. If
programmers are dumb enough to call functions in a library which
aren't defined in the headers then they are on their own :)
Looking at a few linux libraries seems to confirm this.
I have a feeling that the code set up by the __declspec function
attributes in the win32 environment is duplicated by the dynamic
linker under linux.
Looking at the documentation, it seems that __declspec(import) and
__declspec(export) are needed to ensure that those functions appear in
the global symbol table under win32. Under linux (and other unices)
the dynamic linker takes care of symbol references, and no explicit
'export' of a library function is necessary for a library to link
properly.
This implies that you're looking at things from the wrong point of
view. __declspec under windows is needed to ensure that functions
appear in the global symbol table. The lack of a 'declspec' may not
remove the function symbol from the dll itself, though it presumably
won't appear in the global symbol table for that process (this is
supposition incidentally---I don't have a windows machine available to
check)
So I think what you are trying to do may be based on a misapprehension
of what __declspec actually does under windows. (sticking my neck out
here :) --- I know *nothing* about win32 linking, and this is
inference from hints in the linux documentation.)
Does this make sense?
Phil Armstrong
--
nosig
Jonas Utterstrom
c9...@comlab.ox.ac.uk (Philip Armstrong) wrote:
> In article <7plpf8$qsd$1...@news.netvision.net.il>,
> Noam K <k...@netvision.net.il> wrote:
> >Hi,
> >
> >I am trying to write a shared library which exports only some of its
> >functions.
> >
> >In windows this is easily done by creating a DLL and using
> >__declspec(import) or (export).
> >
> >Any ideas how this is done in Linux? are there different solutions
for a.out
> >and ELF libraries?
>
>
> So I think what you are trying to do may be based on a misapprehension
> of what __declspec actually does under windows. (sticking my neck out
> here :) --- I know *nothing* about win32 linking, and this is
> inference from hints in the linux documentation.)
>
> Does this make sense?
Partly, but you may want export/import anyway of bloating reasons. I
explain this in a non-technical way, since I don't know the inner
workings of dlls and shared ELF libraries.
One of purposes of exporting functions in dll:s is to avoid exporting
internal datastructures that shouldn't be visible outside the shared
library. Of course one easy way of hiding the internal data is to
avoiding install the internal *.h files. And this also makes it
necessary to avoid includes to the internal h-files from the public
h-files.
But in C++ this is not enough, since C++ classes makes the shared
libraries substantially larger. An easy example of that is to first
compile a static C++ library and then compile it as a shared library.
Notice that the shared library is much larger.
This means that much of the library is bloated with export information.
In large applications such as Mozilla this is not acceptable. Imagine
the Linux heads wonder why Mozilla/Linux is twice as big as
Mozilla/Windows. Therefore, specifying what classes that you want to
have exported is important.
On the other hand, C would also benefit from this. One problem with
glibc is that developers have relied on internal functions, which have
been removed in newer versions(glibc-2.0 -> glibc-2.1). Those problems
would have been non-existant if exporting similar to Windows had been
used.
As I have said in another thread, exporting chosen functions is possible
if you are willing to use a development version of ld. But my advice is
to live with the code-bloat until a stable version arrives.
You don't need to export functions from shared libraries in Linux, they
are all exported (except from static functions). But if you have equal
internal functions in two different shared libraries, you're out of
luck.
Brian Poe
Under VC++ in Win32, the lack of __declspec(dllexport) indeed does
remove the function symbol from the dll.
-Brian
Kaz Kylheku
>I have a question though. Why do you want to do this? The usual
>pattern as far sa I can see is to put your public functions in the
>header files for the library, and leave the private ones out. If
Because libraries sometimes grow sophisticated enoguh that they have to be
broken into multiple translation units. Yet these modules have to communicate
with each other. These communication pathways are intended to be private to the
library, yet they are visible due to the unavoidable of identifiers with
external linkage.
These ``private externals'' pollute the namespace of the library user.
What's worse, the library user can override these symbols!
Suppose that in the library I have a function foo() that is external because
it's called across translation unit boundaries. If I don't remove its symbol
from the final release of the library, trouble can result. Not knowing about
foo, since it is not documented in my library's interface, the user might
accidentally write his own function foo(). Now my internal calls to foo()
resolve to the user's function! You could take steps to make sure foo can't
be overriden, but then the user's program won't link.
>programmers are dumb enough to call functions in a library which
>aren't defined in the headers then they are on their own :)
The problem is that you don't *want* these functions to be visible in the
public headers that are used by the library *user*. You want to export just the
interface, not the implementation details.
>Looking at a few linux libraries seems to confirm this.
The Linux libraries have extra freedoms given to them by the C language, POSIX,
and Unix Specification standards. They can hide their internal symbols by
using names that begin with two underscores, or an underscore and a capital
letter. ANSI C or POSIX programs can't use identifiers from these namespaces
without causing formally undefined behavior. So the problem is solved from the
point of view of the system implementor. It doesn't help someone who is writing
an independent library that isn't integrated into the OS.
Philip Armstrong
Jonas Utterstrom <ut...@my-deja.com> wrote:
>In article <7pomi8$9fv$1...@news.ox.ac.uk>,
> c9...@comlab.ox.ac.uk (Philip Armstrong) wrote:
>> Does this make sense?
>
>Partly, but you may want export/import anyway of bloating reasons. I
>explain this in a non-technical way, since I don't know the inner
>workings of dlls and shared ELF libraries.
true, I had forgotten this issue.
>As I have said in another thread, exporting chosen functions is possible
>if you are willing to use a development version of ld. But my advice is
>to live with the code-bloat until a stable version arrives.
And pester Ulrich Drepper + co to get it working properly in the
stable releases :)
>You don't need to export functions from shared libraries in Linux, they
>are all exported (except from static functions). But if you have equal
>internal functions in two different shared libraries, you're out of
>luck.
I'll have to try this, but isn't this is only a problem if you want to
call those functions in your code? Presumably internal (private)
functions called from code in the same libary don't need the symbol
table entry, as they can be called directly.
Anyway, this is getting well beyond my secure base of knowledge. I'll
write some test code and see what happens...
Phil
--
nosig
Josh Stern
>One of purposes of exporting functions in dll:s is to avoid exporting
>internal datastructures that shouldn't be visible outside the shared
>library. Of course one easy way of hiding the internal data is to
>avoiding install the internal *.h files. And this also makes it
>necessary to avoid includes to the internal h-files from the public
>h-files.
>
>But in C++ this is not enough, since C++ classes makes the shared
>libraries substantially larger. An easy example of that is to first
>compile a static C++ library and then compile it as a shared library.
>Notice that the shared library is much larger.
>
>This means that much of the library is bloated with export information.
>In large applications such as Mozilla this is not acceptable. Imagine
>the Linux heads wonder why Mozilla/Linux is twice as big as
>Mozilla/Windows. Therefore, specifying what classes that you want to
>have exported is important.
I really wonder about the merits of this generalization. In the
case of Mozilla, it uses a lot of shared libraries but those are
not built into the executable, which is extremely large to begin
with. Part of the reason for this is undoubtably due to the fact
that it is a very full-featured program, and part of the reason
is because it coded in a very careless and sloppy way. Also note
that for both C++ shared libraries and objects, the size of the
files on disk is not a good indication of the amount of run-time
resources that they use. It would be better to compile the library
as both static and shared, and then measure things like how
long the program takes to load and what 'top' says about its
size when it is running (multiple processes if that is relevant
to the application). Looking at the size of the files on
disk is primarily relevant to concerns about disk usage.
- Josh
Ulrich Weigand
>Because libraries sometimes grow sophisticated enoguh that they have to be
>broken into multiple translation units. Yet these modules have to communicate
>with each other. These communication pathways are intended to be private to the
>library, yet they are visible due to the unavoidable of identifiers with
>external linkage.
>These ``private externals'' pollute the namespace of the library user.
>What's worse, the library user can override these symbols!
Provided you use binutils version 2.9.1 or later, there is a way around
this: you can 'localize' symbols using the objcopy -L flag, which means
the symbol is treated from now on as if it were local (had been declared
'static' in C).
So, you link all your library objects together into one big .o file
(using ld -r or so), apply objcopy -L to localize all symbols you
don't want to export, and then create the shared object from the
resulting .o file ...
While this is somewhat convoluted, it should work :-/
--
Ulrich Weigand,
IMMD 1, Universitaet Erlangen-Nuernberg,
Martensstr. 3, D-91058 Erlangen, Phone: +49 9131 85-7688
Jonas Utterstrom
c9...@comlab.ox.ac.uk (Philip Armstrong) wrote:
> >As I have said in another thread, exporting chosen functions is
possible
> >if you are willing to use a development version of ld. But my advice
is
> >to live with the code-bloat until a stable version arrives.
>
> And pester Ulrich Drepper + co to get it working properly in the
> stable releases :)
>
Of course it won't do no harm testing it before it is stable of course.
I was thinking that people leaving Windows shouldn't have to deal with
alpha code, it might turn them off. ;)
> >You don't need to export functions from shared libraries in Linux,
they
> >are all exported (except from static functions). But if you have
equal
> >internal functions in two different shared libraries, you're out of
> >luck.
>
> I'll have to try this, but isn't this is only a problem if you want to
> call those functions in your code? Presumably internal (private)
> functions called from code in the same libary don't need the symbol
> table entry, as they can be called directly.
>
> Anyway, this is getting well beyond my secure base of knowledge. I'll
> write some test code and see what happens...
>
You got me uncertain so I made two shared libraries. No linking
problems, but the function from the first linked library was always
used.
Jonas Utterstrom
jst...@foshay.citilink.com (Josh Stern) wrote:
>
> I really wonder about the merits of this generalization. In the
> case of Mozilla, it uses a lot of shared libraries but those are
> not built into the executable, which is extremely large to begin
> with. Part of the reason for this is undoubtably due to the fact
> that it is a very full-featured program, and part of the reason
> is because it coded in a very careless and sloppy way. Also note
Badmouthing Mozilla developers is not a good way of explaining the size
of Mozilla. One of the reasons we don't have a new version of Mozilla is
because of their decision to rewrite the sloppy code.
> that for both C++ shared libraries and objects, the size of the
> files on disk is not a good indication of the amount of run-time
> resources that they use. It would be better to compile the library
> as both static and shared, and then measure things like how
> long the program takes to load and what 'top' says about its
> size when it is running (multiple processes if that is relevant
> to the application). Looking at the size of the files on
> disk is primarily relevant to concerns about disk usage.
>
If a library is twice as big, it takes longer to load. I do not claim
that I know how things are organized in memory. But there must be some
way to perform dynamic linking, and if this symbol-table is twice as
big, it probably also uses more memory. But that is only my gut feeling,
I have no proof.
I guess I have to read the ELF specifications to discuss this further.
;)
Josh Stern
>> I really wonder about the merits of this generalization. In the
>> case of Mozilla, it uses a lot of shared libraries but those are
>> not built into the executable, which is extremely large to begin
>> with. Part of the reason for this is undoubtably due to the fact
>> that it is a very full-featured program, and part of the reason
>> is because it coded in a very careless and sloppy way. Also note
>Badmouthing Mozilla developers is not a good way of explaining the size
>of Mozilla.
What I wrote is not tantamount to bad-mouthing Mozilla developers.
They may have had very important reasons (e.g. time pressure)
for coding Netscape in a careless and sloppy way. My hypothesis
of why Netscape is so big is a) it has a lot of features,
and b) it was coded with the priority of minimizing time to market
for a complex cross-platform app. Your "explanation" of why
the Netscape executable is so big - because C++ shared libraries
are much bigger than C shared libraries - was nonsensical bullshit.
>One of the reasons we don't have a new version of Mozilla is
>because of their decision to rewrite the sloppy code.
Indeed.
>> that for both C++ shared libraries and objects, the size of the
>> files on disk is not a good indication of the amount of run-time
>> resources that they use. It would be better to compile the library
>> as both static and shared, and then measure things like how
>> long the program takes to load and what 'top' says about its
>> size when it is running (multiple processes if that is relevant
>> to the application). Looking at the size of the files on
>> disk is primarily relevant to concerns about disk usage.
>If a library is twice as big, it takes longer to load.
The run time performance depends on what gets loaded when.
On Linux, resolution of references from a shared library
is 'lazy' by default, meaning that they are resolved as
needed during execution, so a particular module of code
that is not used is likely to remain un-read on disk.
Try 'man dlopen'. Or better yet, just adopt an empirical
approach to the question and experiment as I suggested
previously.
Also, for C++ libraries, code related to exception
handling is likely to be stored in different sections
and not read unless an exception gets thrown.
>I do not claim
>that I know how things are organized in memory. But there must be some
>way to perform dynamic linking, and if this symbol-table is twice as
>big, it probably also uses more memory. But that is only my gut feeling,
>I have no proof.
>I guess I have to read the ELF specifications to discuss this further.
>;)
Yes, and do the easy experiment with your own applications to
see what the effects of shared libraries and C++ really are. I
think you will be pleasantly surprised.
- Josh
Philip Armstrong
Kaz Kylheku <k...@ashi.FootPrints.net> wrote:
>On 22 Aug 1999 11:22:48 GMT, Philip Armstrong <c9...@comlab.ox.ac.uk> wrote:
>What's worse, the library user can override these symbols!
which can be a good thing. But not very often :) I presume you meant
that the library user can *accidently* override these symbols, which
would definately be a bad thing.
>Suppose that in the library I have a function foo() that is external because
>it's called across translation unit boundaries. If I don't remove its symbol
>from the final release of the library, trouble can result. Not knowing about
>foo, since it is not documented in my library's interface, the user might
>accidentally write his own function foo(). Now my internal calls to foo()
>resolve to the user's function! You could take steps to make sure foo can't
>be overriden, but then the user's program won't link.
In this situation, then its arguable that the latter is better than
the former. At least the programmer gets feedback (Though the linker ought
to complain about redefinition/re-use of symbol names anyway at
compile time. Does it?)
>>programmers are dumb enough to call functions in a library which
>>aren't defined in the headers then they are on their own :)
>
>The problem is that you don't *want* these functions to be visible in
>the public headers that are used by the library *user*. You want to
>export just the interface, not the implementation details.
obviously. And other posts have pointed out that this situation gets
really ugly when you have libraries written in c++ , I was just
pointing out what seemed to be the current 'state of the art' as it
were. ie people aren't hiding internal library symbols at this time.
>>Looking at a few linux libraries seems to confirm this.
>
>The Linux libraries have extra freedoms given to them by the C
>language, POSIX, and Unix Specification standards. They can hide
>their internal symbols by using names that begin with two
>underscores, or an underscore and a capital letter. ANSI C or POSIX
>programs can't use identifiers from these namespaces without causing
>formally undefined behavior. So the problem is solved from the point
>of view of the system implementor. It doesn't help someone who is
>writing an independent library that isn't integrated into the OS.
Well, since we were talking about win32 specific library building
details in the first place, I think its reasonable to assume the
availability of OS specific extensions...but this is a side issue.
Phil
--
nosig
Philip Armstrong
Jonas Utterstrom <ut...@my-deja.com> wrote:
>In article <7ppbt4$iqb$1...@news.ox.ac.uk>,
> c9...@comlab.ox.ac.uk (Philip Armstrong) wrote:
>Of course it won't do no harm testing it before it is stable of course.
>I was thinking that people leaving Windows shouldn't have to deal with
>alpha code, it might turn them off. ;)
probably not a good idea, no :)
>> >You don't need to export functions from shared libraries in Linux,
>> >they are all exported (except from static functions). But if you
>> >have equal internal functions in two different shared libraries,
>> >you're out of luck.
>> I'll have to try this, but isn't this is only a problem if you want to
>> call those functions in your code? Presumably internal (private)
>> functions called from code in the same libary don't need the symbol
>> table entry, as they can be called directly.
>>
>> Anyway, this is getting well beyond my secure base of knowledge. I'll
>> write some test code and see what happens...
>>
>
>You got me uncertain so I made two shared libraries. No linking
>problems, but the function from the first linked library was always
>used.
by code in both libraries?
Phil
--
nosig
Mark Hamstra
> To clarify;
>
> Under VC++ in Win32, the lack of __declspec(dllexport) indeed does
> remove the function symbol from the dll.
Meaning what, exactly? Yes, the symbol may no longer be exported,
but the instructions that implement the now-static function must
still exist in the DLL -- at least if the function is referenced
by other functions in the library. Or am I missing something?
--
Mark Hamstra
Bentley Systems, Inc.
Tristan Wibberley
The dynamic loader makes note of which symbols are needed as a library
says it needs it, if it comes across the definition of that symbol in
the same library or later it will resolve the symbol *and* *remember*
*it* *for* *later* *resolutions* - If the next library needs the same
symbol resolved, the dynamic linker sees that it already knows where to
find the definition and uses that.
I think you can force non-PIC behaviour for specific symbols when you
link the second library at compile time, but I'm not sure how.
--
Tristan Wibberley
Noam K
<7ppc7p$5...@faui11.informatik.uni-erlangen.de>...
>Provided you use binutils version 2.9.1 or later, there is a way around
>this: you can 'localize' symbols using the objcopy -L flag, which means
>the symbol is treated from now on as if it were local (had been declared
>'static' in C).
>
>So, you link all your library objects together into one big .o file
>(using ld -r or so), apply objcopy -L to localize all symbols you
>don't want to export, and then create the shared object from the
>resulting .o file ...
>
>While this is somewhat convoluted, it should work :-/
>
>--
> Ulrich Weigand,
> IMMD 1, Universitaet Erlangen-Nuernberg,
> Martensstr. 3, D-91058 Erlangen, Phone: +49 9131 85-7688
Thanks Ulrich, this works great!
Noam
Brian Poe
-Brian
Jonas Utterstrom
blo...@cus.umist.ac.uk wrote:
[...]
> > >You got me uncertain so I made two shared libraries. No linking
> > >problems, but the function from the first linked library was always
> > >used.
> >
> > by code in both libraries?
>
> The dynamic loader makes note of which symbols are needed as a library
> says it needs it, if it comes across the definition of that symbol in
> the same library or later it will resolve the symbol *and* *remember*
> *it* *for* *later* *resolutions* - If the next library needs the same
> symbol resolved, the dynamic linker sees that it already knows where
to
> find the definition and uses that.
>
> I think you can force non-PIC behaviour for specific symbols when you
> link the second library at compile time, but I'm not sure how.
Mr Tristan Wibberley is correct. I just wanted to add that in my case, I
used functions with different names in the exe. These two functions used
an internal function, with the same name in both libraries. And the
function from the first shared linked library was always used.
/JOnas U
Jonas Utterstrom
jst...@foshay.citilink.com (Josh Stern) wrote:
>
> What I wrote is not tantamount to bad-mouthing Mozilla developers.
> They may have had very important reasons (e.g. time pressure)
> for coding Netscape in a careless and sloppy way. My hypothesis
> of why Netscape is so big is a) it has a lot of features,
> and b) it was coded with the priority of minimizing time to market
> for a complex cross-platform app. Your "explanation" of why
> the Netscape executable is so big - because C++ shared libraries
> are much bigger than C shared libraries - was nonsensical bullshit.
>
I am a C++ programmer so I am not trying to fuel the C vs C++ flamewar.
I just compare with the size of Windows dlls. And the fact is that i.e.
Mozilla is twice as big in Linux.
>
> The run time performance depends on what gets loaded when.
> On Linux, resolution of references from a shared library
> is 'lazy' by default, meaning that they are resolved as
> needed during execution, so a particular module of code
> that is not used is likely to remain un-read on disk.
> Try 'man dlopen'. Or better yet, just adopt an empirical
> approach to the question and experiment as I suggested
> previously.
>
I do not really see your point here. One of the good things with shared
libraries is that they aren't loaded until they are actually used. But
when they are used, also the internal classes used by the public
interface will be loaded. And they will pollute the global symbol-table
for the dynamic linker.
>
> Yes, and do the easy experiment with your own applications to
> see what the effects of shared libraries and C++ really are. I
> think you will be pleasantly surprised.
>
Unless someone with more technical ELF merits steps forward I have to
get back to this issue later.
/Jonas U
Josh Stern
>> My hypothesis
>> of why Netscape is so big is a) it has a lot of features,
>> and b) it was coded with the priority of minimizing time to market
>> for a complex cross-platform app. Your "explanation" of why
>> the Netscape executable is so big - because C++ shared libraries
>> are much bigger than C shared libraries - was nonsensical bullshit.
>I am a C++ programmer so I am not trying to fuel the C vs C++ flamewar.
>I just compare with the size of Windows dlls. And the fact is that i.e.
>Mozilla is twice as big in Linux.
There is something missing here. I can't figure out if it
is a breakdown in logic, English comprehension, or basic
understanding of what a shared library is. So let's review:
On Linux, with Communicator 4.6, we have the *executable* size, which
*does not* include the shared libraries that it dynamically links
with, coming in at approximately 14 MB. This is big, but it is
*not* measuring the size of any shared libraries. If you want to
see which shared libraries it links with you look at the output
of the command ldd `which netscape`. There you see a bunch of
different libs that do different things, but these libraries
are not distributed with netscape. They perform various other
collections of functions having nothing to do with Netscape,
so their size is irrelevant to the discussion. So the first
question for you to answer is "What size are you talking about?"
and the second question is "If this size is different on
Linux and Windows, how can it be attributed to a difference
between C and C++ dynamic library sizes?" The second question
completely baffles me, not only because the first hasn't
been answered, but also because you haven't described any
reason to believe that different percentages of C and C++
are used in the Linux and Windows versions. So my response
is not about defending C++ - the point is rather that you
are simply not making any sense. Whatever you seem to
be claiming is not only not correct, it's not even
incorrect! There is some kind of failure in the presuppositions
of your whole analysis.
>> The run time performance depends on what gets loaded when.
>> On Linux, resolution of references from a shared library
>> is 'lazy' by default, meaning that they are resolved as
>> needed during execution, so a particular module of code
>> that is not used is likely to remain un-read on disk.
>> Try 'man dlopen'. Or better yet, just adopt an empirical
>> approach to the question and experiment as I suggested
>> previously.
>I do not really see your point here. One of the good things with shared
>libraries is that they aren't loaded until they are actually used. But
>when they are used, also the internal classes used by the public
>interface will be loaded.
Read the man page again. It clearly says that
when lazy evaluation is used (which is the default) then these
references are resolved as needed during execution.
>And they will pollute the global symbol-table
>for the dynamic linker.
The namespace pollution issue is orthogonal to the size
one. In an earlier post I suggested that C++ namespaces
could help with the original poster's problem of avoiding
conflicts and polluting the global namespace. Other
useful suggestions for linker commands were posted by
other people.
- Josh
Peter Samuelson
> I am a C++ programmer so I am not trying to fuel the C vs C++
> flamewar. I just compare with the size of Windows dlls. And the fact
> is that i.e. Mozilla is twice as big in Linux.
Is that "Mozilla" or "Communicator"? If you're talking about
Communicator, recall that the Linux version has a copy of Motif
statically linked in, which could account for a great deal. Mozilla,
of course, won't have this problem....
--
Peter Samuelson
<sampo.creighton.edu!psamuels>
John Hughes
> If a library is twice as big, it takes longer to load.
What do you mean by big? ls does not show you how much code/data the
library contains.
Shared libraries are not loaded. The pages that are used are loaded.
Having a large symbol table could slow down the initial lookup of
symbols.
--
John Hughes <jo...@Calva.COM>,
Atlantic Technologies Inc. Tel: +33-1-4313-3131
66 rue du Moulin de la Pointe, Fax: +33-1-4313-3139
75013 PARIS.
Mark Hamstra
> The namespace pollution issue is orthogonal to the size
> one.
No, it's not. Reducing namespace pollution by removing
symbols from a shared library directly results in decreased
size of the .dynsym and .dynstr sections, and thus of the
shared library as a whole -- and not by an insignificant
amount if the reports from the mozilla developers are
accurate.
Josh Stern
Well okay, now we've changed what is actually shared by
the shared library (as opposed to just what it is called
in the global namespace). Incidentally, does "directly"
in the text above mean using objcopy?
- Josh
Mark Hamstra
> Mark Hamstra <mark.h...@bentley.com> wrote:
> >jst...@foshay.citilink.com (Josh Stern) writes:
>
> >> The namespace pollution issue is orthogonal to the size
> >> one.
>
> >No, it's not. Reducing namespace pollution by removing
> >symbols from a shared library directly results in decreased
> >size of the .dynsym and .dynstr sections, and thus of the
> >shared library as a whole -- and not by an insignificant
> >amount if the reports from the mozilla developers are
> >accurate.
>
> Well okay, now we've changed what is actually shared by
> the shared library (as opposed to just what it is called
> in the global namespace).
Not sure exactly what you mean here. What I am talking about
is leaving the internal implementation details of the library
unchanged while removing the symbols from the global namespace
(and thus from the .dynsym or .dynstr section), effectively
paring down the public interface of the library.
> Incidentally, does "directly"
> in the text above mean using objcopy?
From the previous discussion in this thread, that would appear
to be one way to do it; using a bleeding edge ld and a new
__attribute__(()) would appear to be another.
Jonas Utterstrom
First off all, could you please keep the insults out? It only makes you
look stupid.
No, I was not talking about Communicator 4.6, I was talking about
Mozilla (www.mozilla.org). Talking about Communicator is a non-issue
since they solved the shared library problem with static linking there.
Only a fractional part of Mozilla is platform specific code, so it is
quite easy to compare the sizes of dlls and Linux shared libraries. If
you had bothered to follow the link I gave in the other thread, this
would have been a non-issue.
Josh Stern
>jst...@foshay.citilink.com (Josh Stern) writes:
>> Mark Hamstra <mark.h...@bentley.com> wrote:
>> >jst...@foshay.citilink.com (Josh Stern) writes:
>> >> The namespace pollution issue is orthogonal to the size
>> >> one.
>> >No, it's not. Reducing namespace pollution by removing
>> >symbols from a shared library directly results in decreased
>> >size of the .dynsym and .dynstr sections, and thus of the
>> >shared library as a whole -- and not by an insignificant
>> >amount if the reports from the mozilla developers are
>> >accurate.
>> Well okay, now we've changed what is actually shared by
>> the shared library (as opposed to just what it is called
>> in the global namespace).
>Not sure exactly what you mean here.
I meant that those symbols can no longer be referenced be
linked to directly by code that is external to the library
using any name. In that sense they are no longer "shared"
and the API functionality of the library in addition to its
name has changed. That's probably all good and desirable in
this case, but just a bit different from what people usually
mean when they talk about avoiding namespace pollution.
- Josh
Noam K
>
>>Because libraries sometimes grow sophisticated enoguh that they have to be
>>broken into multiple translation units. Yet these modules have to
communicate
>>with each other. These communication pathways are intended to be private
to the
>>library, yet they are visible due to the unavoidable of identifiers with
>>external linkage.
>
>>What's worse, the library user can override these symbols!
>
>this: you can 'localize' symbols using the objcopy -L flag, which means
>the symbol is treated from now on as if it were local (had been declared
>'static' in C).
>
>So, you link all your library objects together into one big .o file
>(using ld -r or so), apply objcopy -L to localize all symbols you
>don't want to export, and then create the shared object from the
>resulting .o file ...
>
>While this is somewhat convoluted, it should work :-/
>
>--
> Ulrich Weigand,
> IMMD 1, Universitaet Erlangen-Nuernberg,
> Martensstr. 3, D-91058 Erlangen, Phone: +49 9131 85-7688
Thanks Ulrich, this solution works great!
Noam
Josh Stern
>First off all, could you please keep the insults out? It only makes you
>look stupid.
Sorry if you felt insulted. I still think there is a lot of information
or misunderstanding in what you posted and the latest post only
clears up a part of it.
>No, I was not talking about Communicator 4.6, I was talking about
>Mozilla (www.mozilla.org). Talking about Communicator is a non-issue
>since they solved the shared library problem with static linking there.
Communicator 4.6 does link to a lot of shared libraries.
But anyway, this clarification helps partly.
Now where does the contrast between shared
libraries based on C and C++ come into it?? Could you post the
sizes of the libraries that you think represent the same code
on Linux and on Windows and say what percentage C and C++ you
estimate them to be? Also, do the Windows versions support
exceptions, and if not, are exceptions turned off on the Linux
versions?
-Josh
Jonas Utterstrom
jst...@foshay.citilink.com (Josh Stern) wrote:
> Communicator 4.6 does link to a lot of shared libraries.
> But anyway, this clarification helps partly.
> Now where does the contrast between shared
> libraries based on C and C++ come into it?? Could you post the
> sizes of the libraries that you think represent the same code
> on Linux and on Windows and say what percentage C and C++ you
> estimate them to be? Also, do the Windows versions support
> exceptions, and if not, are exceptions turned off on the Linux
> versions?
Well hold your breath then, Mozilla uses 81 dlls or 88 so-files. Most of
them are small and looks more like plugins.
Here are some comparisons from the Mozilla M8 release:
The Linux version is compiled without rtti or exceptions. Especially
exceptions is an extremely bloat factor. The Linux binaries are also
stripped.
I know nothing of how the Windows version was compiled.
100% C++:
3637416 libraptorhtml.so
552236 librdf.so
424516 libxpcom.so
1204912 raptorhtml.dll
251824 rdf.dll
246336 xpcom.dll
And a 100% C library comparison:
41952 zlib.dll
53904 libz.so.1.1.3
The numbers speak for themselves.
For some more information, check this link out (probably broken but cut &
paste is easy):
http://bx6.deja.com/viewthread.xp?AN=490379424.1&search=thread&svcclass=dnser
ver&ST=PS&CONTEXT=935244203.987693076&HIT_CONTEXT=935244203.987693076&HIT_NUM
=3&recnum=%3c3767FCF...@mozilla.org%3e%231/2&group=netscape.public.moz
illa.performance.size-matters&frpage=getdoc.xp&back=clarinet
Tristan Wibberley
> I
> used functions with different names in the exe. These two functions used
> an internal function, with the same name in both libraries. And the
> function from the first shared linked library was always used.
Isn't there a linker option that lets you instruct the run-time linker
to discard the pointer to the definition of a particular symbol between
so files.
eg (pseudo command line - I can't find my documentation for ld :(,
link -o myexe myexe.o -llibA.so -discard foo -llibB.so
so the dynamic linker gets the correct definition of foo for A, drops it
for future resolutions, and gets the correct definition for B? Or is
this only for static linking?
I can imagine complex situations getting ugly though.
--
Tristan Wibberley
Peter Mutsaers
BP> Under VC++ in Win32, the lack of __declspec(dllexport) indeed
BP> does remove the function symbol from the dll.
AIX has a similar thing with its shared libraries: you have to list
every symbol that you intend to export. Personally I think that is a
horror.
In 9 out of 10 cases, using static is good enough (or better yet, when
using C++ you put all non-exported stuff in private and exported in
public sections). These are the generic language methods for selecting
whether to export something or not, and IMO shared libraries shouldn't
use too much extra magic from normal programs.
OK, C's way (static) is somewhat limited and crude; you have to
organize you program over source files so that everything which much
see each other is in a single file (module), that's just the way it is
with C. If you need more fine-grain control, use C++ classes.
--
Peter Mutsaers | Abcoude (Utrecht), | Trust me, I know
p...@xs4all.nl | the Netherlands | what I'm doing.
Kelly Burkhart
> Hi,
>
> I am trying to write a shared library which exports only some of its
> functions.
>
> In windows this is easily done by creating a DLL and using
> __declspec(import) or (export).
>
> Any ideas how this is done in Linux? are there different solutions for a.out
> and ELF libraries?
>
> Thanks,
> Noam
>
>
>
You should use a version script:
slib1.h -------------------------------------------------
extern int slib_i;
int slib00();
int slib01();
int slib02();
slib1.c --------------------------------------------------
#include <stdio.h>
int slib_i=5;
int slib00() { printf("slib00\n"); }
int slib01() { printf("slib01\n"); }
int slib02() { printf("slib02\n"); }
main.c -----------------------------------------------------
#include <stdio.h>
#include "slib1.h"
int main()
{
int i = slib_i;
slib00();
slib01();
slib02();
}
libslib1.ver ------------------------------------------------
#
# Export file for libslib1
#
libslib1.so.1.0 {
global:
slib_i;
slib00;
slib01;
slib02;
local:
*;
};
Makefile ---------------------------------------------------
main: main.c libslib1.so.1.0
gcc -o main main.c -L. -lslib1
libslib1.so.1.0 : slib1.c libslib1.ver
gcc -c -fPIC slib1.c
gcc -shared -Wl,-soname,libslib1.so.1,-version-script=libslib1.ver -o libslib1.so.1.0 slib1.o
ln -sf libslib1.so.1.0 libslib1.so.1
ln -sf libslib1.so.1.0 libslib1.so
clean:
rm -f *.o *.a *.so *.so.* *~ main
-------------------------------------------------------------
To demonstrate, try removing a symbol from the version script and rebuilding.
--
Kelly R. Burkhart
kb...@sky.net
[The litigation] industry was, of course, up and running before the
tobacco litigation, but that taught lawyers just how lucrative it
could be to blame individuals' foolishness on, say, Joe Camel.
-- George F. Will
Jens-Uwe Mager
>>> "BP" == Brian Poe <bp...@flash.net> writes:
>
> BP> Under VC++ in Win32, the lack of __declspec(dllexport) indeed
> BP> does remove the function symbol from the dll.
>
>AIX has a similar thing with its shared libraries: you have to list
>every symbol that you intend to export. Personally I think that is a
>horror.
It works very nicely, and it avoids the name space pollution you get
otherwise. Fortunately the HP-UX and IRIX linkers are able to do that
as well. With larger libraries you simply cannot avoid global symbols
so it is nice if you can hide them. It also helps finding if someone
does use functions outside of the defined API, this would help with
problems like these that showed up with the glibc 2.0 to 2.1 transition.
--
Jens-Uwe Mager <pgp-mailto:62CFDB25>
David Schwartz
Ulrich Weigand wrote:
> So, you link all your library objects together into one big .o file
> (using ld -r or so), apply objcopy -L to localize all symbols you
> don't want to export, and then create the shared object from the
> resulting .o file ...
>
> While this is somewhat convoluted, it should work :-/
A version of '-K' that read the symbols to keep from a file would be
handy. That way you could just make a list of all the symbols that you
wanted to be externally visible.
DS
Peter Samuelson
> A version of '-K' that read the symbols to keep from a file would be
> handy. That way you could just make a list of all the symbols that
> you wanted to be externally visible.
Kludge it:
objcopy `sed 's/^/-K /' syms.exp` foo.o bar.o
Which of course won't work if you have enough symbols to hit
command-line-length limits.
--
Peter Samuelson
<sampo.creighton.edu!psamuels>
Andreas Jaeger
> Ulrich Weigand wrote:
>> So, you link all your library objects together into one big .o file
>> (using ld -r or so), apply objcopy -L to localize all symbols you
>> don't want to export, and then create the shared object from the
>> resulting .o file ...
>>
>> While this is somewhat convoluted, it should work :-/
> A version of '-K' that read the symbols to keep from a file would be
> handy. That way you could just make a list of all the symbols that you
> wanted to be externally visible.
Isn't this possible with version-script? Have a look at glibc 2.1.x.
Andreas
--
Andreas Jaeger
SuSE Labs a...@suse.de
private a...@arthur.rhein-neckar.de
David Schwartz
Andreas Jaeger wrote:
> > A version of '-K' that read the symbols to keep from a file would be
> > handy. That way you could just make a list of all the symbols that you
> > wanted to be externally visible.
>
> Isn't this possible with version-script? Have a look at glibc 2.1.x.
Yes, it is. That's actually pretty handy.
DS