Should shared _r libraries depend on threading library?
Peter
ldd -r shows not undefined symbols for libmysqlclient_r, but shows
some for libldap_r:
/usr/lib $ ldd -r libldap_r.so
undefined symbol: pthread_create (./libldap_r.so)
[snip other undefined symbols]
linux-gate.so.1 => (0x00dfc000)
[snip other dependent libs]
I've checked this on fedora and gentoo linux. I found mail on ldap
mailing list about this issue [1] but developers seems to ignore that
and thus I'm not sure now, is libldap_r.so broken?
I have not checked with ldap_r.so but I have another library with the
same issues and I know that it *is* possible to link user applications
against libldap_r.so so previously I though that this _r libraries are
intentionally left without dependency on libpthread to allow users to
link either threaded application with -pthread and thus use libpthread
or link without libpthread and thus use glibc stubs but in reality
seems that is my imagination. Right? And if so the difference between
_r library and library without this suffix is that first is threadsafe
(and thus it is possible to call their functions from different
threads without special locking) while the latter is not threadsafe
but a bit faster. Again right?
Sorry if I'm asking completely trivial question but I've spent 3 full
days searching google and learning on how we should link with thread
libraries on different architectures but have not found clear answer
on this questions.
[1] http://www.openldap.org/lists/openldap-devel/200305/msg00067.html
Thank you very much in advance,
Peter.
David Schwartz
> I've checked this on fedora and gentoo linux. I found mail on ldap
> mailing list about this issue [1] but developers seems to ignore that
> and thus I'm not sure now, is libldap_r.so broken?
If the library needs to create a thread, how should it do it other
than by calling pthread_create? Why is that broken?
> I have not checked with ldap_r.so but I have another library with the
> same issues and I know that it *is* possible to link user applications
> against libldap_r.so so previously I though that this _r libraries are
> intentionally left without dependency on libpthread to allow users to
> link either threaded application with -pthread and thus use libpthread
> or link without libpthread and thus use glibc stubs but in reality
> seems that is my imagination. Right?
I'm not sure I understand the question. Having libraries that can work
with or without threading support is a platform-specific thing that
may or may not work on various platforms in various different ways.
For portable code, if it supports threads, it must be compiled with
the appropriate flags (usually -pthread).
Glibc stubs and having a 'libpthread' that implements POSIX threads
are platform implementation details. Without knowing your platform,
there's really not much we can say about them.
> And if so the difference between
> _r library and library without this suffix is that first is threadsafe
> (and thus it is possible to call their functions from different
> threads without special locking) while the latter is not threadsafe
> but a bit faster. Again right?
Ironically, the _r libraries are usually faster on modern hardware.
> Sorry if I'm asking completely trivial question but I've spent 3 full
> days searching google and learning on how we should link with thread
> libraries on different architectures but have not found clear answer
> on this questions.
Almost every modern platform supports '-pthread'. All you have to do
is pass that flag to *every* compile and link that you do. If that
doesn't do us, at least tell us what platform you are using.
You should not have to dive into these implementation details. It's
not clear why you are bothering when you don't seem to have any errors
or problems. If everything works perfectly, why make trouble?
DS
Peter
library has undefined symbols is it broken or not? In my case I have
libldap_r.so which have calls to pthread_create but does not have
dependency on libpthread. Here is the full output of ldd -r:
$ ldd -r /usr/lib/libldap_r.so
linux-gate.so.1 => (0xb7f7f000)
liblber-2.3.so.0 => /usr/lib/liblber-2.3.so.0 (0xb7f14000)
libresolv.so.2 => /lib/libresolv.so.2 (0xb7f01000)
libssl.so.0.9.8 => /usr/lib/libssl.so.0.9.8 (0xb7ec2000)
libcrypto.so.0.9.8 => /usr/lib/libcrypto.so.0.9.8 (0xb7d82000)
libc.so.6 => /lib/libc.so.6 (0xb7c37000)
libdl.so.2 => /lib/libdl.so.2 (0xb7c33000)
/lib/ld-linux.so.2 (0x80000000)
undefined symbol: pthread_mutex_trylock (/usr/lib/libldap_r.so)
undefined symbol: pthread_getconcurrency (/usr/lib/
libldap_r.so)
undefined symbol: pthread_attr_setstacksize (/usr/lib/
libldap_r.so)
undefined symbol: pthread_kill (/usr/lib/libldap_r.so)
undefined symbol: pthread_create (/usr/lib/libldap_r.so)
undefined symbol: pthread_setconcurrency (/usr/lib/
libldap_r.so)
undefined symbol: pthread_join (/usr/lib/libldap_r.so)
So you see that library uses pthread_create but do not have dependency
libpthread.so.0 => /lib/libpthread.so.0. Is such library broken or
does there exist special cases when such library with undefined
symbols is required? At least it is possible to build programs against
such library and use them.
On 29 апр, 14:53, David Schwartz <dav...@webmaster.com> wrote:
> If the library needs to create a thread, how should it do it other
> than by calling pthread_create? Why is that broken?
The library itself does not depend on the pthread library but later
application could be linked with library and with libptreads. Linkage
succeeds and program works. But you asked me what problem I have then.
So with LD_FLAGS="-Wl,--as-needed" and only for C++ library and
application linkage stops on unresolved symbols. Of course I use -
pthread everywhere. For C applications this works. I've looked at
verbose output of gcc and found that gcc compared to g++ have "--no-as-
needed -lpthread -lc -lgcc --as-needed" somewhere after libs. But I do
not understand is this a bug in gcc and then what is the purpose of
library with undefined symbols or is this bug in library build process
(and may be libtool) similar to [1]?
> Ironically, the _r libraries are usually faster on modern hardware.
Does ironically negates the other part of the sentence? So did you
mean that there is no interest in not having threads at all from
performance point of view? But could you then comment why there are
two versions of libmysqlclient with and without _r?
I do not know what you mean by platform but I have linux-2.6.19, i686,
libc-2.5 compiled with NPTL. Same observations I've reproduced in
Fedora Core release 4.
[1] http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25460
Peter.
Paul Pluzhnikov
> Ironically, the _r libraries are usually faster on modern hardware.
That statement is likely bogus.
While it is true that the overhead of an (almost) empty function
call is pretty minor on modern processors, I can't imagine how *not*
calling pthread_mutex_{un}lock at all can be slower then calling it.
Would you care to explain how 'libfoo_r' library could be faster
than 'libfoo', if the only difference between the two is that
libfoo_r uses locking and libfoo assumes single-threaded execution
(and thus does not need any locking)?
> Almost every modern platform supports '-pthread'.
Not literally. Every modern platform has an equivalent, but it may
be called '-mt', or you might need to compile with 'cc_r', etc. etc.
Cheers,
--
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.
Paul Pluzhnikov
> I reformulate the question: When shared
> library has undefined symbols is it broken or not?
Not necessarily.
Consider a library that must create a new thread (since libldap_r
depends on pthread_create, it probably does create new threads).
Consider what happens when you dlopen() such library from a process
that was not linked with libpthread.so (IOW, a single-threaded
process).
Very few operating systems support "process can become multi-threaded
at any moment" operation. Most libc implementations (glibc included)
test whether threads are present at startup, and if not, use
static globals (and turn off locking altogether). Such an
implementation will fail with races galore if the process later
creates multiple threads.
Now, if libldap_r was linked with libpthread, then you *could*
dlopen() it from a single-threaded process, and call it, it would
fail (potentially with quite unobvious symtoms).
OTOH, if it is not linked with libpthread, then dlopen() will fail
with unresolved pthread_create, etc. This is (obviously) a much
preferable failure mode.
...
> But you asked me what problem I have then.
> So with LD_FLAGS="-Wl,--as-needed" and only for C++ library and
> application linkage stops on unresolved symbols.
Fails with *which* unresolved symbols?
Presumably some 'pthread_*' ones. That would imply that you link the
main exe (or whichever link is actually failing) without '-pthread',
but that contradicts your next statement:
> Of course I use -pthread everywhere.
Please provide further details about the exact failure (error
message, link command line).
David Schwartz
> Thank you for answer, David. I reformulate the question: When shared
> library has undefined symbols is it broken or not?
There is no way to tell. It depends on the platform.
> In my case I have
> libldap_r.so which have calls to pthread_create but does not have
> dependency on libpthread. Here is the full output of ldd -r:
I still don't understand why you think that's a problem.
> $ ldd -r /usr/lib/libldap_r.so
> linux-gate.so.1 => (0xb7f7f000)
> liblber-2.3.so.0 => /usr/lib/liblber-2.3.so.0 (0xb7f14000)
> libresolv.so.2 => /lib/libresolv.so.2 (0xb7f01000)
> libssl.so.0.9.8 => /usr/lib/libssl.so.0.9.8 (0xb7ec2000)
> libcrypto.so.0.9.8 => /usr/lib/libcrypto.so.0.9.8 (0xb7d82000)
> libc.so.6 => /lib/libc.so.6 (0xb7c37000)
> libdl.so.2 => /lib/libdl.so.2 (0xb7c33000)
> /lib/ld-linux.so.2 (0x80000000)
> undefined symbol: pthread_mutex_trylock (/usr/lib/libldap_r.so)
> undefined symbol: pthread_getconcurrency (/usr/lib/
> libldap_r.so)
> undefined symbol: pthread_attr_setstacksize (/usr/lib/
> libldap_r.so)
> undefined symbol: pthread_kill (/usr/lib/libldap_r.so)
> undefined symbol: pthread_create (/usr/lib/libldap_r.so)
> undefined symbol: pthread_setconcurrency (/usr/lib/
> libldap_r.so)
> undefined symbol: pthread_join (/usr/lib/libldap_r.so)
>
> So you see that library uses pthread_create but do not have dependency
> libpthread.so.0 => /lib/libpthread.so.0. Is such library broken or
> does there exist special cases when such library with undefined
> symbols is required? At least it is possible to build programs against
> such library and use them.
Are you following your platform's rules for how to compile and link
pthreads code? You must do so on every single compile and link command
for pthreads code.
DS
David Schwartz
wrote:
> David Schwartz <dav...@webmaster.com> writes:
> > Ironically, the _r libraries are usually faster on modern hardware.
>
> That statement is likely bogus.
Not at all.
> While it is true that the overhead of an (almost) empty function
> call is pretty minor on modern processors, I can't imagine how *not*
> calling pthread_mutex_{un}lock at all can be slower then calling it.
Consider the following typical example. A function call requires some
work to be done and then a network protocol to be shut down in an
organized way. The single-threaded library cannot return to the caller
until the network protocol is completely shut down because it has no
way to assure it will ever get control back. The multi-threaded
library can dispatch a thread to handle the shutdown and return to the
caller immediately. The result is that the multi-threaded library is
faster.
> Would you care to explain how 'libfoo_r' library could be faster
> than 'libfoo', if the only difference between the two is that
> libfoo_r uses locking and libfoo assumes single-threaded execution
> (and thus does not need any locking)?
Since libfoo_r is a multi-threaded library, it can implement functions
such that there is less blocking.
DS
Paul Pluzhnikov
> On Apr 29, 9:39 am, Paul Pluzhnikov <ppluzhnikov-...@charter.net>
> wrote:
>> David Schwartz <dav...@webmaster.com> writes:
>> > Ironically, the _r libraries are usually faster on modern hardware.
>>
>> That statement is likely bogus.
...
> Consider the following typical example. A function call requires some
> work to be done and then a network protocol to be shut down in an
> organized way.
This may be a typical example in your area of expertise, but I assert
that over 90% of libfoo_r libraries are *not* written that way [1] [2].
The case above constitues exactly 0% of libfoo_r libraries that I
have ever seen.
> Since libfoo_r is a multi-threaded library, it can implement functions
> such that there is less blocking.
That may sometimes be possible, and if it is possible, it has nothing
to do with modern hardware -- there would be just as much "less
blocking" on ancient hardware.
Cheers,
[1]. But 87% percent of all statistics are made up.
[2]. Find 2 open-source libraries that work that way, and I'll
be impressed. Find 10 and I'll bow to your wisdom.
Peter
> Peter <volkov.pe...@gmail.com> writes:
> > I reformulate the question: When shared
> > library has undefined symbols is it broken or not?
>
> Not necessarily.
Thank you. That's what I wanted to know.
> OTOH, if it is not linked with libpthread, then dlopen() will fail
> with unresolved pthread_create, etc. This is (obviously) a much
> preferable failure mode.
But why then this behavior is not default? Most libraries linux are
threaded and reference on libpthreads is included and thus dl_open()
succeeds...
> Please provide further details about the exact failure (error
> message, link command line).
This is how I link the objects which will be included in library:
libtool --mode=compile i686-pc-linux-gnu-g++ -c -g -O2 -pthread
sync.cpp
i686-pc-linux-gnu-g++ -c -g -O2 -pthread sync.cpp -fPIC -DPIC -
o .libs/sync.o
i686-pc-linux-gnu-g++ -c -g -O2 -pthread sync.cpp -o sync.o >/dev/
null 2>&1
This is how library is linked itself:
libtool --mode=link i686-pc-linux-gnu-g++ -o libgigabase_r.la class.lo
compiler.lo database.lo replicator.lo hashtab.lo file.lo symtab.lo
btree.lo rtree.lo cursor.lo query.lo pagepool.lo wwwapi.lo unisock.lo
blob.lo container.lo exception.lo localcli.lo sync.lo -pthread -rpath /
usr/lib -version-info 2
i686-pc-linux-gnu-g++ -shared -nostdlib /usr/lib/gcc/i686-pc-linux-gnu/
4.1.2/../../../crti.o /usr/lib/gcc/i686-pc-linux-gnu/4.1.2/
crtbeginS.o .libs/class.o .libs/compiler.o .libs/database.o .libs/
replicator.o .libs/hashtab.o .libs/file.o .libs/symtab.o .libs/
btree.o .libs/rtree.o .libs/cursor.o .libs/query.o .libs/
pagepool.o .libs/wwwapi.o .libs/unisock.o .libs/blob.o .libs/
container.o .libs/exception.o .libs/localcli.o .libs/sync.o -L/usr/
lib/gcc/i686-pc-linux-gnu/4.1.2 -L/usr/lib/gcc/i686-pc-linux-gnu/
4.1.2/../../../../i686-pc-linux-gnu/lib -L/usr/lib/gcc/i686-pc-linux-
gnu/4.1.2/../../.. -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/i686-pc-linux-
gnu/4.1.2/crtendS.o /usr/lib/gcc/i686-pc-linux-gnu/4.1.2/../../../
crtn.o -pthread -Wl,-soname -Wl,libgigabase_r.so.2 -o .libs/
libgigabase_r.so.2.0.0
(cd .libs && rm -f libgigabase_r.so.2 && ln -s libgigabase_r.so.2.0.0
libgigabase_r.so.2)
(cd .libs && rm -f libgigabase_r.so && ln -s libgigabase_r.so.2.0.0
libgigabase_r.so)
i686-pc-linux-gnu-ar cru .libs/libgigabase_r.a class.o compiler.o
database.o replicator.o hashtab.o file.o symtab.o btree.o rtree.o
cursor.o query.o pagepool.o wwwapi.o unisock.o blob.o container.o
exception.o localcli.o sync.o
i686-pc-linux-gnu-ranlib .libs/libgigabase_r.a
creating libgigabase_r.la
(cd .libs && rm -f libgigabase_r.la && ln -s ../libgigabase_r.la
libgigabase_r.la)
Then subsql program links against libgigabase_r.so successfully:
i686-pc-linux-gnu-g++ -c -g -O2 -pthread subsql.cpp
i686-pc-linux-gnu-g++ -c -g -O2 -pthread server.cpp
libtool --mode=link i686-pc-linux-gnu-g++ -Wl,--as-needed -pthread -o
subsql subsql.o server.o libgigabase_r.la
i686-pc-linux-gnu-g++ -Wl,--as-needed -pthread -o .libs/subsql
subsql.o server.o ./.libs/libgigabase_r.so
creating subsql
And next program fails:
i686-pc-linux-gnu-g++ -c -g -O2 -pthread guess.cpp
libtool --mode=link i686-pc-linux-gnu-g++ -Wl,--as-needed -pthread -o
guess guess.o libgigabase_r.la
i686-pc-linux-gnu-g++ -Wl,--as-needed -pthread -o .libs/guess
guess.o ./.libs/libgigabase_r.so
./.libs/libgigabase_r.so: undefined reference to `pthread_key_create'
./.libs/libgigabase_r.so: undefined reference to `pthread_getspecific'
./.libs/libgigabase_r.so: undefined reference to `pthread_create'
./.libs/libgigabase_r.so: undefined reference to `pthread_key_delete'
./.libs/libgigabase_r.so: undefined reference to `pthread_detach'
./.libs/libgigabase_r.so: undefined reference to `pthread_setspecific'
./.libs/libgigabase_r.so: undefined reference to `pthread_join'
collect2: ld returned 1 exit status
make: *** [guess] Ошибка 1
The differences between them is that object sever.o has pthread_*
functions in elf Symbol table (part of output of readelf -W -a):
00002899 00006602 R_386_PC32 00000000 pthread_create
and in .symtab:
102: 00000000 0 NOTYPE GLOBAL DEFAULT UND pthread_create
Now I'm not sure if I did anything wrong or is this a bug in gcc/
binutils.
In any way, David and Paul, thank you very much for your help. Now I
know that I should not fix libldap_r.so. )
Peter.
David Schwartz
wrote:
> David Schwartz <dav...@webmaster.com> writes:
> > Consider the following typical example. A function call requires some
> > work to be done and then a network protocol to be shut down in an
> > organized way.
> This may be a typical example in your area of expertise, but I assert
> that over 90% of libfoo_r libraries are *not* written that way [1] [2].
That's just one example. I am not claiming that that particular
example applies in every case.
> The case above constitues exactly 0% of libfoo_r libraries that I
> have ever seen.
Again, it's just an example. There are many other reasons.
> > Since libfoo_r is a multi-threaded library, it can implement functions
> > such that there is less blocking.
> That may sometimes be possible, and if it is possible, it has nothing
> to do with modern hardware -- there would be just as much "less
> blocking" on ancient hardware.
Again, you mistake an example for the argument. That there can be less
blocking is just one reason a multi-threaded library can be faster
than the corresponding single-threaded library.
Another reason is the ability to take advantage of multiple CPU cores.
The majority of machines being sold today have more than one core. The
majority of machines sold two years ago had but one.
DS
Paul Pluzhnikov
> But why then this behavior is not default? Most libraries linux are
> threaded and reference on libpthreads is included and thus dl_open()
> succeeds...
Most libraries are *not* threaded (at least not on any of my systems).
As to default, it all depends on how the developers chose to link
them -- link with '-pthread' and you get the dependencly; link
without and you do not.
Many libraries are probably compiled and linked with '-pthread'
without much consideration for what happens when this library
is dlopen()ed.
> libtool --mode=link i686-pc-linux-gnu-g++ -o libgigabase_r.la class.lo
> compiler.lo database.lo replicator.lo hashtab.lo file.lo symtab.lo
> btree.lo rtree.lo cursor.lo query.lo pagepool.lo wwwapi.lo unisock.lo
> blob.lo container.lo exception.lo localcli.lo sync.lo -pthread -rpath /
> usr/lib -version-info 2
> i686-pc-linux-gnu-g++ -shared -nostdlib /usr/lib/gcc/i686-pc-linux-gnu/
> 4.1.2/../../../crti.o /usr/lib/gcc/i686-pc-linux-gnu/4.1.2/
> crtbeginS.o .libs/class.o .libs/compiler.o .libs/database.o .libs/
> replicator.o .libs/hashtab.o .libs/file.o .libs/symtab.o .libs/
> btree.o .libs/rtree.o .libs/cursor.o .libs/query.o .libs/
> pagepool.o .libs/wwwapi.o .libs/unisock.o .libs/blob.o .libs/
> container.o .libs/exception.o .libs/localcli.o .libs/sync.o -L/usr/
> lib/gcc/i686-pc-linux-gnu/4.1.2 -L/usr/lib/gcc/i686-pc-linux-gnu/
> 4.1.2/../../../../i686-pc-linux-gnu/lib -L/usr/lib/gcc/i686-pc-linux-
> gnu/4.1.2/../../.. -lstdc++ -lm -lc -lgcc_s /usr/lib/gcc/i686-pc-linux-
> gnu/4.1.2/crtendS.o /usr/lib/gcc/i686-pc-linux-gnu/4.1.2/../../../
> crtn.o -pthread -Wl,-soname -Wl,libgigabase_r.so.2 -o .libs/
> libgigabase_r.so.2.0.0
...
> And next program fails:
This looks like a bug in libtool -- why on earth is it doing
'-nostdlib' link, and supplying crt*.o objects directly?
Linking with 'g++ -pthread -nostdlib' *drops* libpthread from the
link, and causes your subsequent problem:
$ g++ -g foo.c -fPIC -o foo.so -pthread -shared && ldd foo.so | grep pthread
libpthread.so.0 => /lib/tls/libpthread.so.0 (0x00aed000)
$ g++ -g foo.c -fPIC -o foo.so -pthread -shared -nostdlib && ldd foo.so
statically linked
Peter
> This looks like a bug in libtool -- why on earth is it doing
> '-nostdlib' link, and supplying crt*.o objects directly?
>
> Linking with 'g++ -pthread -nostdlib' *drops* libpthread from the
> link, and causes your subsequent problem:
>
> $ g++ -g foo.c -fPIC -o foo.so -pthread -shared && ldd foo.so | grep pthread
> libpthread.so.0 => /lib/tls/libpthread.so.0 (0x00aed000)
> $ g++ -g foo.c -fPIC -o foo.so -pthread -shared -nostdlib && ldd foo.so
> statically linked
Thank you again. There was a report in libtool mailing list
http://lists.gnu.org/archive/html/libtool/2005-12/msg00065.html
and further report in gcc bugzilla
http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25460
It was two years ago, but still (I'm using here libtool 1.5.23b) is
not fixed. I was aware about that bug, but just got it wrong. But
> link them -- link with '-pthread' and you get the dependencly; link
> without and you do not.
you told me that libraries without dependency on libpthread are *not*
broken and in some cases are even preferable. So may be it's better to
fix gcc so further linking executables with such libraries succeed?
This works without --as-needed but became broken with this ld flag.
Also with --as-needed gcc works while for g++ is not (so if I
substitute g++ to gcc for the linking everything works). Comparing
verbose gcc and g++ I found that difference in the output is the
following
(well there is a bit more, but this helps collect finish its work):
--no-as-needed -lpthread -lc -lgcc --as-needed
Thus may be g++ should be fixed?
Peter.
Paul Pluzhnikov
> http://gcc.gnu.org/bugzilla/show_bug.cgi?id=25460
>
> It was two years ago, but still (I'm using here libtool 1.5.23b) is
> not fixed.
The gcc bug is resolved/invalid, so it is unlikely to ever be fixed.
> you told me that libraries without dependency on libpthread are *not*
> broken and in some cases are even preferable. So may be it's better to
> fix gcc so further linking executables with such libraries succeed?
Maybe ...
> This works without --as-needed but became broken with this ld flag.
> Also with --as-needed gcc works while for g++ is not (so if I
> substitute g++ to gcc for the linking everything works). Comparing
> verbose gcc and g++ I found that difference in the output is the
> following
> (well there is a bit more, but this helps collect finish its work):
>
> --no-as-needed -lpthread -lc -lgcc --as-needed
>
> Thus may be g++ should be fixed?
Sure -- it seems that gcc and g++ handling of "-pthread" should
be consistent. You can file another bug against g++, and/or fix
this in your g++ specs file.
David Hopwood
> On Apr 29, 3:07 am, Peter <volkov.pe...@gmail.com> wrote:
>
>>Sorry if I'm asking completely trivial question but I've spent 3 full
>>days searching google and learning on how we should link with thread
>>libraries on different architectures but have not found clear answer
>>on this questions.
>
> Almost every modern platform supports '-pthread'.
<http://gcc.gnu.org/bugzilla/show_bug.cgi?id=20705>
2 years and still not assigned to anyone. Doesn't seem like the gcc
maintainers are giving it a high priority.
--
David Hopwood <david....@industrial-designers.co.uk>