Running initial code when in library
Paulo Matos
I have a software which links to some plugins loaded dynamically. The
plugin self-registers using a proxy class as described in:
http://www.linuxjournal.com/article/3687
class proxy {
public:
proxy(){
factory["shape name"] = maker;
}
};
proxy p;
The code is executed with when:
void *hndl = dlopen("libfoo.so", RTLD_NOW);
and the class register. Problem is that sometimes (I know this is not
usual in current software but I really need this) I need to pack all
the software and plugins into one big piece of code so I link every
plugin statically to the core. However, this proxy approach doesn't
seem to work because the plugins are not registering themselves.
Any ideas on how can I make them self-register? I do not have control
about the number of plugins, or their names for that matter in the core
software so I can't just register them manually in the core.
Cheers,
Paulo Matos
Paul Pluzhnikov
> Problem is that sometimes (I know this is not
> usual in current software but I really need this) I need to pack all
> the software and plugins into one big piece of code so I link every
> plugin statically to the core.
This should work just fine.
You still have your global objects, and their constructors should
still be firing at process startup.
> However, this proxy approach doesn't
> seem to work because the plugins are not registering themselves.
They probably are. Run your program in debugger, set a breakpoint on
'proxy::proxy()' and observe that the breakpoint is hit.
What is probably preventing this from working is that "factory"
itself has not been constructed yet (the order of construction for
globals from separate translation units is unspecified).
So you must modify your code to insure that it is constructed before
any plugin tries to register with it, e.g.:
proxy() { getFactory()->insert("shape name", maker); }
where getFactory() constructs the "factory" singleton on the
first call, and returns it again on all subsequent ones.
Cheers,
--
In order to understand recursion you must first understand recursion.
Remove /-nsp/ for email.
Paulo Matos
> "Paulo Matos" <pocm...@gmail.com> writes:
>
> > Problem is that sometimes (I know this is not
> > usual in current software but I really need this) I need to pack all
> > the software and plugins into one big piece of code so I link every
> > plugin statically to the core.
>
> This should work just fine.
> You still have your global objects, and their constructors should
> still be firing at process startup.
>
I have singleton and everything should work fine but it is not:
extern "C" {
esatEnumerator *create() {
return new esatEnumeratorCR();
}
void destroy(esatEnumerator *e) {
delete e;
}
class proxy {
public:
proxy() {
esatModuleDispatcher::Instance()->registerEnumerator(create,
destroy);
}
};
proxy p;
}
This should be working but it is not. esatModuleDispatcher::Instance()
returns the singleton instance. Even if I add a exit(EXIT_FAILURE);
before the call nothing happens. Execution is not hitting proxy
constructor.
Paul Pluzhnikov
> I have singleton and everything should work fine but it is not:
In that case, two possibilities come to mind:
- you didn't build your exe correctly (always link C++ code with 'g++')
- the code containing global 'proxy p;' objects isn't actually
linked into the executable at all.
The second explanation is more likely, especially if that code is
supposed to be linked in from an archive library.
Suggested reading:
http://webpages.charter.net/ppluzhnikov/linker.html
Paulo Matos
Paul Pluzhnikov wrote:
> "Paulo Matos" <pocm...@gmail.com> writes:
>
> > I have singleton and everything should work fine but it is not:
>
> In that case, two possibilities come to mind:
> - you didn't build your exe correctly (always link C++ code with 'g++')
> - the code containing global 'proxy p;' objects isn't actually
> linked into the executable at all.
>
> The second explanation is more likely, especially if that code is
> supposed to be linked in from an archive library.
>
Thanks for all your help. I'm using autotools: auto make/conf and
libtool for this stuff. I'm getting quite confused on all the autoconf
flags. I think that the ones which I'm currently using are the correct
ones for this: AC_DISABLE_SHARED and AC_PROG_LIBTOOL. Everything is
built with autotools.
Any ideas on how can I know if the code is linked as shared or as
static?
Thanks once again.
> Suggested reading:
> http://webpages.charter.net/ppluzhnikov/linker.html
>
I'll be reading this now, thanks.
Paulo Matos
Paul Pluzhnikov
> Any ideas on how can I know if the code is linked as shared or as
> static?
What's the link command line?
Add to it '-Wl,--verbose' and you'll see which libraries the linker
is actually using.
You can also easily verify my guess (that the objects you expect
aren't actually linked in):
- edit one of the sources that "fails to register", add:
void function_that_is_never_called() { }
- rebuild
- run
nm your_exe | grep function_that_is_never
If the output is nothing, my guess is corect.
If you get
<some-hex-number> T function_that_is...
then my guess is wrong.
Paulo Matos
Paul Pluzhnikov wrote:
> "Paulo Matos" <pocm...@gmail.com> writes:
>
> > Any ideas on how can I know if the code is linked as shared or as
> > static?
>
> What's the link command line?
>
> Add to it '-Wl,--verbose' and you'll see which libraries the linker
> is actually using.
>
> You can also easily verify my guess (that the objects you expect
> aren't actually linked in):
>
> - edit one of the sources that "fails to register", add:
>
> void function_that_is_never_called() { }
>
> - rebuild
> - run
>
> nm your_exe | grep function_that_is_never
>
> If the output is nothing, my guess is corect.
> If you get
>
> <some-hex-number> T function_that_is...
>
> then my guess is wrong.
>
Your guess is correct. Verbose output from linker is wierd:
After the .o files:
ttempt to open ../lib/bf/src/.libs/libbf.a succeeded
attempt to open ../lib/ot/src/.libs/libot.a succeeded
attempt to open ../lib/fm/src/.libs/libfm.a succeeded
attempt to open ../lib/glpkinterface/src/.libs/libglpkinterface.a
succeeded
attempt to open /usr/i686-pc-linux-gnu/bin/libglpk.so failed
attempt to open /usr/i686-pc-linux-gnu/bin/libglpk.a failed
attempt to open /usr/i686-pc-linux-gnu/lib/libglpk.so failed
attempt to open /usr/i686-pc-linux-gnu/lib/libglpk.a failed
attempt to open
/usr/lib/gcc/i686-pc-linux-gnu/../../../i686-pc-linux-gnu/lib/libglpk.so
failed
attempt to open
/usr/lib/gcc/i686-pc-linux-gnu/../../../i686-pc-linux-gnu/lib/libglpk.a
failed
attempt to open /usr/lib/gcc/i686-pc-linux-gnu/4.0.2/libglpk.so failed
attempt to open /usr/lib/gcc/i686-pc-linux-gnu/4.0.2/libglpk.a failed
attempt to open /usr/lib/gcc/i686-pc-linux-gnu/4.0.2/libglpk.so failed
attempt to open /usr/lib/gcc/i686-pc-linux-gnu/4.0.2/libglpk.a failed
attempt to open
/usr/lib/gcc/i686-pc-linux-gnu/4.0.2/../../../../i686-pc-linux-gnu/lib/libglpk.so
faile
attempt to open
/usr/lib/gcc/i686-pc-linux-gnu/4.0.2/../../../../i686-pc-linux-gnu/lib/libglpk.a
failed
attempt to open
/usr/lib/gcc/i686-pc-linux-gnu/4.0.2/../../../libglpk.so failed
attempt to open /usr/lib/gcc/i686-pc-linux-gnu/4.0.2/../../../libglpk.a
succeeded
attempt to open ../lib/ncquest/src/.libs/libncquest.a succeeded
attempt to open /usr/lib/gcc/i686-pc-linux-gnu/4.0.2/libstdc++.so
succeeded
/usr/lib/gcc/i686-pc-linux-gnu/4.0.2/libstdc++.so
attempt to open /usr/lib/liblog4cpp.so succeeded
/usr/lib/liblog4cpp.so
....
and it goes on. The first lines are the libtool libraries that it
should be linking and that doesn't seem to be happening... right? Why?
I'm using the -static flag, what's the problem?
Last lines before linker are:
/bin/sh ../libtool --tag=CXX --mode=link g++ -ggdb -Wall -std=c++98
-static -Wl,--verbose -o extsat esatmathcluster.o esatmath.o
esatmoduledispatcher.o esatstatistics.o esattrue.o esatvariable.o
esatcoeflist.o esatfalse.o esatbooleanop.o esateqt.o esatbooleanvar.o
esatineqsystem.o esatcodegen.o graph.o esatefficientcharstr.o
graphdense.o vartable.o graphsparse.o esatstring.o esatsysutils.o
esatargumentsparser.o esator.o esatequiv.o esatand.o esatboolite.o
esatopcreate.o esatnot.o esatxor.o esatimplies.o esatenumerator.o
smt-parser.o smt-lexer.o esatfile.o esatsolver.o esatmodel.o
esatcluster.o extsat.o esatlogger.o ../lib/bf/src/libbf.la
../lib/ot/src/libot.la ../lib/fm/src/libfm.la
../lib/glpkinterface/src/libglpkinterface.la
../lib/ncquest/src/libncquest.la -lm -lnsl -llog4cpp
mkdir .libs
g++ -ggdb -Wall -std=c++98 -Wl,--verbose -o extsat esatmathcluster.o
esatmath.o esatmoduledispatcher.o esatstatistics.o esattrue.o
esatvariable.o esatcoeflist.o esatfalse.o esatbooleanop.o esateqt.o
esatbooleanvar.o esatineqsystem.o esatcodegen.o graph.o
esatefficientcharstr.o graphdense.o vartable.o graphsparse.o
esatstring.o esatsysutils.o esatargumentsparser.o esator.o esatequiv.o
esatand.o esatboolite.o esatopcreate.o esatnot.o esatxor.o
esatimplies.o esatenumerator.o smt-parser.o smt-lexer.o esatfile.o
esatsolver.o esatmodel.o esatcluster.o extsat.o esatlogger.o
../lib/bf/src/.libs/libbf.a -L/usr/i686-pc-linux-gnu/bin
-L/usr/i686-pc-linux-gnu/lib
-L/usr/lib/gcc/i686-pc-linux-gnu/../../../i686-pc-linux-gnu/lib
../lib/ot/src/.libs/libot.a ../lib/fm/src/.libs/libfm.a
../lib/glpkinterface/src/.libs/libglpkinterface.a -lglpk
../lib/ncquest/src/.libs/libncquest.a
/usr/lib/gcc/i686-pc-linux-gnu/4.0.2/libstdc++.so -lm
/usr/lib/liblog4cpp.so -lnsl
Any ideas on where the problem might be?
Thanks,
Paulo Matos
Paul Pluzhnikov
> Your guess is correct.
I thought so :)
> Verbose output from linker is wierd:
Nothing weird about it ...
> After the .o files:
> attempt to open ../lib/bf/src/.libs/libbf.a succeeded
> attempt to open ../lib/ot/src/.libs/libot.a succeeded
> attempt to open ../lib/fm/src/.libs/libfm.a succeeded
...
> The first lines are the libtool libraries that it
> should be linking and that doesn't seem to be happening... right? Why?
It *is* linking them, but it isn't pulling any objects from them
(or at least not the objects you expect to be pulled),
because nothing in them is in the librarian's *needed* list.
> I'm using the -static flag, what's the problem?
You should *not* be using -static if you want your executable to
run on any systems other then the one it was built on (building
anything with -static is a very bad idea(TM)), but that's completely
orthogonal to your current problem.
> Any ideas on where the problem might be?
The answer above (and understanding and the linker.html I mentioned
earlier).
GNU-ld specific solution (assuming you really do want all of
libbf libot and libfm to be included in the executable):
> /bin/sh ../libtool --tag=CXX --mode=link g++ -ggdb -Wall -std=c++98 ...
> esatcluster.o extsat.o esatlogger.o
-Wl,--whole-archive
> ../lib/bf/src/libbf.la ../lib/ot/src/libot.la ../lib/fm/src/libfm.la
-Wl,--no-whole-archive
> ../lib/glpkinterface/src/libglpkinterface.la
> ../lib/ncquest/src/libncquest.la -lm -lnsl -llog4cpp
Cheers,
Paulo Matos
Paul Pluzhnikov wrote:
>
> GNU-ld specific solution (assuming you really do want all of
> libbf libot and libfm to be included in the executable):
>
> > /bin/sh ../libtool --tag=CXX --mode=link g++ -ggdb -Wall -std=c++98 ...
> > esatcluster.o extsat.o esatlogger.o
> -Wl,--whole-archive
> > ../lib/bf/src/libbf.la ../lib/ot/src/libot.la ../lib/fm/src/libfm.la
> -Wl,--no-whole-archive
> > ../lib/glpkinterface/src/libglpkinterface.la
> > ../lib/ncquest/src/libncquest.la -lm -lnsl -llog4cpp
>
I didn't know the linker was going to search for unused objects and if
not used it would not include them. Nice to know. Thanks also for the
reference to your bookshelf metafore. It's nice! Still I don't
understand why this works this way. This is an example where there are
not specific references to the objects in a given module but they are
needed. They self register and then the core software calls their
methods through the interface which works with polymorphism so it's not
possible to know if something is needed at compile time. Everything is
done through the module interface which is also used to access each
module's (compiled statically) functions through polymorphism... kind I
have no other solution than using --whole-archive to linker right now,
unless there's a specific autotools solution I don't know of which does
this automatically.
Thanks for all your help, and if you have any other suggestion, I'm all
ears! :)
Cheers,
Paulo Matos
Paul Pluzhnikov
> I didn't know the linker was going to search for unused objects and if
> not used it would not include them. Nice to know.
All linkers I know (even the MS ones) work that way, and (AFAICT)
always did.
> I don't understand why this works this way.
It's the *only* reasonable way for linker to work.
If it didn't work that way, every program (when linked statically)
would include the *entire* code for all libraries it was linked
against (including libc, which is quite large).
> I have no other solution than using --whole-archive to linker right now,
You do have other solutions, e.g. explicitly list objects that must
be linked in, and the linker will do the right thing (pulling in
only necessary parts of the libraries).
Paulo Matos
Paul Pluzhnikov wrote:
> > I have no other solution than using --whole-archive to linker right now,
>
> You do have other solutions, e.g. explicitly list objects that must
> be linked in, and the linker will do the right thing (pulling in
> only necessary parts of the libraries).
>
Unfortunately using --whole-archive doesn't seem viable right now since
autotools don't allow positioned ldflags which means that when I add
--whole-archive I really don't know how it will be positioned within
other arguments. Right now it's in the beginning which will include
_all_ of the archives and results in error for some reason I don't
know. Still, including all archives is not ok.
Again, although I may have to all this stuff automatically it is very
wierd that this is not an automated process. Isn't this a normal
problem: an interface with some libraries using the interface. The core
creates and destroys the object of the libraries through the interface.
Obviously there is no reference from the core to the object files of
the libraries. No reference from the interface to anything in the
library... This seems a normal issue which turns out to be a nightmare
to solve. :-/
Paul Pluzhnikov
> Isn't this a normal
> problem: an interface with some libraries using the interface. The core
> creates and destroys the object of the libraries through the interface.
No, I don't believe this is a widely-used design pattern,
at least not with static linking.
> This seems a normal issue which turns out to be a nightmare
> to solve. :-/
Firstly, your nightmare is entirely of your own doing.
Secondly, it has trivial solutions.
One I already mentioned: listing objects explicitly.
Here is another:
- rename your "registration" objects such that their names are
unique, e.g. instead of "proxy p;" say
proxy p_libbf; // in libbf.a
proxy p_libot; // in libot.a
// etc.
- into your main.o add this:
#ifdef PULL_LIBBF
extern proxy p_libbf;
proxy &p_libbf_ref = p_libbf;
#endif
// etc.
- now you can control what is pulled into the static link:
g++ -DPULL_LIBBF main.cc -c
- link just the way you do now, and the object(s) that you need
from libbf.a will be pulled in into the main exe, because p_libbf
is now on the "needed" list.
Paulo Matos
Paul Pluzhnikov wrote:
> "Paulo Matos" <pocm...@gmail.com> writes:
>
> > Isn't this a normal
> > problem: an interface with some libraries using the interface. The core
> > creates and destroys the object of the libraries through the interface.
>
> No, I don't believe this is a widely-used design pattern,
> at least not with static linking.
>
OK, I may agree with you on this.
> > This seems a normal issue which turns out to be a nightmare
> > to solve. :-/
>
> Firstly, your nightmare is entirely of your own doing.
>
Why is it of my doing? Because I'm trying a design pattern which is not
so common after all?
> Secondly, it has trivial solutions.
> One I already mentioned: listing objects explicitly.
>
> Here is another:
> - rename your "registration" objects such that their names are
> unique, e.g. instead of "proxy p;" say
>
> proxy p_libbf; // in libbf.a
> proxy p_libot; // in libot.a
> // etc.
>
>
> - into your main.o add this:
>
> #ifdef PULL_LIBBF
> extern proxy p_libbf;
> proxy &p_libbf_ref = p_libbf;
> #endif
> // etc.
>
> - now you can control what is pulled into the static link:
>
> g++ -DPULL_LIBBF main.cc -c
>
> - link just the way you do now, and the object(s) that you need
> from libbf.a will be pulled in into the main exe, because p_libbf
> is now on the "needed" list.
>
This seems a nice solution. I'll only be able to try this later. Thanks
a lot for you help, comments and suggestions. I do appreciate them. :-)
Cheers,
Paulo Matos
Paul Pluzhnikov
> Why is it of my doing? Because I'm trying a design pattern which is not
> so common after all?
No, it's because you choose to build your design with autotools,
which appear not to be flexible enough to do so without special
hacks.
Paulo Matos
Paul Pluzhnikov wrote:
> "Paulo Matos" <pocm...@gmail.com> writes:
>
> > Why is it of my doing? Because I'm trying a design pattern which is not
> > so common after all?
>
> No, it's because you choose to build your design with autotools,
> which appear not to be flexible enough to do so without special
> hacks.
>
Still, your hack worked beautifully. And I'm open to other
suggestions... I do not _need_ to use autotools but they seem rather
_standard_ in terms of open source software and I don't know anything
else.