Problems compiling static for BOINC

[anon...@gmail.com]

Hi all,

I just tried

export CXXFLAGS=-static
./build-garli.sh --enable-boinc

 But all I get is this:

configure: error: Problem with link flags.  Most likely a library cannot be found.
Check that any libraries listed below can be found in a system default location or a location specified in LDFLAGS with a -L.
LIBS= -lpthread -lboinc -lboinc_api -lncl
LDFLAGS= -L/usr/share/boinc-dev/lib -L/media/RAIDA/Tech/GridCrunch/garli/garli-2.01/ncl-2.1.18/installed/lib/ncl

Dynamical compilation for BOINC and static compilation without BOINC works fine.

I'm on Debian 7.6 amd64 with dash=0.5.7-3, bash=4.2+dfsg-0.1, gcc=4:4.7.2-1, libboinc-app-dev=7.2.47+dfsg-3~bpo70+1, boinc-dev=7.2.47+dfsg-3~bpo70+1
I tried downgrading to 7.0.27 but that only gave me another error. I tried to give other CXXFLAGS, CPPFLAGS and LDFLAGS but I'm not confident with this. I even compiled libboinc.a and libboinc_api.a but didn't get any wiser.

I could give you the config.log or the output of sh -x

Any help appreciated

A

[Adam Bazinet]

I haven't tried a static compile in several years, so none of the following may apply any more. However, I won't be able to look into this issue in more detail until I attempt to do it myself (which will hopefully be within a few weeks)... in the meantime, I'll print some correspondence with Derrick from approximately five years ago.

DZ

I also just checked in changes that should allow Boinc compilation through configure (on linux at least).  Just do --enable-boinc and --with-boinc=<dir>.  By default it dynamically links.  In theory it should be possible to make it statically link by adding env LDFLAGS="-static -static-libgcc" at the start of the configure line, although I'm not sure if the static-libgcc part is really necessary.  But I had problems with any form of static linking. 
What seems to be happening with your Makefile (for me) is that the .a static libraries are being manually added as objects, which statically includes any functions included therein.  But, you also include the -lboinc and -lboinc_api flags, which by default links to the dynamic .so libraries if they are found, but ONLY if there are some functions that weren't defined in the static .a libraries already included.  For some reason a few functions were not in the static files, so they are found in the .so dynamics and I get a error about loading dynamic libraries at runtime.  If you add the -static flag during compilation then it forces it to only use .a files (and manually specifying them to link to isn't necessary), and then mine can't finish compilation because a few functions are undefined.  The same thing happens if you compile BOINC with --disable-shared, in which case it can only link to the .a statics.

Anyway, as I understand it you shouldn't generally need to both specify the libraries with the -lboinc and -lboinc_api and simultaneously manually include the .a's.  My configure route doesn't include the .a's, but a -static should force their use.  What this all boils down to is that I can't get a fully static executable (ie, doing ./GarliXXX always gives an error about shared libraries) via configure nor vs your Makefile.  It may be that what I'm seeing is system specific.  You probably have reasons for doing it your way.

ALB

Hi Derrick,
I've been working on this today (all on Linux), and here's what I've discovered.
1) The configure script with --enable-boinc and --with-boinc=<dir> seems to work as you said it would, but the problems with dynamic/static linking are still there and are exactly as you described them (I saw all the same things)
2) So, I went back to trying to use the Makefile.ser that I'd been successful with in the past.  With the default one (that I sent you, that does the funky .a specification in addition to the -L/-l stuff, and points at a BOINC build (not install) location), it compiled a dynamic executable, but one that did not have a dependency on a BOINC library (those must have been statically linked, perhaps as a result of specifying the .a files?)  I went back and verified and this is the normal way we've compiled it in the past... haven't been specifying -static on the link line.  The good news is I tested this version and the progress bar updating worked!  That's the main thing, in my view, which is great.
3) Just for kicks, I went back and added -static to the link line in Makefile.ser... it built, but the executable crashed when run under the BOINC client - I'm not sure why.
4) Removing either the .a file specification or the $(BOINCLIBPATHS) $(BOINCLIBS) from the link line, I couldn't get it to build successfully.
So, one thing would be to figure out why you couldn't get it to build with the Makefile.ser on your machine... if you want, I can always pack up my working directory and BOINC source tree and send it to you, although I didn't do anything special (config'd BOINC with --disable-client, is all, and didn't run make install) If you could get the Makefile.ser method working, perhaps you could figure out why that's working the way it is.

DZ

Right, by manually specifying those .a files it will include everything in them in the executable, just as if they were a compiled .o object file.  Then if any functions are referred to in other code but not found in the .a's, it will dynamically link to dynamic libraries.  And if it doesn't find them in the dynamic libraries (or doesn't find the dynamic libraries) it should fail to build.
So, how are you verifying that it doesn't depend on the BOINC library?  If it is dynamically linked to it, then it should require it by definition and shouldn't even execute if it can't find it at runtime.  What do you get when you do "ldd Garli0.961b"?
Here's what ldd is telling me for any case in which I can get it to build Garli :
###
GARLI configured with
--with-boinc=<boinc_install> --enable-boinc
With Boinc that install configured with:
--disable-client --disable-server
ldd Garli0.961b
       libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003fe7000000)
       libboinc.so.6 => not found
       libboinc_api.so.6 => not found
       libstdc++.so.6 => /share/apps/opt/gcc/lib64/libstdc++.so.6 (0x00002b02191dc000)
       libm.so.6 => /lib64/libm.so.6 (0x0000003fe6800000)
       libgcc_s.so.1 => /share/apps/opt/gcc/lib64/libgcc_s.so.1 (0x00002b02194da000)
       libc.so.6 => /lib64/libc.so.6 (0x0000003fe6400000)
       /lib64/ld-linux-x86-64.so.2 (0x0000003fe6000000)
This is dynamically linked.  Providing the --with-boinc let GARLI compile correctly because at compile time it can see that what it needs is in the libraries in that directory.  But, the libraries aren't somewhere that it known to look at runtime, hence the "not found".  This fails at runtime.  If I specify the directory with the libraries in LD_LIBRARY_PATH, then it finds them and runs fine.  But it will require those libraries somewhere on any machine that it runs on.

 ALB

Just to jump back to the Linux stuff again for a second - I understood the problems you were having, and when I tried using your configure script I got the same things you're showing (using ldd showed the dependence on the BOINC api/lib libraries)

Using the Makefile, with the .a specification to statically link the BOINC libraries and then to dynamically link everything else, I get an executable like this:

linux-gate.so.1 =>  (0x00a35000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00246000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x02ca7000)
libm.so.6 => /lib/libm.so.6 (0x00dca000)
libc.so.6 => /lib/libc.so.6 (0x00101000)
/lib/ld-linux.so.2 (0x003fa000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x02c99000)

[anon...@gmail.com]

Yes, thanks for digging this out (0.961 seems to be from 2006?!). Unfortunately you do not specify "the funky .a specification in addition to the -L/-l stuff, and points at a BOINC build".
So you maybe think I know what I'm doing. Or maybe are aware that I won't be able to help you with the building procedure. I will try to go deeper into this (I made some acquaintances).

[Derrick Zwickl]

Hello,

Adam was going out of town, so he dumped these previous communications here before he left in the hope that they might be helpful, although I can see why it would be hard to decipher what we were talking about back then. 

Anyway, he or I will take a further look at this when we can.  I'm not aware of anyone compiling the BOINC version besides Adam and myself, so it isn't something that we frequently have to troubleshoot.  Including your config.log might be helpful.  It is possible that we'll need to change the configuration process itself (i.e. configure.ac), so changing flags externally may not be able to fix it.

Best,
Derrick

[anon...@gmail.com]

Thanks, DJZ. But this is beyond my comprehension. After reconsideration I think compiling BOINC for this is not sane. Since you asked for it (though ALB's archives say you need to specify .a and despecify -static) here's my config.log after a normal

env LDFLAGS="-static" ./build_garli.sh --enable-boinc

https://drive.google.com/file/d/0B8gqmpXudS7ccUd4bDk1bHZtbnM/edit?usp=sharing

[Derrick Zwickl]

Hello,

Ok, I got it to work on Ubuntu at least.  Just compile the BOINC libraries without the shared versions (--disable-shared causes it to compile only the .a files, not the .so), install them somewhere and then point the GARLI configuration to that location with the --with-boinc flag.  In my experience when you point any configuration to a library location and only static libraries exist, then they are statically linked in, otherwise it defaults to linking dynamically.

You'll probably want to compile NCL statically with --disable-shared as well, install it somewhere, and point GARLI to it.

So, something like this should work:
BOINC configuration:
./configure --disable-shared --disable-client --disable-server --disable-manager --prefix=static_boinc_install_dir
GARLI configuration:
./configure --enable-boinc --with-boinc=static_boinc_install_dir --with-ncl=static_ncl_install_dir

You can use the "ldd" command on a binary to see what shared libraries it requires at runtime.  Statically linked libraries won't appear there.

Let me know if that works for you.
Best,
Derrick

[anon...@gmail.com]

Great.
Which versions? Can't get it to work with boinc_7.0.27+dfsg.orig.tar.gz and garli-2.01.tar.gz

/media/RAIDA/Tech/GridCrunch/garli/garli-2.01$ make -j2

make  all-recursive
make[1]: Entering directory `/media/RAIDA/Tech/GridCrunch/garli/garli-2.01'
Making all in src
make[2]: Entering directory `/media/RAIDA/Tech/GridCrunch/garli/garli-2.01/src'
g++ -DHAVE_CONFIG_H -I. -I..  -I/media/RAIDA/Tech/GridCrunch/garli/garli-2.01/ncl-2.1.18/installed/include -I/media/RAIDA/Tech/GridCrunch/garli/garli-2.01/ncl-2.1.18/installed/include/ncl -DIGNORE_NXS_ASSERT -I/media/RAIDA/Tech/GridCrunch/garli/garli-2.01/ncl-2.1.18/installed/include -I/media/RAIDA/Tech/GridCrunch/garli/garli-2.01/ncl-2.1.18/installed/include/ncl -DIGNORE_NXS_ASSERT  -DUNIX -DNCL_CONST_FUNCS -DNDEBUG -DBOINC -Wno-uninitialized -fsigned-char -O3 -ffast-math -funroll-loops -fstrict-aliasing -I/media/RAIDA/Tech/GridCrunch/static_boinc_install_dir/include/boinc -MT adaptation.o -MD -MP -MF .deps/adaptation.Tpo -c -o adaptation.o adaptation.cpp
g++ -DHAVE_CONFIG_H -I. -I..  -I/media/RAIDA/Tech/GridCrunch/garli/garli-2.01/ncl-2.1.18/installed/include -I/media/RAIDA/Tech/GridCrunch/garli/garli-2.01/ncl-2.1.18/installed/include/ncl -DIGNORE_NXS_ASSERT -I/media/RAIDA/Tech/GridCrunch/garli/garli-2.01/ncl-2.1.18/installed/include -I/media/RAIDA/Tech/GridCrunch/garli/garli-2.01/ncl-2.1.18/installed/include/ncl -DIGNORE_NXS_ASSERT  -DUNIX -DNCL_CONST_FUNCS -DNDEBUG -DBOINC -Wno-uninitialized -fsigned-char -O3 -ffast-math -funroll-loops -fstrict-aliasing -I/media/RAIDA/Tech/GridCrunch/static_boinc_install_dir/include/boinc -MT bipartition.o -MD -MP -MF .deps/bipartition.Tpo -c -o bipartition.o bipartition.cpp
In file included from defs.h:125:0,
                 from bipartition.cpp:18:
/media/RAIDA/Tech/GridCrunch/static_boinc_install_dir/include/boinc/boinc_api.h:86:63: error: ‘NULL’ was not declared in this scope
make[2]: *** [bipartition.o] Error 1
make[2]: *** Waiting for unfinished jobs....
mv -f .deps/adaptation.Tpo .deps/adaptation.Po
make[2]: Leaving directory `/media/RAIDA/Tech/GridCrunch/garli/garli-2.01/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/media/RAIDA/Tech/GridCrunch/garli/garli-2.01'
make: *** [all] Error 2

(Reason I called this insane was because of the possibilities to optimize BOINC.)

Regards
ASR^^

[Derrick Zwickl]

I just checked out the BOINC trunk svn repo, which I guess is 7.5.  The error you are getting there is I think due to changes in more recent versions of gcc.  Mine is probably older.  Some standard definitions aren't included by default anymore.  Try adding
#include <stddef.h>
to the GARLI defs.h file after the
#define DEFS
line and recompiling.

Best,
Derrick

--
You received this message because you are subscribed to the Google Groups "garli_users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to garli_users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[anon...@gmail.com]

I get more errors I guess because of the older BOINC version. Not really partial to download new.
[If you need someone for testing you can send it to me (AMD64).]

make  all-recursive
make[1]: Entering directory `/garli/garli-2.01'
Making all in src
make[2]: Entering directory `/garli/garli-2.01/src'
g++  -DUNIX -DNCL_CONST_FUNCS -DNDEBUG -DBOINC -Wno-uninitialized -fsigned-char -O3 -ffast-math -funroll-loops -fstrict-aliasing -I/static_boinc_install_dir/include/boinc -L/static_boinc_install_dir/lib -L/garli/garli-2.01/ncl-2.1.18/installed/lib/ncl -L/static_boinc_install_dir/lib -L/garli/garli-2.01/ncl-2.1.18/installed/lib/ncl -o Garli adaptation.o bipartition.o condlike.o configoptions.o configreader.o datamatr.o funcs.o garlimain.o garlireader.o individual.o linalg.o model.o mpifuncs.o optimization.o population.o rng.o sequencedata.o set.o translatetable.o tree.o treenode.o mpitrick.o  -lncl -lpthread -lboinc -lboinc_api -lncl
/static_boinc_install_dir/lib/libboinc_api.a(boinc_api.o): In function `suspend_activities':
/boinc-7.0.27+dfsg/api/boinc_api.cpp:828: undefined reference to `suspend_or_resume_descendants(int, bool)'
/static_boinc_install_dir/lib/libboinc_api.a(boinc_api.o): In function `boinc_exit':
/boinc-7.0.27+dfsg/api/boinc_api.cpp:660: undefined reference to `boinc_finish_diag'
/boinc-7.0.27+dfsg/api/boinc_api.cpp:666: undefined reference to `boinc_info'
/boinc-7.0.27+dfsg/api/boinc_api.cpp:682: undefined reference to `set_signal_exit_code'
/boinc-7.0.27+dfsg/api/boinc_api.cpp:657: undefined reference to `kill_descendants(int)'
/static_boinc_install_dir/lib/libboinc_api.a(boinc_api.o): In function `boinc_init_options_general(BOINC_OPTIONS&)':
/boinc-7.0.27+dfsg/api/boinc_api.cpp:476: undefined reference to `diagnostics_is_initialized'
/boinc-7.0.27+dfsg/api/boinc_api.cpp:477: undefined reference to `boinc_init_diagnostics'
/static_boinc_install_dir/lib/libboinc_api.a(boinc_api.o): In function `setup_shared_mem':
/boinc-7.0.27+dfsg/api/boinc_api.cpp:277: undefined reference to `attach_shmem(int, void**)'
/boinc-7.0.27+dfsg/api/boinc_api.cpp:271: undefined reference to `attach_shmem_mmap(char const*, void**)'
/static_boinc_install_dir/lib/libboinc_api.a(boinc_api.o): In function `boinc_init':
/boinc-7.0.27+dfsg/api/boinc_api.cpp:419: undefined reference to `diagnostics_is_initialized'
/boinc-7.0.27+dfsg/api/boinc_api.cpp:420: undefined reference to `boinc_init_diagnostics'
/static_boinc_install_dir/lib/libboinc_api.a(boinc_api.o): In function `handle_process_control_msg':
/boinc-7.0.27+dfsg/api/boinc_api.cpp:918: undefined reference to `boinc_info'
/boinc-7.0.27+dfsg/api/boinc_api.cpp:933: undefined reference to `boinc_info'
/static_boinc_install_dir/lib/libboinc_api.a(boinc_api.o): In function `resume_activities':
/boinc-7.0.27+dfsg/api/boinc_api.cpp:845: undefined reference to `suspend_or_resume_descendants(int, bool)'
/static_boinc_install_dir/lib/libboinc_api.a(boinc_api.o): In function `handle_process_control_msg':
/boinc-7.0.27+dfsg/api/boinc_api.cpp:941: undefined reference to `boinc_info'
/static_boinc_install_dir/lib/libboinc_api.a(boinc_api.o): In function `handle_trickle_down_msg':
/boinc-7.0.27+dfsg/api/boinc_api.cpp:890: undefined reference to `boinc_info'
/static_boinc_install_dir/lib/libboinc_api.a(boinc_api.o): In function `send_trickle_up_msg':
/boinc-7.0.27+dfsg/api/boinc_api.cpp:568: undefined reference to `boinc_info'
/static_boinc_install_dir/lib/libboinc_api.a(boinc_api.o): In function `handle_process_control_msg':
/boinc-7.0.27+dfsg/api/boinc_api.cpp:948: undefined reference to `boinc_info'
/boinc-7.0.27+dfsg/api/boinc_api.cpp:951: undefined reference to `diagnostics_set_aborted_via_gui'
collect2: error: ld returned 1 exit status

...

[Derrick Zwickl]

Hi,

Hmm.  I don't think that it's the BOINC version.  It looks like the libboinc_api.a static library is not finding symbols that should be in the libboinc.a static library at GARLI link time.  Is there a libboinc.a file in static_boinc_install_dir/lib/?

Derrick

--

[anon...@gmail.com]

Hi.

Yes, it should all be here:

ls -Ahln static_boinc_install_dir/lib/
total 7.3M
-rw-r--r-- 1 1000 1000 5.2M Jul 31 20:38 libboinc.a
-rw-r--r-- 1 1000 1000 380K Jul 31 20:38 libboinc_api.a
-rwxr-xr-x 1 1000 1000 1001 Jul 31 20:38 libboinc_api.la
-rw-r--r-- 1 1000 1000 157K Jul 31 20:38 libboinc_crypt.a
-rwxr-xr-x 1 1000 1000 1016 Jul 31 20:38 libboinc_crypt.la
-rwxr-xr-x 1 1000 1000  998 Jul 31 20:38 libboinc.la
-rw-r--r-- 1 1000 1000 4.0K Jul 31 20:38 libboinc_opencl.a
-rwxr-xr-x 1 1000 1000 1010 Jul 31 20:38 libboinc_opencl.la
-rw-r--r-- 1 1000 1000 1.6M Jul 31 20:38 libboinc_zip.a
-rwxr-xr-x 1 1000 1000 1001 Jul 31 20:38 libboinc_zip.la

<span style="font-family:arial,sans

...