[Boost-users] libboost_date_time: undefined reference to __sync_fetch_and_add_4
Carsten Raas
I built a (complete) Boost 1.38 library with the Intel compilers
10.1.021, 11.0.073, and 11.0.081 for an EM64T (x86_64) Linux
architecture. But I end up with the same problem for all builds: the
shared version of libboost_date_time cannot be linked to this simple
demo program:
# cat conftest.cpp
-----------------------------------------------------------------
#include <boost/date_time/date_generators.hpp>
int main ()
{
const char *ptr = boost::date_time::nth_as_str(0); return 0;
}
-----------------------------------------------------------------
# icpc -V
Intel(R) C++ Intel(R) 64 Compiler Professional
for applications running on Intel(R) 64, Version 11.0
Build 20090131 Package ID: l_cproc_p_11.0.081
-----------------------------------------------------------------
Static linking:
# icpc -I /opt/boost/1.38/intel/include conftest.cpp \
/opt/boost/1.38/intel/lib/libboost_date_time.a
=> Works.
-----------------------------------------------------------------
Shared linking:
# icpc -I /opt/boost/1.38/intel/include conftest.cpp
-L /opt/boost/1.38/intel/lib -lboost_date_time
/opt/boost/1.38/intel/lib/libboost_date_time.so:
undefined reference to `__sync_fetch_and_add_4'
-----------------------------------------------------------------
Shared linking with gcc:
# g++ -I /opt/boost/1.38/intel/include conftest.cpp \
-L /opt/boost/1.38/intel/lib -lboost_date_time
/opt/boost/1.38/intel/lib/libboost_date_time.so:
undefined reference to `__sync_fetch_and_add_4'
collect2: ld returned 1 exit status
-----------------------------------------------------------------
Shared linking with gcc to libraries compiled with gcc itself:
# g++ -I /opt/boost/1.38/gcc/include conftest.cpp \
-L /opt/boost/1.38/gcc/lib -lboost_date_time
=> Works.
-----------------------------------------------------------------
I have been googling around for a while, but all I found was related to
GCC sync primitives (e.g. sync_lock_test_and_set, sync_fetch_and_add)
and different architectures (armel, sparc, etc.). I also tried the
workaround with
#define BOOST_SP_USE_PTHREADS
in boost/config/user.hpp, without success.
I also checked 1.37 and today's SVN version, but they show the same
problems.
Any hints what I should look at next?
Best regards,
-Carsten
_______________________________________________
Boost-users mailing list
Boost...@lists.boost.org
http://lists.boost.org/mailman/listinfo.cgi/boost-users
Vladimir Prus
> Hello!
>
> I built a (complete) Boost 1.38 library with the Intel compilers
> 10.1.021, 11.0.073, and 11.0.081 for an EM64T (x86_64) Linux
> architecture. But I end up with the same problem for all builds: the
> shared version of libboost_date_time cannot be linked to this simple
> demo program:
...
> I have been googling around for a while, but all I found was related to
> GCC sync primitives (e.g. sync_lock_test_and_set, sync_fetch_and_add)
> and different architectures (armel, sparc, etc.). I also tried the
> workaround with
> #define BOOST_SP_USE_PTHREADS
> in boost/config/user.hpp, without success.
Did you rebuild date_time itself with this change? Is the file actually
included anywhere? Does adding such define near the point where BOOST_SP_USE_PTHREADS
is tested helps?
- Volodya
tom fogal
> -----------------------------------------------------------------
> Shared linking:
> # icpc -I /opt/boost/1.38/intel/include conftest.cpp
> -L /opt/boost/1.38/intel/lib -lboost_date_time
>
> /opt/boost/1.38/intel/lib/libboost_date_time.so:
> undefined reference to `__sync_fetch_and_add_4'
> -----------------------------------------------------------------
> Shared linking with gcc:
>
> # g++ -I /opt/boost/1.38/intel/include conftest.cpp \
> -L /opt/boost/1.38/intel/lib -lboost_date_time
>
> /opt/boost/1.38/intel/lib/libboost_date_time.so:
> undefined reference to `__sync_fetch_and_add_4'
> collect2: ld returned 1 exit status
> -----------------------------------------------------------------
This is an internal function for gcc's atomics extensions.
http://gcc.gnu.org/onlinedocs/gcc-4.3.3/gcc/Atomic-Builtins.html
Can you try adding -lgcc (IIRC; maybe lgcc_s?) to your link line?
Looks like the more global problem is that boost_date_time is using
this symbol, but was not linked against it at build time. It's a bit
strange that your compiler isn't adding this implicitly though...
Cheers,
-tom
Carsten Raas
Tom Fogal <tfo...@alumni.unh.edu> writes:
> Carsten Raas <carste...@tu-dortmund.de> writes:
>> -----------------------------------------------------------------
>> Shared linking:
>> # icpc -I /opt/boost/1.38/intel/include conftest.cpp
>> -L /opt/boost/1.38/intel/lib -lboost_date_time
>>
>> /opt/boost/1.38/intel/lib/libboost_date_time.so:
>> undefined reference to `__sync_fetch_and_add_4'
>> -----------------------------------------------------------------
>> Shared linking with gcc:
>>
>> # g++ -I /opt/boost/1.38/intel/include conftest.cpp \
>> -L /opt/boost/1.38/intel/lib -lboost_date_time
>>
>> /opt/boost/1.38/intel/lib/libboost_date_time.so:
>> undefined reference to `__sync_fetch_and_add_4'
>> collect2: ld returned 1 exit status
>> -----------------------------------------------------------------
>
> This is an internal function for gcc's atomics extensions.
>
> http://gcc.gnu.org/onlinedocs/gcc-4.3.3/gcc/Atomic-Builtins.html
Thanks for the hints!
Just to make this point clear: The boost build is done completely with
the Intel compiler suite (using the intel-linux toolset):
./configure --prefix=/opt/boost/1.38/intel --with-toolset=intel-linux
bjam --toolset=intel-linux -prefix=/opt/boost/1.38/intel \
--build-type=complete --with-date_time install
The only thing I changed was
tools/build/v2/tools/intel-linux.jam
to get rid of all these warnings:
icpc: command line remark #10010: option '-Ob' is deprecated and will be
removed in a future release. See '-help deprecated'
Thus I replaced -Obx with -inline-level=x (x=0,1,2).
This makes the build free of warnings.
> Can you try adding -lgcc (IIRC; maybe lgcc_s?) to your link line?
I checked this, but both (-lgcc and -lgcc_s) are already in the linker
call of icpc as well as in the g++ linking (visible when using the -v
switch). Adding the libraries in addition (in various orders) did not help.
> Looks like the more global problem is that boost_date_time is using
> this symbol, but was not linked against it at build time. It's a bit
> strange that your compiler isn't adding this implicitly though...
What is confusing for me is that the static libraries work:
While
icpc -I /opt/boost/1.38/intel/include \
-L /opt/boost/1.38/intel/lib conftest.cpp -lboost_date_time
fails, this one works:
icpc --static -I /opt/boost/1.38/intel/include \
-L /opt/boost/1.38/intel/lib conftest.cpp -lboost_date_time
The same for using g++ instead of icpc (for the demo program).
-Carsten
Carsten Raas
Vladimir Prus wrote:
> Carsten Raas wrote:
>>
>> I built a (complete) Boost 1.38 library with the Intel compilers
>> 10.1.021, 11.0.073, and 11.0.081 for an EM64T (x86_64) Linux
>> architecture. But I end up with the same problem for all builds: the
>> shared version of libboost_date_time cannot be linked to this simple
>> demo program:
> ...
>> I have been googling around for a while, but all I found was related to
>> GCC sync primitives (e.g. sync_lock_test_and_set, sync_fetch_and_add)
>> and different architectures (armel, sparc, etc.). I also tried the
>> workaround with
>> #define BOOST_SP_USE_PTHREADS
>> in boost/config/user.hpp, without success.
>
> Did you rebuild date_time itself with this change?
Yes, I did.
I followed the hint of Peter Dimov in
http://lists.boost.org/Archives/boost/2008/04/135521.php
This was a different context, I just wanted to have this checked.
> Is the file actually included anywhere?
Not directly, but config.hpp includes user.hpp. And compiler_config.hpp
(of date_time) includes config.hpp. So I just was not sure whether this
#define would influence boost_date_time at all or not.
> Does adding such define near the point where BOOST_SP_USE_PTHREADS
> is tested helps?
Well, it is not used directly. But I found things like BOOST_HAS_THREADS
and I don't know where they come from. So I used user.hpp to define
BOOST_SP_USE_PTHREADS.
I could not find a better header file to define it.
detail/sp_counted_base.hpp refers to BOOST_SP_USE_PTHREADS, but it
includes config.hpp which includes user.hpp.
Best regards,
-Carsten
Peter Dimov
>> Did you rebuild date_time itself with this change?
>
> Yes, I did.
>
> I followed the hint of Peter Dimov in
> http://lists.boost.org/Archives/boost/2008/04/135521.php
That's very odd. It's true that Datetime uses shared_ptr, and that
shared_ptr uses __sync_fetch_and_add on some platforms, but it shouldn't on
Intel x64 because:
#elif defined(__GNUC__) && ( __GNUC__ * 100 + __GNUC_MINOR__ >= 401 ) &&
!defined( __arm__ ) && !defined( __hppa ) && ( !defined( __INTEL_COMPILER )
|| defined( __ia64__ ) )
# include <boost/detail/sp_counted_base_sync.hpp>
so I've no idea where the reference to __sync_fetch_and_add comes from. Can
you try rebuilding Datetime with
#define __sync_fetch_and_add ERROR
and see what in Boost is trying to make use of it?
(The other alternative is that libstdc++ itself is using
__sync_fetch_and_add in a header.)
Carsten Raas
> Carsten Raas:
>>> Did you rebuild date_time itself with this change?
>>
>> Yes, I did.
>>
>> I followed the hint of Peter Dimov in
>> http://lists.boost.org/Archives/boost/2008/04/135521.php
>
> That's very odd. It's true that Datetime uses shared_ptr, and that
> shared_ptr uses __sync_fetch_and_add on some platforms, but it shouldn't
> on Intel x64 because:
>
> #elif defined(__GNUC__) && ( __GNUC__ * 100 + __GNUC_MINOR__ >= 401 ) &&
> !defined( __arm__ ) && !defined( __hppa ) && ( !defined(
> __INTEL_COMPILER ) || defined( __ia64__ ) )
> # include <boost/detail/sp_counted_base_sync.hpp>
>
> so I've no idea where the reference to __sync_fetch_and_add comes from.
> Can you try rebuilding Datetime with
>
> #define __sync_fetch_and_add ERROR
>
> and see what in Boost is trying to make use of it?
>
> (The other alternative is that libstdc++ itself is using
> __sync_fetch_and_add in a header.)
I tried this one
echo "#define __sync_fetch_and_add ERROR" >> boost/config/user.hpp
echo "#define __sync_fetch_and_add_4 ERROR" >> boost/config/user.hpp
ending up with a complete library (without errors while building) and
/opt/boost/1.38/intel/lib/libboost_date_time.so:
undefined reference to `__sync_fetch_and_add_4'
again when trying to link.
------------------------------------------------------------------------
Then I deleted sp_counted_base_sync.hpp before building the library and
in the installation include directory after rebuilding: this did not
change anything. The build works, but the shared library cannot be linked.
------------------------------------------------------------------------
Finally, I tried to compile my demo program with the "-H" flag. Then I
grep'ed all included header files for "sync_fetch". The only file with
"sync_fetch" in it is atomicity.h and it appears three times, cf. the
dependency tree:
/opt/boost/1.38/intel/include/boost/date_time/date_generators.hpp
/usr/include/c++/4.2.1/stdexcept
/usr/include/c++/4.2.1/string
/usr/include/c++/4.2.1/bits/basic_string.h
/usr/include/c++/4.2.1/ext/atomicity.h
/usr/include/c++/4.2.1/sstream
/usr/include/c++/4.2.1/istream
/usr/include/c++/4.2.1/ios
/usr/include/c++/4.2.1/bits/ios_base.h
/usr/include/c++/4.2.1/ext/atomicity.h
/usr/include/c++/4.2.1/bits/locale_classes.h
/usr/include/c++/4.2.1/ext/atomicity.h
Again, the static boost library (from the same build) works (with the
same header files included).
------------------------------------------------------------------------
And note that all libboost_date_time libraries (static and shared) have
the string "__sync_fetch_and_add_4" in it:
grep __sync_fetch_and_add_4 /opt/boost/1.38/intel-11.0.081-shared/lib/*
------------------------------------------------------------------------
-Carsten
Peter Dimov
> Finally, I tried to compile my demo program with the "-H" flag. Then I
> grep'ed all included header files for "sync_fetch". The only file with
> "sync_fetch" in it is atomicity.h and it appears three times, cf. the
> dependency tree:
> /opt/boost/1.38/intel/include/boost/date_time/date_generators.hpp
> /usr/include/c++/4.2.1/stdexcept
> /usr/include/c++/4.2.1/string
> /usr/include/c++/4.2.1/bits/basic_string.h
> /usr/include/c++/4.2.1/ext/atomicity.h
> /usr/include/c++/4.2.1/sstream
> /usr/include/c++/4.2.1/istream
> /usr/include/c++/4.2.1/ios
> /usr/include/c++/4.2.1/bits/ios_base.h
> /usr/include/c++/4.2.1/ext/atomicity.h
> /usr/include/c++/4.2.1/bits/locale_classes.h
> /usr/include/c++/4.2.1/ext/atomicity.h
So, it is what I thought; <string> and <istream> use __sync_fetch_and_add,
not Boost.
> I tried this one
> echo "#define __sync_fetch_and_add ERROR" >> boost/config/user.hpp
> echo "#define __sync_fetch_and_add_4 ERROR" >> boost/config/user.hpp
user.hpp is probably included after the standard library headers, so the
#define didn't have a chance to break them. :-)
Carsten Raas
> Carsten Raas:
>
>> Finally, I tried to compile my demo program with the "-H" flag. Then I
>> grep'ed all included header files for "sync_fetch". The only file with
>> "sync_fetch" in it is atomicity.h and it appears three times, cf. the
>> dependency tree:
>> /opt/boost/1.38/intel/include/boost/date_time/date_generators.hpp
>> /usr/include/c++/4.2.1/stdexcept
>> /usr/include/c++/4.2.1/string
>> /usr/include/c++/4.2.1/bits/basic_string.h
>> /usr/include/c++/4.2.1/ext/atomicity.h
>> /usr/include/c++/4.2.1/sstream
>> /usr/include/c++/4.2.1/istream
>> /usr/include/c++/4.2.1/ios
>> /usr/include/c++/4.2.1/bits/ios_base.h
>> /usr/include/c++/4.2.1/ext/atomicity.h
>> /usr/include/c++/4.2.1/bits/locale_classes.h
>> /usr/include/c++/4.2.1/ext/atomicity.h
>
> So, it is what I thought; <string> and <istream> use
> __sync_fetch_and_add, not Boost.
Yes, these headers include atomicity.h.
But they don't trigger the problem directly:
#include <iostream>
#include <string>
#include <istream>
int main()
{ std::string s="test"; std::cout << s << std::endl; }
Works fine, if compiled with Intel icpc.
# icpc string_istream.cpp
------------------------------------------------------------------------
Linking with shared libboost_date_time fails:
# icpc -L /opt/boost/1.38/intel/lib string_istream.cpp \
-lboost_date_time-il
/opt/boost/1.38/intel/lib/libboost_date_time-il.so:
undefined reference to `__sync_fetch_and_add_4'
So, using libstdc++ headers itself does not trigger the problem.
In some sense the boost library build manages to "break" it.
And only for the shared library. Very strange.
------------------------------------------------------------------------
The block
#ifdef _GLIBCXX_ATOMIC_BUILTINS
static inline _Atomic_word
__exchange_and_add(volatile _Atomic_word* __mem, int __val)
{ return __sync_fetch_and_add(__mem, __val); }
static inline void
__atomic_add(volatile _Atomic_word* __mem, int __val)
{ __sync_fetch_and_add(__mem, __val); }
#else
in atomicity.h might be the bad part. So I disabled this block and only
activate the #else part:
#else
_Atomic_word
__attribute__ ((__unused__))
__exchange_and_add(volatile _Atomic_word*, int);
void
__attribute__ ((__unused__))
__atomic_add(volatile _Atomic_word*, int);
#endif
NOW it works.
------------------------------------------------------------------------
I wanted to use the bjam flags
<compileflags>-U _GLIBCXX_ATOMIC_BUILTINS
<compileflags>-u _GLIBCXX_ATOMIC_BUILTINS
first, but this didnt' work out.
So I provide different bits/c++config.h (where _GLIBCXX_ATOMIC_BUILTINS
is defined) to build a clean (linkable) Boost library:
/* Predefined symbols and macros:
wrapper for Intel compilers and Boost */
/* original c++config,
i.e. /usr/include/c++/4.2.1/x86_64-suse-linux/bits/c++config.h */
#include <bits/c++config.gcc.h>
/* Define if builtin atomic operations are supported on this host. */
#if defined(__INTEL_COMPILER)
#undef _GLIBCXX_ATOMIC_BUILTINS
#endif
------------------------------------------------------------------------
I don't think this is a good solution.
But I cannot figure out where is the source of the problem:
libstdc++42 <-> boost <-> icpc
As the static libraries worked, one could guess it's a compiler bug. But
then nobody would have been able to build shared boost with the Intel
compiler suite (version 10 and 11)!?
-Carsten
Peter Dimov
> Yes, these headers include atomicity.h.
> But they don't trigger the problem directly:
>
> #include <iostream>
> #include <string>
> #include <istream>
> int main()
> { std::string s="test"; std::cout << s << std::endl; }
>
> Works fine, if compiled with Intel icpc.
If <string> uses reference counting, it will use __sync_fetch_and_add on
copy:
#include <string>
int main()
{ std::string s="test"; std::string s2( s ); }
The use in <istream> is probably triggered by installing a locale facet,
something the DateTime library does.