Compilation problems on RHEL7

[Hans Peter Verne]

I have compiled deal.II 8.4.1 earlier, but version 9.0.0 gets me in a pickle.

My system is RedHat Enterprise Linux, version 7.5

I think I meet all the requirements:

$ cmake --version
cmake version 2.8.12.2

$ make --version
GNU Make 3.82

$ gcc --version

gcc (GCC) 7.2.0

I unpack the archive, creates a "build" subdirectory.

cmake runs fine, I use the following command line (split for readability):

cmake -DCMAKE_INSTALL_PREFIX=/opt/uio/modules/rhel7/packages/dealii/9.0.0\

           -DP4EST_DIR=/opt/uio/modules/rhel7/packages/p4est/1.1/ \

           -DDEAL_II_WITH_P4EST=ON -DDEAL_II_WITH_MPI=ON \

           -DTRILINOS_DIR=/opt/uio/modules/rhel7/packages/trilinos/12.4.2/  \

           ..

But "make" then produces the following error:

[ 38%] Building CXX object source/sundials/CMakeFiles/obj_sundials_release.dir/arkode.cc.o
In file included from /usr/include/tbb/enumerable_thread_specific.h:32:0,
                 from /opt/uio/modules/src/dealii-9.0.0/include/deal.II/base/thread_local_storage.h:23,
                 from /opt/uio/modules/src/dealii-9.0.0/include/deal.II/base/logstream.h:22,
                 from /opt/uio/modules/src/dealii-9.0.0/include/deal.II/lac/vector_memory.h:22,
                 from /opt/uio/modules/src/dealii-9.0.0/include/deal.II/lac/lapack_full_matrix.h:24,
                 from /opt/uio/modules/src/dealii-9.0.0/include/deal.II/base/array_view.h:22,
                 from /opt/uio/modules/src/dealii-9.0.0/include/deal.II/base/mpi.h:20,
                 from /opt/uio/modules/src/dealii-9.0.0/include/deal.II/sundials/arkode.h:20,
                 from /opt/uio/modules/src/dealii-9.0.0/source/sundials/arkode.cc:17:
/usr/include/tbb/concurrent_vector.h: In member function ‘tbb::concurrent_vector<T, A>::iterator tbb::concurrent_vector<T, A>::grow_by(tbb::concurrent_vector<T, A>::size_type)’:
/usr/include/tbb/concurrent_vector.h:667:38: error: operands to ?: have different types ‘tbb::internal::concurrent_vector_base_v3::size_type {aka long unsigned int}’ and ‘tbb::atomic<long unsigned int>’
         return iterator(*this, delta ? internal_grow_by( delta, sizeof(T), &initialize_array, NULL ) : my_early_size);
                                ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/tbb/concurrent_vector.h:667:38: note:   and each type can be converted to the other
/usr/include/tbb/concurrent_vector.h: In member function ‘tbb::concurrent_vector<T, A>::iterator tbb::concurrent_vector<T, A>::grow_by(tbb::concurrent_vector<T, A>::size_type, tbb::concurrent_vector<T, A>::const_reference)’:
/usr/include/tbb/concurrent_vector.h:680:38: error: operands to ?: have different types ‘tbb::internal::concurrent_vector_base_v3::size_type {aka long unsigned int}’ and ‘tbb::atomic<long unsigned int>’
         return iterator(*this, delta ? internal_grow_by( delta, sizeof(T), &initialize_array_by, static_cast<const void*>(&t) ) : my_early_size);
                                ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/tbb/concurrent_vector.h:680:38: note:   and each type can be converted to the other
make[2]: *** [source/sundials/CMakeFiles/obj_sundials_release.dir/arkode.cc.o] Error 1
make[1]: *** [source/sundials/CMakeFiles/obj_sundials_release.dir/all] Error 2

Apparently an error in the system file concurrent_vector.h, or perhaps some

inconsistency with expected types?  (I'm not a c++ programmer)

I can find the corresponding package file:

$ rpm -qf /usr/include/tbb/concurrent_vector.h
tbb-devel-4.1-9.20130314.el7.x86_64

I can force the build system to use the tbb that's bundled with the deal.ii

sources by uninstalling tbb-devel (and OCE-devel by dependency), but it

seems a harsh cure.

Still, doing that, I can move on.  Next problem:

[ 92%] Building CXX object examples/CMakeFiles/step-13.release.dir/step-13/step-13.cc.o
Linking CXX executable ../bin/step-10.release
../lib/libdeal_II.so.9.0.0: error: undefined reference to 'Ifpack::Create(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, Epetra_RowMatrix*, int, bool)'
../lib/libdeal_II.so.9.0.0: error: undefined reference to 'ML_Epetra::SetDefaults(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, Teuchos::ParameterList&, int*, double*, bool)'
../lib/libdeal_II.so.9.0.0: error: undefined reference to 'Epetra_Object::ReportError(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) const'
../lib/libdeal_II.so.9.0.0: error: undefined reference to 'Teuchos::mpiErrorCodeToString[abi:cxx11](int)'
../lib/libdeal_II.so.9.0.0: error: undefined reference to 'MueLu::Level::DeclareInput(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, MueLu::FactoryBase const*, MueLu::FactoryBase const*)'
../lib/libdeal_II.so.9.0.0: error: undefined reference to 'Teuchos::StrUtils::printLines(std::ostream&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'

etc etc ad nauseam

This seems trilinos-related, but I have no idea how to go on from here.

I googled a bit, but found no workable suggestions.

I have trilinos 12.4.2 (as seen in the cmake command), it worked fine with

deal.ii 8.4., and is apparently within the required version range.

Any suggestions on how to resolve this?  I'll be happy to try out

things and supply more sample output, if needed.

Regards,

Hans Peter Verne

IT staff - Dept. of Geosciences

University of Oslo

[Bruno Turcksin]

Hans,

Can you send the detailed.log file that was generated when you configured deal.II.

Best,

Bruno

[Hans Peter Verne]

On Wednesday, 10 October 2018 15:28:42 UTC+2, Bruno Turcksin wrote:

Hans,

Can you send the detailed.log file that was generated when you configured deal.II.

Sure!  Attached.

[Bruno Turcksin]

Hans,

Everything looks fine. I have a question, when you configured deal.II did you load a Trilinos module but then when you compiled Trilinos was not loaded? The other thing to try is to update cmake. cmake 2.8.12 is very old maybe that's the cause of the problem.

Bruno

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to a topic in the Google Groups "deal.II User Group" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dealii/RMlf8XVgXic/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dealii+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Wolfgang Bangerth]

On 10/10/2018 06:52 AM, Hans Peter Verne wrote:
>
> /usr/include/tbb/concurrent_vector.h: In member function
> ‘tbb::concurrent_vector<T, A>::iterator tbb::concurrent_vector<T,
> A>::grow_by(tbb::concurrent_vector<T, A>::size_type)’:
> /usr/include/tbb/concurrent_vector.h:667:38: error: operands to ?: have
> different types ‘tbb::internal::concurrent_vector_base_v3::size_type
> {aka long unsigned int}’ and ‘tbb::atomic<long unsigned int>’
>          return iterator(*this, delta ? internal_grow_by( delta,
> sizeof(T), &initialize_array, NULL ) : my_early_size);

I don't know the exact context, but have seen this error before. The
problem is that the TBB you have installed on your system can no longer
be compiled by the compiler (GCC 7.2) you are using. Thinking about it
some more, the details are all here:
https://github.com/dealii/dealii/issues/6711

In newer versions of deal.II, we just blacklist the version of the TBB
installed on the system and fall back on the one in bundled/. The patch
to do this is in
https://github.com/dealii/dealii/pull/6793

I believe that you can *force* the cmake script to use the bundled
version of TBB over the one on your system. I forgot the flag, but you
may be able to find it in the doc/readme.html file.

Best
W.

--
------------------------------------------------------------------------
Wolfgang Bangerth email: bang...@colostate.edu
www: http://www.math.colostate.edu/~bangerth/

[Matthias Maier]

> I believe that you can *force* the cmake script to use the bundled
> version of TBB over the one on your system. I forgot the flag, but you
> may be able to find it in the doc/readme.html file.

Correct, -DDEAL_II_FORCE_BUNDLED_THREADS=TRUE does the trick.

Best,
Matthias


[1] https://dealii.org/9.0.0/users/cmake.html

[Hans Peter Verne]

On Wednesday, 10 October 2018 17:29:38 UTC+2, Bruno Turcksin wrote:

Hans,

Everything looks fine. I have a question, when you configured deal.II did you load a Trilinos module but then when you compiled Trilinos was not loaded?

I never loaded a Trilinos module, we don't have it as a module. It just happens

to be installed in the same directory tree as our modules. I just pointed cmake

to the trilinis library, ie. cmake -DTRILINOS_DIR=/opt/uio/modules/rhel7/packages/trilinos/12.4.2/

 

The other thing to try is to update cmake. cmake 2.8.12 is very old maybe that's the cause of the problem.

So I try again with this:

$ cmake --version
cmake version 3.11.4

I also used the new 9.0.1 source, and the tip from Wolfgang and

Matthias (-DDEAL_II_FORCE_BUNDLED_THREADS=TRUE)

Linking still fails.  Running make VERBOSE=1, I can see

[ 85%] Linking CXX executable ../bin/step-9.release
cd /opt/uio/modules/src/dealii-9.0.1/build/examples && /opt/uio/modules/packages/cmake/3.11.4/bin/cmake -E cmake_link_script CMakeFiles/step-9.release.dir/link.txt --verbose=1

And then a crazy long line I hope I can be forgiven for pasting in:

/opt/uio/modules/packages/gcc/7.2.0/bin/c++    -rdynamic -Wl,--as-needed -rdynamic -fuse-ld=gold -Wl,-rpath -Wl,/usr/lib64/openmpi/lib -Wl,--enable-new-dtags -pthread -pthread  CMakeFiles/step-9.release.dir/step-9/step-9.cc.o  -o ../bin/step-9.release -Wl,-rpath,/opt/uio/modules/src/dealii-9.0.1/build/lib:/opt/uio/modules/rhel7/packages/p4est/1.1/FAST/lib:/usr/lib64/openmpi/lib:/opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib::::: ../lib/libdeal_II.so.9.0.1 /opt/uio/modules/rhel7/packages/p4est/1.1/FAST/lib/libp4est.so /opt/uio/modules/rhel7/packages/p4est/1.1/FAST/lib/libsc.so /usr/lib64/openmpi/lib/libmpi_usempi.so /usr/lib64/openmpi/lib/libmpi_mpifh.so -ldl -lz -lrt -lmetis /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libmuelu-adapters.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libmuelu-interface.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libmuelu.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libteko.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libstratimikos.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libstratimikosbelos.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libstratimikosaztecoo.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libstratimikosamesos.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libstratimikosml.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libstratimikosifpack.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libifpack2-adapters.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libifpack2.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libzoltan2.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libanasazitpetra.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libModeLaplace.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libanasaziepetra.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libanasazi.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libbelostpetra.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libbelosepetra.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libbelos.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libml.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libifpack.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libpamgen_extras.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libpamgen.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libamesos2.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libamesos.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libgaleri-xpetra.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libgaleri.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libaztecoo.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libisorropia.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libxpetra-sup.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libxpetra.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libthyratpetra.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libthyraepetraext.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libthyraepetra.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libthyracore.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libepetraext.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libtpetraext.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libtpetrainout.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libtpetra.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libkokkostsqr.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libtpetrakernels.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libtpetraclassiclinalg.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libtpetraclassicnodeapi.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libtpetraclassic.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libtriutils.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libzoltan.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libepetra.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libsacado.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/librtop.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libteuchoskokkoscomm.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libteuchoskokkoscompat.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libteuchosremainder.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libteuchosnumerics.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libteuchoscomm.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libteuchosparameterlist.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libteuchoscore.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libkokkosalgorithms.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libkokkoscontainers.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libkokkoscore.so /opt/uio/modules/rhel7/packages/trilinos/12.4.2/lib/libgtest.so -llapack -lblas /usr/lib64/openmpi/lib/libmpi_cxx.so -larpack -lgsl -lgslcblas -lnetcdf_c++ -lnetcdf -lopenblas -lgfortran -lquadmath -lm -lc -lgcc_s -lgcc /usr/lib64/openmpi/lib/libmpi.so

This gives the error:

../lib/libdeal_II.so.9.0.1: error: undefined reference to 'Ifpack::Create(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, Epetra_RowMatrix*, int, bool)'
../lib/libdeal_II.so.9.0.1: error: undefined reference to 'ML_Epetra::SetDefaults(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, Teuchos::ParameterList&, int*, double*, bool)'
../lib/libdeal_II.so.9.0.1: error: undefined reference to 'Epetra_Object::ReportError(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, int) const'
../lib/libdeal_II.so.9.0.1: error: undefined reference to 'Teuchos::mpiErrorCodeToString[abi:cxx11](int)'
../lib/libdeal_II.so.9.0.1: error: undefined reference to 'MueLu::Level::DeclareInput(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, MueLu::FactoryBase const*, MueLu::FactoryBase const*)'
../lib/libdeal_II.so.9.0.1: error: undefined reference to 'Teuchos::StrUtils::printLines(std::ostream&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
etc etc etc etc.....

Funny.  It looks to me as if all the trilinos libraries are presented to the linker.

But perhaps the order is wrong or something?

Thanks for your effort.  Does the above make any sense to you?

[Wolfgang Bangerth]

On 10/17/2018 09:21 AM, Hans Peter Verne wrote:
> This gives the error:
>
> ../lib/libdeal_II.so.9.0.1: error: undefined reference to
> 'Ifpack::Create(std::__cxx11::basic_string<char, std::char_traits<char>,
> std::allocator<char> >, Epetra_RowMatrix*, int, bool)'
> ../lib/libdeal_II.so.9.0.1: error: undefined reference to
> 'ML_Epetra::SetDefaults(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> >, Teuchos::ParameterList&,
> int*, double*, bool)'

Every one of the functions you list here have a std::string argument
that is qualified with the C++11 ABI qualifier. Did you per chance
compile Trilinos with a different compiler than the one you are using
here? Or, maybe more likely, did you compile Trilinos with a compiler
that didn't use C++11, whereas deal.II was compiled with a C++11 compiler?

[Hans Peter Verne]

On Wed, 17 Oct 2018 at 17:44, Wolfgang Bangerth <bang...@colostate.edu> wrote:
>
> Every one of the functions you list here have a std::string argument
> that is qualified with the C++11 ABI qualifier. Did you per chance
> compile Trilinos with a different compiler than the one you are using
> here? Or, maybe more likely, did you compile Trilinos with a compiler
> that didn't use C++11, whereas deal.II was compiled with a C++11 compiler?

Bingo! This didn't occur to me (I'm still not a c++ programmer).

I didn't note down what compiler I used for trilinos, but assuming it was
standard RedHat /usr/bin/g++, I can try again with

$ g++ --version
g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28)

And (abridged):

$ rm -r build
$ mkdir build
$ cd build
$ cmake (like before)
$ make; make install; make test
(...)
100% tests passed, 0 tests failed out of 13
Total Test time (real) = 259.76 sec

Still quite a few warnings and notes, but I won't worry about that :-)

Thanks a lot!


Regards,
--
Hans Peter Verne

There are three things I have learned never to discuss with people... religion,
politics and The Great Pumpkin! -- Linus van Pelt

[qin...@stanford.edu]

Hi Hans,

I also encounter the same issue, did it get resolved? Thanks.

Will

[Hans Peter Verne]

On Mon, 12 Nov 2018 at 05:52, <qin...@stanford.edu> wrote:
>
> Hi Hans,
>
> I also encounter the same issue, did it get resolved? Thanks.

Yes, as explained in my last message in this thread. It is important
that all libraries and object files are compiled with the same version
of the compiler (or at least similar version).

I don't know the details, but I got the impression this is a namespace
issue, ie. how the global functions and variables etc. are referred to.

If you can't select what compiler to use, you may have to recompile
all libraries. It would then be sensible to store the finished library
in a separate location -- or all other programs using them will fail.

Of course, I can't say if this will fix your problems. Good luck!