Compiling deal.II with GCC version 9.3.0 results in missing C++11 features error

[Alex Cumberworth]

Hello,

I am trying to compile deal.II, version 9.2, on CentOS 8.3.2011 (the last version). I have gcc 9.3.0, which is relatively recent. However, when I run cmake, I end up with the following message:

  The current combination of compiler and C++ version flag is missing some
  features of the C++11 version of the language necessary for compiling
  deal.II.  Please ensure that the CMake variable DEAL_II_CXX_VERSION_FLAG is
  set to the correct flag to enable C++11 support; if it is already set and
  you still see this message then you will need to upgrade your compiler.

I don't really see how this can be, as this version of the compiler fully supports C++11. I also tried setting the flag DEAL_II_CXX_VERSION_FLAG, but using valid standard specifiers just results in errors like "The supplied flag "c++11" was not recognized by the compiler." To be clear, I ran

cmake .. -DDEAL_II_CXX_VERSION_FLAG=c++11

The version of cmake is 3.11.4.

Any thoughts would be greatly appreciated.

Alex

[Bruno Turcksin]

Alex,

Can you check that deal.II pick up the right compiler? It should be printed on your screen. You don't need to specify DEAL_II_CXX_VERSION_FLAG when using gcc.

Best,

Bruno

[Alex Cumberworth]

Hi Bruno,

From the output when I run a fresh configuration:

-- The CXX compiler identification is GNU 9.3.0
-- The C compiler identification is GNU 9.3.0
-- Check for working CXX compiler: /opt/ohpc/pub/compiler/gcc/9.3.0/bin/c++
-- Check for working CXX compiler: /opt/ohpc/pub/compiler/gcc/9.3.0/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done

and then bit further down

-- Include /home/ipausers/cumberworth/src/dealii-9.2.0/cmake/checks/check_01_cxx_features.cmake
-- Performing Test DEAL_II_HAVE_FLAG_stdcxx17
-- Performing Test DEAL_II_HAVE_FLAG_stdcxx17 - Success
-- Using C++ version flag "-std=c++17"
-- Performing Test DEAL_II_HAVE_CXX17_ATTRIBUTES
-- Performing Test DEAL_II_HAVE_CXX17_ATTRIBUTES - Failed
-- Performing Test DEAL_II_HAVE_CXX17_IF_CONSTEXPR
-- Performing Test DEAL_II_HAVE_CXX17_IF_CONSTEXPR - Success
-- Performing Test DEAL_II_NON_CONSTEXPR_LAMBDA
-- Performing Test DEAL_II_NON_CONSTEXPR_LAMBDA - Failed
-- Performing Test DEAL_II_HAVE_FLAG_stdcxx14
-- Performing Test DEAL_II_HAVE_FLAG_stdcxx14 - Success
-- Performing Test DEAL_II_HAVE_CXX14_MAKE_UNIQUE
-- Performing Test DEAL_II_HAVE_CXX14_MAKE_UNIQUE - Failed
-- Performing Test DEAL_II_HAVE_CXX14_CONSTEXPR_STDMAXMIN
-- Performing Test DEAL_II_HAVE_CXX14_CONSTEXPR_STDMAXMIN - Failed
-- Performing Test DEAL_II_HAVE_CXX14_CLANGAUTODEBUG_BUG_OK
-- Performing Test DEAL_II_HAVE_CXX14_CLANGAUTODEBUG_BUG_OK - Success
-- Performing Test DEAL_II_HAVE_FLAG_stdcxx11
-- Performing Test DEAL_II_HAVE_FLAG_stdcxx11 - Success
-- Performing Test DEAL_II_HAVE_CXX11_FEATURES
-- Performing Test DEAL_II_HAVE_CXX11_FEATURES - Failed
-- Performing Test DEAL_II_HAVE_CXX11_FUNCTIONAL_LLVMBUG20084_OK
-- Performing Test DEAL_II_HAVE_CXX11_FUNCTIONAL_LLVMBUG20084_OK - Failed
-- Performing Test DEAL_II_HAVE_CXX11_MACOSXC99BUG_OK
-- Performing Test DEAL_II_HAVE_CXX11_MACOSXC99BUG_OK - Success
-- Performing Test DEAL_II_HAVE_CXX11_ICCNUMERICLIMITSBUG_OK
-- Performing Test DEAL_II_HAVE_CXX11_ICCNUMERICLIMITSBUG_OK - Success
-- Performing Test DEAL_II_HAVE_CXX11_ICCLIBSTDCPP47CXX11BUG_OK
-- Performing Test DEAL_II_HAVE_CXX11_ICCLIBSTDCPP47CXX11BUG_OK - Success
CMake Error at cmake/checks/check_01_cxx_features.cmake:472 (MESSAGE):

Alex

[Bruno Turcksin]

Alex,

I think that deal.II picks the compiler correctly but it looks like
the stl library is too old. What version of libstdc++ is installed on
the system?

Best,

Bruno

Le ven. 21 mai 2021 à 13:41, Alex Cumberworth
<alexanderc...@gmail.com> a écrit :

> --
> The deal.II project is located at http://www.dealii.org/
> For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
> ---
> You received this message because you are subscribed to a topic in the Google Groups "deal.II User Group" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/dealii/3_Tr8HLd7UM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to dealii+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/8e4425ff-c296-4753-8f4d-f37668306373n%40googlegroups.com.

[Alex Cumberworth]

Hi Bruno,

It turns out that it was unable to find the header files, as once I set CPATH, C_INCLUDE_PATH, and CPLUS_INCLUDE_PATH, it was able to pass the tests.

Thanks for your help,

Alex

[Wolfgang Bangerth]

On 5/21/21 11:03 AM, Alex Cumberworth wrote:
>
> I don't really see how this can be, as this version of the compiler fully
> supports C++11. I also tried setting the flag DEAL_II_CXX_VERSION_FLAG, but
> using valid standard specifiers just results in errors like "The supplied flag
> "c++11" was not recognized by the compiler." To be clear, I ran
>
> cmake .. -DDEAL_II_CXX_VERSION_FLAG=c++11

Try
cmake .. -DDEAL_II_CXX_VERSION_FLAG="-std=c++11"
or
cmake .. -DDEAL_II_CXX_VERSION_FLAG=-std=c++11
and see whether that works.

GCC 9.2 is new enough so that it should work. There is probably something else
that doesn't work, and you can find out what that is by looking at
CMakeFiles/CMakeOutput.log
CMakeFiles/CMakeError.log

These files are lengthy, so it may take a bit of searching for the place where
the C++11 flags are set.

Best
W.

--
------------------------------------------------------------------------
Wolfgang Bangerth email: bang...@colostate.edu
www: http://www.math.colostate.edu/~bangerth/

[Alex Cumberworth]

It seems that the issues stem from not setting the include directories properly. Even with CPATH, C_INCLUDE_PATH, and CPLUS_INCLUDE_PATH set as I mentioned in my previous response, I am running into errors. The compilation proceeds for a while, but then at some point when cmath is included, it fails to find math.h,

[ 42%] Building CXX object source/sundials/CMakeFiles/obj_sundials_release.dir/arkode.cc.o

In file included from /home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/base/numbers.h:28,
                 from /home/ipausers/cumberworth/src/dealii-9.2.0/build/include/deal.II/base/config.h:449,
                 from /home/ipausers/cumberworth/src/dealii-9.2.0/source/sundials/arkode.cc:17:
/opt/ohpc/pub/compiler/gcc/9.3.0/include/c++/9.3.0/cmath:45:15: fatal error: math.h: No such file or directory
   45 | #include_next <math.h>
      |               ^~~~~~~~

compilation terminated.

If I use make VERBOSE=1, I can manually modify the command to include

-isystem /opt/ohpc/pub/compiler/gcc/9.3.0/include/c++/9.3.0

which solves that issue, but I don't know how to add this during configuration. I tried setting

CMAKE_CXX_FLAGS_DEBUGRELEASE     -isystem /opt/ohpc/pub/compiler/gcc/9.3.0/include/c++/9.3.0

But it seems to put the flag in the wrong order, as it still is unable to find math.h.

In compiling a later file, it is unable to find the sacado header file, although it exists on the system and cmake seemed to indicate it had found the headers. I have set

SACADO_CONFIG_H                  /opt/ohpc/pub/libs/gnu9/mpich/trilinos/13.0.0/include
SACADO_TRAD_HPP                  /opt/ohpc/pub/libs/gnu9/mpich/trilinos/13.0.0/include

but this doesn't work.

In file included from /home/ipausers/cumberworth/src/dealii-9.2.0/build/include/deal.II/base/config.h:449,
                 from /home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/base/exceptions.h:19,
                 from /home/ipausers/cumberworth/src/dealii-9.2.0/source/dofs/dof_objects.cc:16:
/home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/base/numbers.h:151:12: fatal error: Sacado.hpp: No such file or directory
  151 | #  include <Sacado.hpp>
      |            ^~~~~~~~~~~~
compilation terminated.

Again, I can set

-I/opt/ohpc/pub/libs/gnu9/mpich/trilinos/13.0.0/include

to compile, but then there are still further issues with other headers not being found.

Best,

Alex

[Bruno Turcksin]

Alex,

How did you install your compiler? You should not need to set these
paths yourself.

Best,

Bruno

Le mer. 26 mai 2021 à 13:17, Alex Cumberworth
<alexanderc...@gmail.com> a écrit :
>

> --
> The deal.II project is located at http://www.dealii.org/
> For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
> ---
> You received this message because you are subscribed to a topic in the Google Groups "deal.II User Group" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/dealii/3_Tr8HLd7UM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to dealii+un...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/afdb75a3-90dc-49a7-b0b2-6217a40d77dcn%40googlegroups.com.

[Wolfgang Bangerth]

On 5/26/21 11:17 AM, Alex Cumberworth wrote:
> It seems that the issues stem from not setting the include directories
> properly. Even with CPATH, C_INCLUDE_PATH, and CPLUS_INCLUDE_PATH set as I
> mentioned in my previous response, I am running into errors. The compilation
> proceeds for a while, but then at some point when cmath is included, it fails
> to find math.h,
>
> [ 42%] Building CXX object
> source/sundials/CMakeFiles/obj_sundials_release.dir/arkode.cc.o
>
> In file included from
> /home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/base/numbers.h:28,
>                  from
> /home/ipausers/cumberworth/src/dealii-9.2.0/build/include/deal.II/base/config.h:449,
>                  from
> /home/ipausers/cumberworth/src/dealii-9.2.0/source/sundials/arkode.cc:17:
> /opt/ohpc/pub/compiler/gcc/9.3.0/include/c++/9.3.0/cmath:45:15: fatal error:
> math.h: No such file or directory
>    45 | #include_next <math.h>
>       |               ^~~~~~~~
> compilation terminated.
>
> If I use make VERBOSE=1, I can manually modify the command to include
>
> -isystem /opt/ohpc/pub/compiler/gcc/9.3.0/include/c++/9.3.0
>
> which solves that issue, but I don't know how to add this during
> configuration.

I suspect that you will get an endless sequence of problems beyond just this
one if you go down the road of trying to address this with compiler flags. I
suspect that what is happening is that you compiled and installed GCC 9.3
yourself, and then moved everything from the directory you installed it in to
/opt/ohpc/pub/compiler/gcc/9.3.0? You will have to install GCC in a way so
that it knows where not only its header files are, but also the libraries, the
linker, etc.

What happens if you use that compiler (without passing extra flags) to compile
a trivial .cc file that only #includes <cmath>?

[Alex Cumberworth]

I am working on the cluster that is available at my research institute, so I don't actually have root access. It is CentOS 8.3.2011 with openHPC. With the weird stl header file issues, it seemed that the compiler module had not been properly loaded, and then when I did properly load it, I didn't start a fresh configuration, and so the problems persisted. Now that I have done that the issues with math.h are solved.

However, trilinos is loaded as a module, and even with the fresh configuration it is unable to find the header. Even with

//Path to a file.
SACADO_CONFIG_H:FILEPATH=/opt/ohpc/pub/libs/gnu9/mpich/trilinos/13.0.0/include/Sacado.hpp

//Path to a file.
SACADO_TRAD_HPP:FILEPATH=/opt/ohpc/pub/libs/gnu9/mpich/trilinos/13.0.0/include/Sacado_trad.hpp

in the CMakeCache.txt, and rerunning cmake, it is still unable to actually Sacado.hpp while compiling. I am also trying to contact the cluster administrators to double check they have installed trilinos correctly, but I still find it odd that even specifying the exact location of the headers in the cmake configuration still does not results in a successful compilation.

[Wolfgang Bangerth]

On 5/27/21 4:25 AM, Alex Cumberworth wrote:
>
> However, trilinos is loaded as a module, and even with the fresh configuration
> it is unable to find the header. Even with
>
> //Path to a file.
> SACADO_CONFIG_H:FILEPATH=/opt/ohpc/pub/libs/gnu9/mpich/trilinos/13.0.0/include/Sacado.hpp
>
> //Path to a file.
> SACADO_TRAD_HPP:FILEPATH=/opt/ohpc/pub/libs/gnu9/mpich/trilinos/13.0.0/include/Sacado_trad.hpp
>
> in the CMakeCache.txt, and rerunning cmake, it is still unable to actually
> Sacado.hpp while compiling. I am also trying to contact the cluster
> administrators to double check they have installed trilinos correctly, but I
> still find it odd that even specifying the exact location of the headers in
> the cmake configuration still does not results in a successful compilation.

Are any of the other Trilinos header files found? Is the file readable?

cmake does not communicate the location of individual header files to the
compiler, just include paths. So if the compiler can find one Trilinos header
file, it should find them all.

[Alex Cumberworth]

Hello,

I have been working with the cluster system administrators, but we are still having trouble with Trilinos. Trilinos is a package in the repository of the distribution, and we have installed it and loaded the module. The version is 13.0.0. It does seem to find trilinos, but for whatever reason, it doesn't find the include files. It seems like the variable TRILINOS_INCLUDE_DIRS needs to be set, but when I try to pass this as an argument, it is no processed, i.e.

cmake -DTRILINOS_INCLUDE_DIRS=/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/include  ..

I have also tried

cmake -DTRILINOS_DIR=/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0  ..

It is also unable to find the epetra header file. When I run cmake, I get the following output related to Trilinos:

- Include /home/ipausers/cumberworth/src/dealii-9.2.0/cmake/configure/configure_2_trilinos.cmake
-- Enabled Kokkos devices: OPENMP;SERIAL
CMake Warning at /opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/cmake/Kokkos/KokkosConfig.cmake:180 (MESSAGE):
  The installed Kokkos configuration does not support CXX extensions.
  Forcing -DCMAKE_CXX_EXTENSIONS=Off
Call Stack (most recent call first):
  /opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/cmake/ShyLU_NodeTacho/ShyLU_NodeTachoConfig.cmake:150 (INCLUDE)
  /opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/cmake/Amesos2/Amesos2Config.cmake:149 (INCLUDE)
  /opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/cmake/Stratimikos/StratimikosConfig.cmake:149 (INCLUDE)
  /opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/cmake/Teko/TekoConfig.cmake:149 (INCLUDE)
  /opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/cmake/MueLu/MueLuConfig.cmake:150 (INCLUDE)
  /opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/cmake/Stokhos/StokhosConfig.cmake:149 (INCLUDE)
  /opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/cmake/TrilinosCouplings/TrilinosCouplingsConfig.cmake:149 (INCLUDE)
  /opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/cmake/Trilinos/TrilinosConfig.cmake:130 (INCLUDE)
  cmake/macros/macro_find_package.cmake:27 (_FIND_PACKAGE)
  cmake/modules/FindTRILINOS.cmake:38 (FIND_PACKAGE)
  cmake/macros/macro_find_package.cmake:27 (_FIND_PACKAGE)
  cmake/configure/configure_2_trilinos.cmake:22 (FIND_PACKAGE)
  build6/CMakeFiles/CMakeTmp/evaluate_expression.tmp:1 (FEATURE_TRILINOS_FIND_EXTERNAL)
  cmake/macros/macro_evaluate_expression.cmake:30 (INCLUDE)
  cmake/macros/macro_configure_feature.cmake:237 (EVALUATE_EXPRESSION)
  cmake/configure/configure_2_trilinos.cmake:386 (CONFIGURE_FEATURE)
  cmake/macros/macro_verbose_include.cmake:19 (INCLUDE)
  CMakeLists.txt:124 (VERBOSE_INCLUDE)


-- EPETRA_CONFIG_H not found! Call:
--     FIND_FILE(EPETRA_CONFIG_H Epetra_config.h HINTS /include NO_DEFAULT_PATH NO_CMAKE_ENVIRONMENT_PATH NO_CMAKE_PATH NO_SYSTEM_ENVIRONMENT_PATH NO_CMAKE_SYSTEM_PATH NO_CMAKE_FIND_ROOT_PATH)
-- Found TRILINOS_LIBRARY_trilinoscouplings
-- Found TRILINOS_LIBRARY_stokhos_muelu
-- Found TRILINOS_LIBRARY_stokhos_muelu_pce_openmp
-- Found TRILINOS_LIBRARY_stokhos_muelu_mp_16_openmp
-- Found TRILINOS_LIBRARY_stokhos_ifpack2
-- Found TRILINOS_LIBRARY_stokhos_ifpack2_pce_openmp
-- Found TRILINOS_LIBRARY_stokhos_ifpack2_mp_16_openmp
-- Found TRILINOS_LIBRARY_stokhos_amesos2
-- Found TRILINOS_LIBRARY_stokhos_xpetra
-- Found TRILINOS_LIBRARY_stokhos_xpetra_pce_openmp
-- Found TRILINOS_LIBRARY_stokhos_xpetra_mp_16_openmp
-- Found TRILINOS_LIBRARY_stokhos_tpetra
-- Found TRILINOS_LIBRARY_stokhos_tpetraext_pce_openmp
-- Found TRILINOS_LIBRARY_stokhos_tpetra_pce_openmp
-- Found TRILINOS_LIBRARY_stokhos_tpetra_sd_pce_openmp
-- Found TRILINOS_LIBRARY_stokhos_tpetraext_mp_16_openmp
-- Found TRILINOS_LIBRARY_stokhos_tpetra_mp_16_openmp
-- Found TRILINOS_LIBRARY_stokhos_tpetra_sd_mp_16_openmp
-- Found TRILINOS_LIBRARY_stokhos_sacado
-- Found TRILINOS_LIBRARY_stokhos
-- Found TRILINOS_LIBRARY_muelu-adapters
-- Found TRILINOS_LIBRARY_muelu-interface
-- Found TRILINOS_LIBRARY_muelu
-- Found TRILINOS_LIBRARY_locathyra
-- Found TRILINOS_LIBRARY_locaepetra
-- Found TRILINOS_LIBRARY_localapack
-- Found TRILINOS_LIBRARY_loca
-- Found TRILINOS_LIBRARY_noxepetra
-- Found TRILINOS_LIBRARY_noxlapack
-- Found TRILINOS_LIBRARY_nox
-- Found TRILINOS_LIBRARY_phalanx
-- Found TRILINOS_LIBRARY_intrepid2
-- Found TRILINOS_LIBRARY_intrepid
-- Found TRILINOS_LIBRARY_teko
-- Found TRILINOS_LIBRARY_stratimikos
-- Found TRILINOS_LIBRARY_stratimikosbelos
-- Found TRILINOS_LIBRARY_stratimikosamesos2
-- Found TRILINOS_LIBRARY_stratimikosaztecoo
-- Found TRILINOS_LIBRARY_stratimikosamesos
-- Found TRILINOS_LIBRARY_stratimikosml
-- Found TRILINOS_LIBRARY_stratimikosifpack
-- Found TRILINOS_LIBRARY_ifpack2-adapters
-- Found TRILINOS_LIBRARY_ifpack2
-- Found TRILINOS_LIBRARY_anasazitpetra
-- Found TRILINOS_LIBRARY_ModeLaplace
-- Found TRILINOS_LIBRARY_anasaziepetra
-- Found TRILINOS_LIBRARY_anasazi
-- Found TRILINOS_LIBRARY_amesos2
-- Found TRILINOS_LIBRARY_tacho
-- Found TRILINOS_LIBRARY_shylu_nodehts
-- Found TRILINOS_LIBRARY_belosxpetra
-- Found TRILINOS_LIBRARY_belostpetra
-- Found TRILINOS_LIBRARY_belosepetra
-- Found TRILINOS_LIBRARY_belos
-- Found TRILINOS_LIBRARY_ml
-- Found TRILINOS_LIBRARY_ifpack
-- Found TRILINOS_LIBRARY_zoltan2
-- Found TRILINOS_LIBRARY_pamgen_extras
-- Found TRILINOS_LIBRARY_pamgen
-- Found TRILINOS_LIBRARY_amesos
-- Found TRILINOS_LIBRARY_galeri-xpetra
-- Found TRILINOS_LIBRARY_galeri-epetra
-- Found TRILINOS_LIBRARY_aztecoo
-- Found TRILINOS_LIBRARY_isorropia
-- Found TRILINOS_LIBRARY_xpetra-sup
-- Found TRILINOS_LIBRARY_xpetra
-- Found TRILINOS_LIBRARY_thyratpetra
-- Found TRILINOS_LIBRARY_thyraepetraext
-- Found TRILINOS_LIBRARY_thyraepetra
-- Found TRILINOS_LIBRARY_thyracore
-- Found TRILINOS_LIBRARY_trilinosss
-- Found TRILINOS_LIBRARY_tpetraext
-- Found TRILINOS_LIBRARY_tpetrainout
-- Found TRILINOS_LIBRARY_tpetra
-- Found TRILINOS_LIBRARY_kokkostsqr
-- Found TRILINOS_LIBRARY_tpetraclassiclinalg
-- Found TRILINOS_LIBRARY_tpetraclassicnodeapi
-- Found TRILINOS_LIBRARY_tpetraclassic
-- Found TRILINOS_LIBRARY_epetraext
-- Found TRILINOS_LIBRARY_triutils
-- Found TRILINOS_LIBRARY_shards
-- Found TRILINOS_LIBRARY_zoltan
-- Found TRILINOS_LIBRARY_epetra
-- Found TRILINOS_LIBRARY_sacado
-- Found TRILINOS_LIBRARY_rtop
-- Found TRILINOS_LIBRARY_kokkoskernels
-- Found TRILINOS_LIBRARY_teuchoskokkoscomm
-- Found TRILINOS_LIBRARY_teuchoskokkoscompat
-- Found TRILINOS_LIBRARY_teuchosremainder
-- Found TRILINOS_LIBRARY_teuchosnumerics
-- Found TRILINOS_LIBRARY_teuchoscomm
-- Found TRILINOS_LIBRARY_teuchosparameterlist
-- Found TRILINOS_LIBRARY_teuchosparser
-- Found TRILINOS_LIBRARY_teuchoscore
-- Found TRILINOS_LIBRARY_kokkosalgorithms
-- Found TRILINOS_LIBRARY_kokkoscontainers
-- Found TRILINOS_LIBRARY_kokkoscore
-- Found TRILINOS_LIBRARY_gtest
--   TRILINOS_VERSION: 13.0
--   TRILINOS_LIBRARIES: /opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libtrilinoscouplings.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_muelu.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_muelu_pce_openmp.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_muelu_mp_16_openmp.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_ifpack2.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_ifpack2_pce_openmp.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_ifpack2_mp_16_openmp.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_amesos2.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_xpetra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_xpetra_pce_openmp.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_xpetra_mp_16_openmp.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_tpetra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_tpetraext_pce_openmp.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_tpetra_pce_openmp.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_tpetra_sd_pce_openmp.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_tpetraext_mp_16_openmp.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_tpetra_mp_16_openmp.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_tpetra_sd_mp_16_openmp.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos_sacado.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstokhos.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libmuelu-adapters.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libmuelu-interface.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libmuelu.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/liblocathyra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/liblocaepetra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/liblocalapack.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libloca.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libnoxepetra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libnoxlapack.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libnox.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libphalanx.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libintrepid2.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libintrepid.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libteko.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstratimikos.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstratimikosbelos.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstratimikosamesos2.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstratimikosaztecoo.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstratimikosamesos.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstratimikosml.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libstratimikosifpack.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libifpack2-adapters.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libifpack2.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libanasazitpetra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libModeLaplace.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libanasaziepetra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libanasazi.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libamesos2.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libtacho.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libshylu_nodehts.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libbelosxpetra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libbelostpetra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libbelosepetra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libbelos.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libml.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libifpack.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libzoltan2.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libpamgen_extras.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libpamgen.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libamesos.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libgaleri-xpetra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libgaleri-epetra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libaztecoo.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libisorropia.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libxpetra-sup.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libxpetra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libthyratpetra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libthyraepetraext.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libthyraepetra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libthyracore.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libtrilinosss.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libtpetraext.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libtpetrainout.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libtpetra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libkokkostsqr.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libtpetraclassiclinalg.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libtpetraclassicnodeapi.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libtpetraclassic.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libepetraext.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libtriutils.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libshards.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libzoltan.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libepetra.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libsacado.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/librtop.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libkokkoskernels.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libteuchoskokkoscomm.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libteuchoskokkoscompat.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libteuchosremainder.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libteuchosnumerics.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libteuchoscomm.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libteuchosparameterlist.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libteuchosparser.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libteuchoscore.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libkokkosalgorithms.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libkokkoscontainers.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libkokkoscore.so;/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/lib/libgtest.so;/opt/ohpc/pub/libs/gnu9/openmpi4/hdf5/1.10.6/lib/libhdf5.so;/usr/lib64/libz.so;/opt/ohpc/pub/libs/gnu9/openblas/0.3.7/lib/libopenblas.so;/usr/lib64/libdl.so
--   TRILINOS_INCLUDE_DIRS: /include;/opt/ohpc/pub/libs/gnu9/openmpi4/hdf5/1.10.6/include;/usr/include;/opt/ohpc/pub/libs/gnu9/openmpi4/boost/1.73.0/include
--   TRILINOS_USER_INCLUDE_DIRS: /include;/opt/ohpc/pub/libs/gnu9/openmpi4/hdf5/1.10.6/include;/usr/include;/opt/ohpc/pub/libs/gnu9/openmpi4/boost/1.73.0/include
--   TRILINOS_LINKER_FLAGS: -lgfortran
-- Found TRILINOS
-- Check whether the found trilinos package contains all required modules:
-- Found Amesos
-- Found Epetra
-- Found Ifpack
-- Found AztecOO
-- Found Teuchos
-- Found ML
-- Found EpetraExt
-- Module ROL not found!
-- Found Sacado
-- Found Tpetra
-- Found MueLu
-- Found Zoltan
-- Module MUMPS not found!
-- Looking for KOKKOS_ENABLE_CUDA_LAMBDA
-- Looking for KOKKOS_ENABLE_CUDA_LAMBDA - not found
-- Performing Test TRILINOS_TPETRA_IS_FUNCTIONAL
-- Performing Test TRILINOS_TPETRA_IS_FUNCTIONAL - Failed
-- Tpetra was found but is not usable! Disabling Tpetra support.
-- Performing Test TRILINOS_MUELU_IS_FUNCTIONAL
-- Performing Test TRILINOS_MUELU_IS_FUNCTIONAL - Failed
-- MueLu was found but is not usable through Epetra! Disabling MueLu support.
-- SACADO_CONFIG_H not found! Call:
--     FIND_FILE(SACADO_CONFIG_H Sacado_config.h HINTS /include NO_DEFAULT_PATH NO_CMAKE_ENVIRONMENT_PATH NO_CMAKE_PATH NO_SYSTEM_ENVIRONMENT_PATH NO_CMAKE_SYSTEM_PATH NO_CMAKE_FIND_ROOT_PATH)
-- SACADO_TRAD_HPP not found! Call:
--     FIND_FILE(SACADO_TRAD_HPP Sacado_trad.hpp HINTS /include NO_DEFAULT_PATH NO_CMAKE_ENVIRONMENT_PATH NO_CMAKE_PATH NO_SYSTEM_ENVIRONMENT_PATH NO_CMAKE_SYSTEM_PATH NO_CMAKE_FIND_ROOT_PATH)
-- DEAL_II_WITH_TRILINOS successfully set up with external dependencies.

Best,

Alex

[Wolfgang Bangerth]

Alex,

> I have also tried
>
> cmake -DTRILINOS_DIR=/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0  ..
>
> It is also unable to find the epetra header file.

I can not tell why that would be so (but you should be able to find out by
searching for the place in CMakeFiles/CMakeErrors.log where it shows you the
command that was executed to find that file). It may be that the file just
doesn't exist. It may be that it's not readable. It may be that the compiler
finds an error in it.

If you can't figure out how to use that installation, why not install Trilinos
yourself in your home directory and take it from there?

[Alex Cumberworth]

Hello,

The file does exist and is readable. If I set a manual include flag it is able to find it:

CMAKE_CXX_FLAGS_DEBUGRELEASE  -I/opt/ohpc/pub/libs/gnu9/openmpi4/trilinos/13.0.0/include

then it is able to get past this point. From the output in my previous message, it seems that cmake is not looking in the right place for these header files, and I have no idea how to set this properly:

--   TRILINOS_INCLUDE_DIRS: /include;/opt/ohpc/pub/libs/gnu9/openmpi4/hdf5/1.10.6/include;/usr/include;/opt/ohpc/pub/libs/gnu9/openmpi4/boost/1.73.0/include
--   TRILINOS_USER_INCLUDE_DIRS: /include;/opt/ohpc/pub/libs/gnu9/openmpi4/hdf5/1.10.6/include;/usr/include;/opt/ohpc/pub/libs/gnu9/openmpi4/boost/1.73.0/include

There are further issues past this point, but perhaps if I understand the problem here that will help with the later issues.

Best,

Alex

[vachan potluri]

Hi Alex,

I previously ran into a lot of issues when I tried to install dealii on our institute's cluster. The OS was different though and I had problems with PETSc.

I don't know if this helps but this is the relevant section in dealii-9.2.0/cmake/modules/FindTRILINOS.cmake file which searches for epetra.

# Look for Epetra_config.h - we'll query it to determine MPI and 64bit
# indices support:
#
DEAL_II_FIND_FILE(EPETRA_CONFIG_H Epetra_config.h
  HINTS ${Trilinos_INCLUDE_DIRS}

  NO_DEFAULT_PATH NO_CMAKE_ENVIRONMENT_PATH NO_CMAKE_PATH
  NO_SYSTEM_ENVIRONMENT_PATH NO_CMAKE_SYSTEM_PATH NO_CMAKE_FIND_ROOT_PATH
  )

Notice the hint: it is Trilinos_INCLUDE_DIRS and not TRILINOS_INCLUDE_DIRS. If your dealii version also has Trilinos in smallcase, then maybe making this change will do the job. You may even try manually adding the full path in the cmake module file as a hint.

Hope this helps

Vachan

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---

You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dealii+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/1f525829-638b-42f4-badd-01d70d5de9efn%40googlegroups.com.

[Alex Cumberworth]

Hi Vachan,

Thanks for your response. This actually works. However, I still end up with an incompatibility of the MPI configuration between the Trilinos library and deal.ii. I ended up compiling Trilinos from source (12-4-1), but during compilation of deal.ii, I have a new error:

In file included from /home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/dofs/dof_accessor.templates.h:35,
                 from /home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/dofs/dof_accessor.h:2069,
                 from /home/ipausers/cumberworth/src/dealii-9.2.0/source/numerics/data_out.cc:18:
/home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/lac/read_write_vector.h:624:21: error: ‘Vector’ in namespace ‘Tpetra’ does not name a template type
  624 |       const Tpetra::Vector<Number, int, types::global_dof_index> &tpetra_vector,
      |                     ^~~~~~
/home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/lac/read_write_vector.h:624:27: error: expected ‘,’ or ‘...’ before ‘<’ token
  624 |       const Tpetra::Vector<Number, int, types::global_dof_index> &tpetra_vector,
      |                           ^
make[2]: *** [source/numerics/CMakeFiles/obj_numerics_debug.dir/build.make:82: source/numerics/CMakeFiles/obj_numerics_debug.dir/data_out.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:3317: source/numerics/CMakeFiles/obj_numerics_debug.dir/all] Error 2
make: *** [Makefile:149: all] Error 2

The same error occurs when I use version 13 of Trilinos.

Best,

Alex

[Alex Cumberworth]

Hello,

I recompiled trilinos without tpetra to sidestep this error, but ended up with another error involving epetra, and apparently epetra is minimal module required for trilinos with deal.ii, so I cannot do the same thing.The error:

[ 31%] Building CXX object source/numerics/CMakeFiles/obj_numerics_debug.dir/data_out_dof_data.cc.o
In file included from /home/ipausers/cumberworth/src/dealii-9.2.0/source/numerics/data_out_dof_data.cc:28:
/home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/numerics/data_out_dof_data.templates.h:429:33: error: ‘EpetraWrappers’ is not a member of ‘dealii::LinearAlgebra’
  429 |     VectorHelper<LinearAlgebra::EpetraWrappers::Vector>::extract(
      |                                 ^~~~~~~~~~~~~~
/home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/numerics/data_out_dof_data.templates.h:429:55: error: template argument 1 is invalid
  429 |     VectorHelper<LinearAlgebra::EpetraWrappers::Vector>::extract(
      |                                                       ^
/home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/numerics/data_out_dof_data.templates.h:430:28: error: ‘EpetraWrappers’ in namespace ‘dealii::LinearAlgebra’ does not name a type
  430 |       const LinearAlgebra::EpetraWrappers::Vector & /*vector*/,
      |                            ^~~~~~~~~~~~~~
/home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/numerics/data_out_dof_data.templates.h:430:51: error: expected unqualified-id before ‘&’ token
  430 |       const LinearAlgebra::EpetraWrappers::Vector & /*vector*/,
      |                                                   ^
/home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/numerics/data_out_dof_data.templates.h:430:50: error: expected ‘)’ before ‘&’ token
  430 |       const LinearAlgebra::EpetraWrappers::Vector & /*vector*/,
      |                                                  ^~
      |                                                  )
/home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/numerics/data_out_dof_data.templates.h:429:65: note: to match this ‘(’
  429 |     VectorHelper<LinearAlgebra::EpetraWrappers::Vector>::extract(
      |                                                                 ^
/home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/numerics/data_out_dof_data.templates.h:430:51: error: non-member function ‘void dealii::internal::DataOutImplementation::extract(...)’ cannot have ref-qualifier
  430 |       const LinearAlgebra::EpetraWrappers::Vector & /*vector*/,
      |                                                   ^
/home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/numerics/data_out_dof_data.templates.h:430:51: error: ‘extract’ is not a template function
make[2]: *** [source/numerics/CMakeFiles/obj_numerics_debug.dir/build.make:303: source/numerics/CMakeFiles/obj_numerics_debug.dir/data_out_dof_data.cc.o] Error 1

make[1]: *** [CMakeFiles/Makefile2:3317: source/numerics/CMakeFiles/obj_numerics_debug.dir/all] Error 2
make: *** [Makefile:149: all] Error 2

Best,

Alex

[Bruno Turcksin]

Alex,

This is really strange. I've never seen that error before. Can you try
to install deal.II 9.3
https://github.com/dealii/dealii/releases/tag/v9.3.0 and see if you
get the same error?

Best,

Bruno

Le lun. 7 juin 2021 à 05:54, Alex Cumberworth
<alexanderc...@gmail.com> a écrit :
>

> You received this message because you are subscribed to a topic in the Google Groups "deal.II User Group" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/dealii/3_Tr8HLd7UM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to dealii+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/10a02d5f-98e2-472b-a21e-edf146acad9bn%40googlegroups.com.

[Alex Cumberworth]

Hi Bruno,

I repeated the compilation with deal.ii 9.3 and had the same error. I looked at the documentation in deal.ii for the class EpetraWrappers, and apparently it requires MPI:

This class implements a wrapper to the Trilinos distributed vector class Epetra_FEVector. This class is derived from the LinearAlgebra::VectorSpaceVector class. Note however that Epetra only works with Number = double. This class requires Trilinos to be compiled with MPI support.

Since epetra is a required trilinos module, does this mean that if trilinos is included, it must be compiled with MPI (and then therefore also deal.ii, as cmake does not allow non-matching MPI configurations between them)?

I recompiled trilinos with mpi on (openmpi 4.0.5), but when attempting to recompile deal.ii with MPI on, I ended up with new errors. There are too many errors to paste here, but it begins with

In file included from /home/ipausers/cumberworth/include/Teuchos_Time.hpp:56,                                                                                                                                       
                 from /home/ipausers/cumberworth/include/Teuchos_TimeMonitor.hpp:70,                                                                                                                                 
                 from /home/ipausers/cumberworth/include/Teuchos_CommUtilities.hpp:45,                                                                                                                               
                 from /home/ipausers/cumberworth/include/Teuchos_CommHelpers.hpp:46,                                                                                                                                 
                 from /home/ipausers/cumberworth/include/Kokkos_TeuchosCommAdapters.hpp:47,                                                                                                                          
                 from /home/ipausers/cumberworth/include/Sacado_Fad_ScalarTraitsImp.hpp:627,                                                                                                                         
                 from /home/ipausers/cumberworth/include/Sacado_Fad_DFadTraits.hpp:144,                                                                                                                              
                 from /home/ipausers/cumberworth/include/Sacado.hpp:51,                                                                                                                                              
                 from /home/ipausers/cumberworth/src/dealii-9.2.0/include/deal.II/base/numbers.h:151,                                                                                                                
                 from /home/ipausers/cumberworth/src/dealii-9.2.0/build13/include/deal.II/base/config.h:449,                                                                                                         
                 from /home/ipausers/cumberworth/src/dealii-9.2.0/source/sundials/arkode.cc:17:                                                                                                                      
/home/ipausers/cumberworth/include/mpi.h:30:13: error: conflicting declaration ‘typedef int MPI_Datatype’                                                                                                            
   30 | typedef int MPI_Datatype;                                                                                                                                                                                    
      |             ^~~~~~~~~~~~                                                                                                    

I have included the rest as an attachment.

Best,

Alex

[Daniel Arndt]

Alex,

it looks like Trilinos and SUNDIALS are using different MPI implementations that are incompatible. There is not much we can do about this within deal.II.

Can you try to reinstall SUNDIALS or Trilinos using the same MPI implementation as the other one?

Best,

Daniel

To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/227af8d3-726e-4675-a54b-79f8626c9d8dn%40googlegroups.com.

[Alex Cumberworth]

Hi Daniel,

I have actually set

DEAL_II_WITH_SUNDIALS:BOOL=OFF

so I don't really understand why it is trying to compile with sundials. Sundials is actually not installed on my system, so deal.ii must be using the bundled version. I will try to compile sundials and see if that helps.

Best,

Alex

[Alex Cumberworth]

I noticed that I attached the error file for 9.2, and that the source of the error in 9.3 is different. In 9.2, something goes wrong when one of the bundled sundials files is compiled, while 9.3, it is when one of the bundled arborx files is compiled. Both involve the same MPI errors. Both are supposed to be turned off (output from cmake):

#      ( DEAL_II_WITH_ARBORX = OFF )

...

#      ( DEAL_II_WITH_SUNDIALS = OFF )

The errors in 9.3 start with

n file included from /home/ipausers/cumberworth/include/Teuchos_Time.hpp:56,                                                                                                                                        
                 from /home/ipausers/cumberworth/include/Teuchos_TimeMonitor.hpp:70,                                                                                                                                 
                 from /home/ipausers/cumberworth/include/Teuchos_CommUtilities.hpp:45,                                                                                                                               
                 from /home/ipausers/cumberworth/include/Teuchos_CommHelpers.hpp:46,                                                                                                                                 
                 from /home/ipausers/cumberworth/include/Kokkos_TeuchosCommAdapters.hpp:47,                                                                                                                          
                 from /home/ipausers/cumberworth/include/Sacado_Fad_ScalarTraitsImp.hpp:627,                                                                                                                         
                 from /home/ipausers/cumberworth/include/Sacado_Fad_DFadTraits.hpp:144,                                                                                                                              
                 from /home/ipausers/cumberworth/include/Sacado.hpp:51,                                                                                                                                              

                 from /home/ipausers/cumberworth/src/dealii-9.3.0/include/deal.II/base/numbers.h:145,                                                                                                                
                 from /home/ipausers/cumberworth/src/dealii-9.3.0/build_mpi/include/deal.II/base/config.h:494,                                                                                                       
                 from /home/ipausers/cumberworth/src/dealii-9.3.0/include/deal.II/arborx/access_traits.h:19,                                                                                                         
                 from /home/ipausers/cumberworth/src/dealii-9.3.0/source/arborx/access_traits.cc:16:                                                                                                                 

/home/ipausers/cumberworth/include/mpi.h:30:13: error: conflicting declaration ‘typedef int MPI_Datatype’                                                                                                            
   30 | typedef int MPI_Datatype;                                                                                                                                                                                    
      |             ^~~~~~~~~~~~                        

Best,

Alex                                                                                

[Wolfgang Bangerth]

On 6/7/21 8:41 AM, Alex Cumberworth wrote:
> I noticed that I attached the error file for 9.2, and that the source of the
> error in 9.3 is different. In 9.2, something goes wrong when one of the
> bundled sundials files is compiled, while 9.3, it is when one of the bundled
> arborx files is compiled. Both involve the same MPI errors. Both are supposed
> to be turned off (output from cmake):

These sorts of things happen because cmake caches stuff from one invocation to
another and doesn't notice that you pass flags on the command line that would
alter what has been cached before.

Just rm -r the entire build directory before you call cmake and see whether
that helps.

[Alex Cumberworth]

The cause of this particular issue actually appears to be some symlinks to the mpi include directory in my home directory. I am surprised that this was the cause of the errors as they still point to the same mpi directory that cmake finds. Once I removed them the errors disappeared. While I am still compiling, I have passed all previous points where I had errors.

Best,

Alex

[Wolfgang Bangerth]

On 6/7/21 10:02 AM, Alex Cumberworth wrote:
> The cause of this particular issue actually appears to be some symlinks to the
> mpi include directory in my home directory. I am surprised that this was the
> cause of the errors as they still point to the same mpi directory that cmake
> finds. Once I removed them the errors disappeared. While I am still compiling,
> I have passed all previous points where I had errors.
>

Good to know -- it had to be something particular about your installation
given that people compile with Trilinos all the time without seeing these
errors :-)

[Alex Cumberworth]

I have now run into another error, although I suspect it should be more straightforward to solve:

make[2]: *** No rule to make target '/home/ipausers/cumberworth/lib/libsacado.a', needed by 'lib/libdeal_II.g.so.9.3.0'.  Stop.
make[1]: *** [CMakeFiles/Makefile2:3238: source/CMakeFiles/deal_II.g.dir/all] Error 2

make: *** [Makefile:149: all] Error 2

There is libsacado.so in the directory I specified for trilinos libraries. I configured both trilinos and deal.ii to build shared libraries. I also have set

DEAL_II_PREFER_STATIC_LIBS       OFF

I'm not really sure why it seems to only look for the static version of the library.

Best,

Alex

[Wolfgang Bangerth]

On 6/7/21 10:35 AM, Alex Cumberworth wrote:
>
> make[2]: *** No rule to make target
> '/home/ipausers/cumberworth/lib/libsacado.a', needed by
> 'lib/libdeal_II.g.so.9.3.0'.  Stop.
> make[1]: *** [CMakeFiles/Makefile2:3238: source/CMakeFiles/deal_II.g.dir/all]
> Error 2
> make: *** [Makefile:149: all] Error 2
>
> There is libsacado.so in the directory I specified for trilinos libraries. I
> configured both trilinos and deal.ii to build shared libraries. I also have set
>
> DEAL_II_PREFER_STATIC_LIBS       OFF
>
> I'm not really sure why it seems to only look for the static version of the
> library.

I don't think that the deal.II configuration specifies this. Check the files
under lib/cmake in your Trilinos installation. For example, I find there the
following information that deal.II simply imports:

trilinos-12.8.1-mpi/lib> grep -r libsacado *
cmake/Sacado/SacadoTargets-release.cmake: IMPORTED_LOCATION_RELEASE
"${_IMPORT_PREFIX}/lib/libsacado.so.12.8.1"
cmake/Sacado/SacadoTargets-release.cmake: IMPORTED_SONAME_RELEASE
"libsacado.so.12"
cmake/Sacado/SacadoTargets-release.cmake:list(APPEND
_IMPORT_CHECK_FILES_FOR_sacado "${_IMPORT_PREFIX}/lib/libsacado.so.12.8.1" )
Binary file libsacado.so matches
Binary file libsacado.so.12 matches
Binary file libsacado.so.12.8.1 matches

[vachan potluri]

Alex,

I think this is a problem related to the cluster's OS. On Cray XC50, I had to explicitly set the link type to dynamic before installation, because by default Cray does a static link. I had to set

export XTPE_LINK_TYPE=dynamic
export CRAYPE_LINK_TYPE=dynamic

before the installation. You can try and see if anything similar is happening.

Regards,

Vachan

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dealii+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/59c24df0-17d6-5c39-4ef9-9ba5fb722ca8%40colostate.edu.

[Alex Cumberworth]

Hello,

I finally managed to get everything compiled and running. To solve the last issue, I started completely fresh with trilinos, and it no longer tried to find the static version of sacado. I think there must have been something leftover in the cmake files in the installation of trilinos from a previous build without the build shared libraries option in trilinos configuration.

To summarize the more generalizable issues from this:

- If there are issues when including trilinos in the build, consider compiling trilinos with MPI to ensure it will have the same MPI version as deal.ii

- If there are weird MPI issues with not finding header files, make sure that MPI was set on the first call of cmake (as I think one thing that happened here was I changed it later and the compiler did not get updated to the proper mpi wrapper, or at least I noticed this when compiling trilinos)

- If you are running on a cluster system with loadable modules, make sure things are properly loaded (and or unloaded) before running cmake. This could be especially relevant when there are multiple MPI implementations present

- Check whether the cmake variables are what you think they are (e.g., check uppercase vs lower case)

I never did get deal.ii to compile when including tpetra, but since I didn't need it I didn't attempt again with the other issues solved, so I can't say too much more about that.

One thing that is still unclear to me: is it possible to compile trilinos and deal.ii without MPI? It seemed here that there are wrapper classes that trilinos will only compile if MPI is enabled? If this is true, perhaps that should be included on this this page?

Thanks to everyone that helped!

Best,

Alex

[Bruno Turcksin]

Alex,

I am glad it is finally working. To answer your question, when using
Trilinos MPI is required. We just merged a fix that makes this clear:
https://github.com/dealii/dealii/pull/12381

Best,

Bruno

Le mer. 9 juin 2021 à 11:18, Alex Cumberworth
<alexanderc...@gmail.com> a écrit :
>

> You received this message because you are subscribed to a topic in the Google Groups "deal.II User Group" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/dealii/3_Tr8HLd7UM/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to dealii+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/97775ce8-94cd-4a78-ac3c-804cdd5aa74cn%40googlegroups.com.