Compilation error in dealii development version (9.1.0-pre, shortrev c6b7876) using gcc/7.2.0 compiler and cuda/9.2

[Sambit Das]

Dear all,

I am trying to compile the latest development version of dealii (9.1.0-pre, shortrev c6b7876) using gcc/7.2.0, openmpi/3.0.0/gcc/7.2.0,  cuda/9.2 and cmake-3.12.2.

During compilation I get the following error:

[ 57%] Building CUDA object source/base/CMakeFiles/obj_base_debug.dir/cuda.cu.o

In file included from /gpfs/gpfs0/groups/gavini/dsambit/software/dealii/dealii/include/deal.II/base/cuda.h:19:0,

                 from /gpfs/gpfs0/groups/gavini/dsambit/software/dealii/dealii/source/base/cuda.cu:16:

/gpfs/gpfs0/groups/gavini/dsambit/software/dealii/build/include/deal.II/base/config.h:421:12: fatal error: mpi.h: No such file or directory

 #  include <mpi.h>

            ^~~~~~~

compilation terminated.

I have attached the detailed.log file. 

I have used the following configuration line:

cmake  -DDEAL_II_CXX_FLAGS_RELEASE="-O3" -DDEAL_II_WITH_CXX17=OFF -DCMAKE_C_COMPILER=mpicc -DCMAKE_CXX_COMPILER=mpicxx 

 -DCMAKE_Fortran_COMPILER=mpif90 -DDEAL_II_WITH_CUDA=ON -DDEAL_II_COMPONENT_EXAMPLES=OFF

 -DDEAL_II_WITH_MPI=ON -DDEAL_II_WITH_64BIT_INDICES=ON

 -DP4EST_DIR="/gpfs/gpfs0/groups/gavini/dsambit/software/p4est/installGcc7.2.0"  ../dealii

 Interestingly, if I remove -DDEAL_II_WITH_CUDA=ON, compilation is successful. I am wondering if I am missing any cuda related compilation flags.

Thanks a lot in advance,

Best,

Sambit

[Bruno Turcksin]

Sambit,

Can you try with a different version of CMake. We do not support CMake 3.12 with CUDA at the moment.

Best,

Bruno

[Sambit Das]

Hi Bruno,

Thank you for your reply. Now I tried CMake 3.9.6 but got the following compilation error

[ 54%] Building CUDA object source/base/CMakeFiles/obj_base_debug.dir/cuda.cu.o

In file included from /gpfs/gpfs0/software/rhel72/packages/cuda/9.2/include/crt/math_functions.h:8853:0,

                 from /gpfs/gpfs0/software/rhel72/packages/cuda/9.2/include/crt/common_functions.h:257,

                 from /gpfs/gpfs0/software/rhel72/packages/cuda/9.2/include/common_functions.h:50,

                 from /gpfs/gpfs0/software/rhel72/packages/cuda/9.2/include/cuda_runtime.h:115,

                 from <command-line>:0:

/gpfs/gpfs0/software/rhel72/packages/gcc/7.2.0/include/c++/7.2.0/cmath:45:15: fatal error: math.h: No such file or directory

 #include_next <math.h>

               ^~~~~~~~

compilation terminated.

Should I try CMake 3.10/11?

Thank you,

Sambit

[Jean-Paul Pelteret]

Hi Sambit,

Your new error looks like the same one reported (https://github.com/dealii/dealii/issues/7204) yesterday so I don’t think that using another version of CMake will fix it. I’m not quite sure what the answer to the problem is, since the discussion is ongoing.

Best,

Jean-Paul

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dealii+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Sambit Das]

Hi Jean-Paul,

Thanks for referring me to the github issue (https://github.com/dealii/dealii/issues/7204). It does seem they are related issues.
I will keep following the github issue.

Thank you,

Sambit

[Sambit Das]

Hi Bruno and Jean,

Based on the discussion in  (https://github.com/dealii/dealii/issues/7204) I used the temporary fix of  setting
SET(ZLIB_INCLUDE_DIR  "/usr/local/include") in FindZLIB.cmake, which addressed the above compilation issue.

However, I get a new compilation error now:

[ 59%] Building CXX object source/particles/CMakeFiles/obj_particle_debug.dir/particle.cc.o

/gpfs/gpfs0/groups/gavini/dsambit/software/dealii/dealii/include/deal.II/matrix_free/cuda_hanging_nodes_internal.h(286): error: no suitable user-defined conversion from "std::vector<unsigned int, std::allocator<unsigned int>>" to "std::vector<boost::ulong_long_type, std::allocator<boost::ulong_long_type>>" exists

          detected during:

            instantiation of "void dealii::CUDAWrappers::internal::HangingNodes<dim>::setup_constraints(std::vector<unsigned int, std::allocator<unsigned int>> &, const CellIterator &, unsigned int &) const [with dim=2, CellIterator=dealii::FilteredIterator<dealii::TriaActiveIterator<dealii::DoFCellAccessor<dealii::DoFHandler<2, 2>, false>>>]" 

/gpfs/gpfs0/groups/gavini/dsambit/software/dealii/dealii/include/deal.II/matrix_free/cuda_matrix_free.templates.h(288): here

            instantiation of "void dealii::CUDAWrappers::internal::ReinitHelper<dim, Number>::get_cell_data(const CellFilter &, unsigned int) [with dim=2, Number=float, CellFilter=dealii::FilteredIterator<dealii::TriaActiveIterator<dealii::DoFCellAccessor<dealii::DoFHandler<2, 2>, false>>>]" 

/gpfs/gpfs0/groups/gavini/dsambit/software/dealii/dealii/include/deal.II/matrix_free/cuda_matrix_free.templates.h(578): here

            instantiation of "void dealii::CUDAWrappers::MatrixFree<dim, Number>::reinit(const dealii::Mapping<dim, dim> &, const dealii::DoFHandler<dim, dim> &, const dealii::AffineConstraints<Number> &, const dealii::Quadrature<1> &, dealii::CUDAWrappers::MatrixFree<dim, Number>::AdditionalData) [with dim=2, Number=float]" 

/gpfs/gpfs0/groups/gavini/dsambit/software/dealii/dealii/source/matrix_free/cuda_matrix_free.cu(25): here

The same error occurs with both bundled BOOST as well linking to a manually installed BOOST/1.66.0. 

Do you think the above compilation error is related to the original issue in  (https://github.com/dealii/dealii/issues/7204)?

Thank you,

Sambit

[Sambit Das]

It seems the issue is related to -DDEAL_II_WITH_64BIT_INDICES=ON.

If I set -DDEAL_II_WITH_64BIT_INDICES=OFF, the compilation worked fine.

Best,

Sambit

[Daniel Arndt]

Sambit,

the first issue you observed is related to https://gitlab.kitware.com/cmake/cmake/issues/17538.

In particular, the MPI include directories might not be set correctly for the CUDA compiler.

This often happens when the CMAKE_CXX_COMPILER does not need additional include directories to

compiler MPI code, e.g. if the CMAKE_CXX_COMPILER is a MPI wrapper.

The third issue is fixed in https://github.com/dealii/dealii/pull/7213.

Best,

Daniel

[Sambit Das]

Hi Daniel,

The third issue is fixed in https://github.com/dealii/dealii/pull/7213.

Thank you for creating the fix.

the first issue you observed is related to https://gitlab.kitware.com/cmake/cmake/issues/17538.
In particular, the MPI include directories might not be set correctly for the CUDA compiler.
This often happens when the CMAKE_CXX_COMPILER does not need additional include directories to
compiler MPI code, e.g. if the CMAKE_CXX_COMPILER is a MPI wrapper.

Ah, I see. 

Best,

Sambit