deal.II installation on cray XC50 giving MPI_VERSION=0.0

[vachan potluri]

Hello,

I am trying to install deal.II on a cray XC50 supercomputer.

cmake -DCMAKE_INSTALL_PREFIX=~/bin/dealii-9.1.1 \

        -DPREFIX_PATH=/opt/cray/pe \

        -DCMAKE_CXX_COMPILER=/opt/cray/pe/craype/2.5.13/bin/CC \

        -DWITH_MPI=ON \

        -DWITH_PETSC=OFF -DPETSC_DIR=$PETSC_DIR -DPETSC_ARCH=$PETSC_ARCH \

        -DWITH_P4EST=ON -DP4EST_DIR=~/bin/p4est-2.2 \

        ~/source/dealii-9.1.1

I have attached detailed.log and summary.log. Although the configuring exits without errors, I can see in detailed.log that MPI_VERSION was not detected correctly. The compilers were correctly detected. All other variables are just blanks. The relevant snippet of detailed.log is as follows:

#        DEAL_II_WITH_MPI set up with external dependencies

#            MPI_VERSION = 0.0

#            MPI_C_COMPILER = /opt/cray/pe/craype/2.5.13/bin/cc

#            MPI_CXX_COMPILER = /opt/cray/pe/craype/2.5.13/bin/CC

#            MPI_Fortran_COMPILER = /opt/cray/pe/craype/2.5.13/bin/ftn

#            MPI_CXX_FLAGS = 

#            MPI_LINKER_FLAGS = 

#            MPI_INCLUDE_DIRS = 

#            MPI_USER_INCLUDE_DIRS = 

#            MPI_LIBRARIES = 

Is this an issue? How to fix this? I had loaded cray-mpich module before invoking cmake and switched PrgEnv-cray with PrgEnv-gnu.

[Daniel Arndt]

Vachan,

You can probably tell better than any of us if this works. Just try. It might quite well be that the compiler doesn't need any extra flags for MPI. Also, it looks like the MPI version could not be detected. This implies that the library doesn't use newer features but should still work. If you know to which standard the MPI installation is conforming, you could try to set it via

cmake -DMPI_VERSION=... 

yourself.

If there any more problems, feel free to tell us.

Best,

Daniel

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dealii+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/71a50bd5-acf1-4cd5-b3a8-bcb0fd295399%40googlegroups.com.

[Wolfgang Bangerth]

On 2/6/20 6:18 AM, Daniel Arndt wrote:
>
> You can probably tell better than any of us if this works. Just try. It might
> quite well be that the compiler doesn't need any extra flags for MPI. Also, it
> looks like the MPI version could not be detected. This implies that the
> library doesn't use newer features but should still work. If you know to which
> standard the MPI installation is conforming, you could try to set it via
>
> cmake -DMPI_VERSION=...
>
> yourself.
> If there any more problems, feel free to tell us.

But separately, we try to obtain the MPI version from the file mpi.h via the
following cmake code in cmake/modules/FindMPI.h:

DEAL_II_FIND_FILE(MPI_MPI_H
NAMES mpi.h
HINTS ${MPI_CXX_INCLUDE_PATH} ${MPI_C_INCLUDE_PATH}
)
IF(NOT MPI_MPI_H MATCHES "-NOTFOUND" AND NOT DEFINED MPI_VERSION)
FILE(STRINGS "${MPI_MPI_H}" MPI_VERSION_MAJOR_STRING
REGEX "#define.*MPI_VERSION")
STRING(REGEX REPLACE "^.*MPI_VERSION[ ]+([0-9]+).*" "\\1"
MPI_VERSION_MAJOR "${MPI_VERSION_MAJOR_STRING}"
)
FILE(STRINGS ${MPI_MPI_H} MPI_VERSION_MINOR_STRING
REGEX "#define.*MPI_SUBVERSION")
STRING(REGEX REPLACE "^.*MPI_SUBVERSION[ ]+([0-9]+).*" "\\1"
MPI_VERSION_MINOR "${MPI_VERSION_MINOR_STRING}"
)
SET(MPI_VERSION "${MPI_VERSION_MAJOR}.${MPI_VERSION_MINOR}")
ENDIF()


It would be interesting to see why that files. Can you try and find out which
mpi.h on your system is being used, and attach it to a reply?

Best
W.

--
------------------------------------------------------------------------
Wolfgang Bangerth email: bang...@colostate.edu
www: http://www.math.colostate.edu/~bangerth/

[vachan potluri]

 If you know to which standard the MPI installation is conforming, you could try to set it via
cmake -DMPI_VERSION=... 
yourself.

The MPI version is 3.1. But will this be of use? After all, the include paths, linker flags and library variables will still be blank.

But separately, we try to obtain the MPI version from the file mpi.h via the 
following cmake code in cmake/modules/FindMPI.h

I tried this. The mpi.h file is present in /opt/cray/pe/mpt/default/gni/mpich-gnu/5.1/include. So I modified my cmake invocation as follows.

cmake -DCMAKE_INSTALL_PREFIX=~/bin/dealii-9.1.1 \

        -DPREFIX_PATH=/opt/cray/pe \

        -DWITH_MPI=ON \

                -DMPI_DIR=/opt/cray/pe/mpt/default/gni/mpich-gnu/5.1 \

                -DMPI_CXX_INCLUDE_PATH=/opt/cray/pe/mpt/default/gni/mpich-gnu/5.1/include \

                -DCMAKE_CXX_COMPILER=/opt/cray/pe/craype/2.5.13/bin/CC \

                -DCMAKE_C_COMPILER=/opt/cray/pe/craype/2.5.13/bin/cc \

                -DCMAKE_Fortran_COMPILER=/opt/cray/pe/craype/2.5.13/bin/ftn \

        -DWITH_LAPACK=OFF -DLAPACK_DIR=/opt/cray/pe/libsci/17.12.1/GNU/6.1/x86_64 \

        -DWITH_PETSC=OFF -DPETSC_DIR=$PETSC_DIR -DPETSC_ARCH=$PETSC_ARCH \

        -DWITH_P4EST=OFF -DP4EST_DIR=~/bin/p4est-2.2 \

        ~/source/dealii-9.1.1

The output still shows this message:

-- Include /home/ComptGasDynLab/vachanpotluri/source/dealii-9.1.1/cmake/configure/configure_1_mpi.cmake

-- MPI_MPI_H not found! Call:

--     FIND_FILE(MPI_MPI_H NAMES mpi.h HINTS)

--   MPI_VERSION: 0.0

--   MPI_LIBRARIES: 

--   MPI_INCLUDE_DIRS: 

--   MPI_USER_INCLUDE_DIRS: 

--   MPI_CXX_FLAGS: 

--   MPI_LINKER_FLAGS: 

-- Found MPI

-- DEAL_II_WITH_MPI successfully set up with external dependencies.

and the detailed log still shows blanks:

#        DEAL_II_WITH_MPI set up with external dependencies

#            MPI_VERSION = 0.0

#            MPI_DIR = /opt/cray/pe/mpt/default/gni/mpich-gnu/5.1

#            MPI_C_COMPILER = /opt/cray/pe/craype/2.5.13/bin/cc

#            MPI_CXX_COMPILER = /opt/cray/pe/craype/2.5.13/bin/CC

#            MPI_Fortran_COMPILER = /opt/cray/pe/craype/2.5.13/bin/ftn

#            MPI_CXX_FLAGS = 

#            MPI_LINKER_FLAGS = 

#            MPI_INCLUDE_DIRS = 

#            MPI_USER_INCLUDE_DIRS = 

#            MPI_LIBRARIES = 

Even after giving the MPI_CXX_INCLUDE_PATH hint, why is cmake not able to detect the version? Am I missing something?

[Wolfgang Bangerth]

> The MPI version is 3.1. But will this be of use? After all, the include paths,
> linker flags and library variables will still be blank.

It's still possible that everything links correctly. What happens if you run
cmake and then compile? Does it work?

In the best of all worlds, everything continues to work, but deal.II
internally just doesn't use the MPI 3 functions and falls back to MPI 2
functions. There are few places where I would expect that to make any kind of
noticeable difference.

> I tried this. The mpi.h file is present in
> /opt/cray/pe/mpt/default/gni/mpich-gnu/5.1/include.

Can you attach it to a reply? It would be interesting to see why the version
detection didn't work. (Although I see that cmake complains that it can't find
the file, so that is probably the issue. I don't know why it can't find the
file...)

[vachan potluri]

Dear Prof. Bangerth,

Can you attach it to a reply? It would be interesting to see why the version 
detection didn't work. (Although I see that cmake complains that it can't find 
the file, so that is probably the issue. I don't know why it can't find the 
file...)

I really appreciate and value your involvement in this thread. I have attached mpi.h with this mail. I want to mention that I added this line in FindMPI.cmake just before DEAL_II_FIND_FILE(MPI_MPI_H ...): 

MESSAGE(STATUS "Searching for mpi.h in ${MPI_CXX_INCLUDE_PATH}")

and found the corresponding output by cmake to be as follows.

-- Include /home/ComptGasDynLab/vachanpotluri/source/dealii-9.1.1/cmake/configure/configure_1_mpi.cmake

-- Searching for mpi.h in 

-- Found MPI_MPI_H

--   MPI_VERSION: 3.1

--   MPI_LIBRARIES: 

--   MPI_INCLUDE_DIRS: 

--   MPI_USER_INCLUDE_DIRS: 

--   MPI_CXX_FLAGS: 

--   MPI_LINKER_FLAGS: 

-- Found MPI

-- DEAL_II_WITH_MPI successfully set up with external dependencies.

Notice that the path is not printed, the variable is empty! Probably the problem is not with reading the header but with the variable being changed somewhere else.

It's still possible that everything links correctly. What happens if you run 
cmake and then compile? Does it work? 

I have not tried compiling the source because there are some other things also that I wanted to figure out. I have started a new thread for this purpose here https://groups.google.com/forum/#!topic/dealii/MCYyPrZNyjg. I request you to have a look at this one too.

Thank you

[Bruno Turcksin]

Vachan,

I saw on the other thread that you were using CMake 3.5. I would advise to download a newer version of CMake. deal.II uses FindMPI from CMake and this was rewritten in CMake 3.10 to fix problems with Cray supercomputers.

Best,

Bruno

[Wolfgang Bangerth]

On 2/7/20 11:43 PM, vachan potluri wrote:
> I really appreciate and value your involvement in this thread. I have
> attached mpi.h with this mail. I want to mention that I added this line
> in FindMPI.cmake just before DEAL_II_FIND_FILE(MPI_MPI_H ...):
> MESSAGE(STATUS "Searching for mpi.h in ${MPI_CXX_INCLUDE_PATH}")
> and found the corresponding output by cmake to be as follows.
> -- Include
> /home/ComptGasDynLab/vachanpotluri/source/dealii-9.1.1/cmake/configure/configure_1_mpi.cmake
> -- Searching for mpi.h in
> -- Found MPI_MPI_H
> --   MPI_VERSION: 3.1
> --   MPI_LIBRARIES:
> --   MPI_INCLUDE_DIRS:
> --   MPI_USER_INCLUDE_DIRS:
> --   MPI_CXX_FLAGS:
> --   MPI_LINKER_FLAGS:
> -- Found MPI
> -- DEAL_II_WITH_MPI successfully set up with external dependencies.
> Notice that the path is not printed, the variable is empty! Probably the
> problem is not with reading the header but with the variable being
> changed somewhere else.

Yes, this is how it looks like. But Then it seems to have found mpi.h
anyway, in the next line -- wasn't the original problem that it didn't
find the file?

> It's still possible that everything links correctly. What happens if
> you run
> cmake and then compile? Does it work?
>
> I have not tried compiling the source because there are some other
> things also that I wanted to figure out.

As I mentioned before, I don't think that the failed detection of the
MPI version has any practical effect as long as compiling and linking
succeeds and unless you want to run on many thousands of nodes.