PETSc is not compatible with current MPI

[yuesu jin]

Dear all,

  I installed the Deal.II on a cluster. 

The first thing I found is that, the /dealii-9.2.0/cmake/configure/configure_1_mpi.cmake automatically set the DEAL_II_WITH_MPI argument as "off", therefore the MPI_FOUND module cannot run. I switched it on and cmake can find the MPI library. 

The second thing I found is very strange. Cmake gives the message "Could not find a sufficient PETSC installation: PETSC is compiled against a different MPI library than the one deal.II picked up." The PETSc library is integrated in the cluster's public library, I can add it by the command "module add". Does this message mean that I need to compile a new version of PETSc different from the one in the public library?  

Thanks!

Best 

**************************************************************************************************

-- Include /home/yjin6/DEALII/dealii-9.2.0/cmake/configure/configure_3_petsc.cmake
-- Found PETSC_LIBRARY
-- Found PETSC_INCLUDE_DIR_ARCH
-- Found PETSC_INCLUDE_DIR_COMMON
-- Found PETSC_PETSCVARIABLES
-- Found PETSC_LIBRARY_mkl_scalapack_lp64
-- Found PETSC_LIBRARY_mkl_blacs_intelmpi_lp64
-- Found PETSC_LIBRARY_mkl_intel_lp64
-- Found PETSC_LIBRARY_mkl_sequential
-- Found PETSC_LIBRARY_mkl_core
-- Found PETSC_LIBRARY_umfpack
-- Found PETSC_LIBRARY_klu
-- Found PETSC_LIBRARY_cholmod
-- Found PETSC_LIBRARY_btf
-- Found PETSC_LIBRARY_ccolamd
-- Found PETSC_LIBRARY_colamd
-- Found PETSC_LIBRARY_camd
-- Found PETSC_LIBRARY_amd
-- Found PETSC_LIBRARY_suitesparseconfig
-- Found PETSC_LIBRARY_HYPRE
-- Found PETSC_LIBRARY_fftw3xc_intel_pic
-- Found PETSC_LIBRARY_fftw3x_cdft_lp64_pic
-- Found PETSC_LIBRARY_mkl_cdft_core
-- Found PETSC_LIBRARY_mkl_blacs_intelmpi_lp64
-- Found PETSC_LIBRARY_mkl_intel_lp64
-- Found PETSC_LIBRARY_mkl_sequential
-- Found PETSC_LIBRARY_mkl_core
-- Found PETSC_LIBRARY_mkl_intel_lp64
-- Found PETSC_LIBRARY_mkl_sequential
-- Found PETSC_LIBRARY_mkl_core
-- Found PETSC_LIBRARY_parmetis
-- Found PETSC_LIBRARY_metis
-- Performing Test PETSC_LIBRARY_dl
-- Performing Test PETSC_LIBRARY_dl - Success
-- Found PETSC_LIBRARY_iomp5
-- Performing Test PETSC_LIBRARY_pthread
-- Performing Test PETSC_LIBRARY_pthread - Success
-- Performing Test PETSC_LIBRARY_rt
-- Performing Test PETSC_LIBRARY_rt - Success
-- Found PETSC_LIBRARY_mpifort
-- Found PETSC_LIBRARY_mpi
-- Found PETSC_LIBRARY_ifport
-- Found PETSC_LIBRARY_ifcoremt_pic
-- Found PETSC_LIBRARY_imf
-- Found PETSC_LIBRARY_svml
-- Performing Test PETSC_LIBRARY_m
-- Performing Test PETSC_LIBRARY_m - Success
-- Found PETSC_LIBRARY_ipgo
-- Found PETSC_LIBRARY_irc
-- Found PETSC_LIBRARY_irc_s
-- Found PETSC_LIBRARY_iomp5
--   PETSC_VERSION: 3.10.2.0
--   PETSC_LIBRARIES: /project/cacds/apps/easybuild/software/PETSc/3.10.2-intel-2017b/lib/libpetsc.a;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/lib/intel64/libmkl_scalapack_lp64.so;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.so;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/lib/intel64/libmkl_intel_lp64.so;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/lib/intel64/libmkl_sequential.so;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/lib/intel64/libmkl_core.so;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/UMFPACK/Lib/libumfpack.a;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/KLU/Lib/libklu.a;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/CHOLMOD/Lib/libcholmod.a;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/BTF/Lib/libbtf.a;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/CCOLAMD/Lib/libccolamd.a;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/COLAMD/Lib/libcolamd.a;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/CAMD/Lib/libcamd.a;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/UMFPACK/Lib/libamd.a;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/SuiteSparse_config/libsuitesparseconfig.a;/project/cacds/apps/easybuild/software/Hypre/2.14.0-intel-2017b/lib/libHYPRE.a;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/lib/intel64/libfftw3xc_intel_pic.a;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/lib/intel64/libfftw3x_cdft_lp64_pic.a;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/lib/intel64/libmkl_cdft_core.so;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.so;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/lib/intel64/libmkl_intel_lp64.so;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/lib/intel64/libmkl_sequential.so;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/lib/intel64/libmkl_core.so;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/lib/intel64/libmkl_intel_lp64.so;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/lib/intel64/libmkl_sequential.so;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/lib/intel64/libmkl_core.so;/project/cacds/apps/easybuild/software/ParMETIS/4.0.3-intel-2017b/lib/libparmetis.a;/project/cacds/apps/easybuild/software/ParMETIS/4.0.3-intel-2017b/lib/libmetis.a;dl;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/lib/intel64/libiomp5.so;pthread;rt;/project/cacds/apps/intelmpi/2019.0.117/intel64/lib/libmpifort.so;/project/cacds/apps/intelmpi/2019.0.117/intel64/lib/release/libmpi.so;/project/cacds/apps/easybuild/software/ifort/2017.8.262-GCC-6.4.0-2.28/compilers_and_libraries_2017.8.262/linux/compiler/lib/intel64/libifport.so;/project/cacds/apps/easybuild/software/ifort/2017.8.262-GCC-6.4.0-2.28/compilers_and_libraries_2017.8.262/linux/compiler/lib/intel64/libifcoremt_pic.a;/project/cacds/apps/easybuild/software/ifort/2017.8.262-GCC-6.4.0-2.28/compilers_and_libraries_2017.8.262/linux/compiler/lib/intel64/libimf.so;/project/cacds/apps/easybuild/software/ifort/2017.8.262-GCC-6.4.0-2.28/compilers_and_libraries_2017.8.262/linux/compiler/lib/intel64/libsvml.so;m;/project/cacds/apps/easybuild/software/ifort/2017.8.262-GCC-6.4.0-2.28/compilers_and_libraries_2017.8.262/linux/compiler/lib/intel64/libipgo.a;/project/cacds/apps/easybuild/software/ifort/2017.8.262-GCC-6.4.0-2.28/compilers_and_libraries_2017.8.262/linux/compiler/lib/intel64/libirc.so;/project/cacds/apps/easybuild/software/ifort/2017.8.262-GCC-6.4.0-2.28/compilers_and_libraries_2017.8.262/linux/compiler/lib/intel64/libirc_s.a;dl;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/lib/intel64/libiomp5.so;pthread;rt
--   PETSC_INCLUDE_DIRS: /project/cacds/apps/easybuild/software/PETSc/3.10.2-intel-2017b/include;/project/cacds/apps/easybuild/software/PETSc/3.10.2-intel-2017b/include;/project/cacds/apps/easybuild/software/PETSc/3.10.2-intel-2017b/include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/UMFPACK/Include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/KLU/Include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/CHOLMOD/Include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/BTF/Include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/CCOLAMD/Include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/COLAMD/Include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/CAMD/Include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/AMD/Include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/SuiteSparse_config;/project/cacds/apps/easybuild/software/Hypre/2.14.0-intel-2017b/include;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/include/fftw;/project/cacds/apps/easybuild/software/ParMETIS/4.0.3-intel-2017b/include;/project/cacds/apps/easybuild/software/METIS/5.1.0-GCCcore-6.4.0/include;/project/cacds/apps/easybuild/software/Boost/1.66.0-intel-2017b/include
--   PETSC_USER_INCLUDE_DIRS: /project/cacds/apps/easybuild/software/PETSc/3.10.2-intel-2017b/include;/project/cacds/apps/easybuild/software/PETSc/3.10.2-intel-2017b/include;/project/cacds/apps/easybuild/software/PETSc/3.10.2-intel-2017b/include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/UMFPACK/Include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/KLU/Include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/CHOLMOD/Include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/BTF/Include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/CCOLAMD/Include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/COLAMD/Include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/CAMD/Include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/AMD/Include;/project/cacds/apps/easybuild/software/SuiteSparse/5.1.2-intel-2017b-METIS-5.1.0/SuiteSparse_config;/project/cacds/apps/easybuild/software/Hypre/2.14.0-intel-2017b/include;/project/cacds/apps/easybuild/software/imkl/2017.4.239-iimpi-2017b/mkl/include/fftw;/project/cacds/apps/easybuild/software/ParMETIS/4.0.3-intel-2017b/include;/project/cacds/apps/easybuild/software/METIS/5.1.0-GCCcore-6.4.0/include;/project/cacds/apps/easybuild/software/Boost/1.66.0-intel-2017b/include
-- Found PETSC
-- Could not find a sufficient PETSC installation: PETSC is compiled against a different MPI library than the one deal.II picked up.
-- DEAL_II_WITH_PETSC has unmet external dependencies.

--

Yuesu Jin,

Ph.D student,

University of Houston,

College of Natural Sciences and Mathematics,

Department of Earth and Atmospheric Sciences,
Houston, Texas 77204-5008


346-404-2062

[Bruno Turcksin]

Yuesu Jin,

You don't need to compile your own PETSc but you need to use the same MPI library than the one that PETSc is using. There are very hard to debug problems that appear when PETSc and deal.II use different MPI libraries. I think that you want to use this MPI library project/cacds/apps/intelmpi/2019.0.117/intel64/lib/release/libmpi.so So you probably need to load a module to use Intel MPI.

Best,

Bruno

[yuesu jin]

Dear Bruno,

    Thank you very much! I am using mpi/gcc right now. I will change to Intel mpi library.

Best regards

--
The deal.II project is located at http://www.dealii.org/
For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en
---
You received this message because you are subscribed to the Google Groups "deal.II User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dealii+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/a5e17b0d-bb81-45b0-a057-eb57de482f3do%40googlegroups.com.