Install scalapack and mumps fail with MPI fortran error

[yy.wayne]

Hello,

I try to install dealii on a CentOS virtual machine. It fails when installing scalapack and mumps, both related to mpi fortran. Is it due to openmpi compatibility?

[yy.wayne]

I upgrad gcc g++ gfortran to 9.3.1 but the error remains.

Another question is SuperLUDist wasn't auto enabled by Trilinos. Any hint on this ?

Best, 

Wayne

[yy.wayne]

Sorry didn't mention I'm using candi to install it.

[Wolfgang Bangerth]

We really don't know. The important part of the error message is this:

-- Check for working Fortran compiler: /usr/lib64/openmpi/bin/mpif90 - broken
CMake Error at
/home/123456/libs/candi-master/candi_install/cmake-3.20.5-linux-x86_64/share/cmake-3.20/Modules/CMakeTestFortranCompiler.cmake:51
(message):
The Fortran compiler

"/usr/lib64/openmpi/bin/mpif90"

is not able to compile a simple test program.

It fails with the following output:

Change Dir:
/home/123456/libs/candi-master/candi_install/tmp/build/scalapack-2.1.0/CMakeFiles/CMakeTmp

Run Build Command(s):/opt/rh/devtoolset-9/root/usr/bin/gmake -f Makefile
cmTC_238a2/fast && /opt/rh/devtoolset-9/root/usr/bin/gmake -f
CMakeFiles/cmTC_238a2.dir/build.make CMakeFiles/cmTC_238a2.dir/build
gmake[1]: Entering directory
'/home/123456/libs/candi-master/candi_install/tmp/build/scalapack-2.1.0/CMakeFiles/CMakeTmp'
Building Fortran object CMakeFiles/cmTC_238a2.dir/testFortranCompiler.f.o
/usr/lib64/openmpi/bin/mpif90 -g -O3 -fallow-argument-mismatch -fPIE
-c
/home/123456/libs/candi-master/candi_install/tmp/build/scalapack-2.1.0/CMakeFiles/CMakeTmp/testFortranCompiler.f
-o CMakeFiles/cmTC_238a2.dir/testFortranCompiler.f.o
gfortran: error: unrecognized command line option
‘-fallow-argument-mismatch’; did you mean ‘-Wno-argument-mismatch’?
gmake[1]: *** [CMakeFiles/cmTC_238a2.dir/build.make:78:
CMakeFiles/cmTC_238a2.dir/testFortranCompiler.f.o] Error 1
gmake[1]: Leaving directory
'/home/123456/libs/candi-master/candi_install/tmp/build/scalapack-2.1.0/CMakeFiles/CMakeTmp'
gmake: *** [Makefile:127: cmTC_238a2/fast] Error 2

For some reason, the Scalapack cmake script tries to call the Fortran compiler
with ‘-fallow-argument-mismatch’ as a flag, but the compiler does not
understand this. This is confusing to me because the gfortran documentation
suggests that this should be possible:
https://gcc.gnu.org/onlinedocs/gfortran/Option-Summary.html
On the other hand, the suggestion ‘-Wno-argument-mismatch’ is not listed on
that page. This may be related to
https://groups.google.com/g/comp.lang.fortran/c/vITMz5e1nHQ
So one could surmise that the cmake script used with Scalapack is simply not
prepared to deal with different compiler versions -- but I really don't know.

I don't know enough about Scalapack or candi to help much more. If you tried
to install Scalapack yourself, without the help from candi, what happens?

Best
W.


--
------------------------------------------------------------------------
Wolfgang Bangerth email: bang...@colostate.edu
www: http://www.math.colostate.edu/~bangerth/

[yy.wayne]

Thank you Wolfgang. I shift from mumps to superlu_dist. The trilinos is compiled with superlu_dist completely, but another error shows up. It says:

• Compiler setup check for DEAL_II_HAVE_USABLE_FLAGS_DEBUG failed with:
• Change Dir: /home/wyy/libs/candi/install/tmp/build/deal.II-master/check_compiler_setup/CheckCompilerSetupDEAL_II_HAVE_USABLE_FLAGS_DEBUG
• Unable to compile a simple test program. Trying to drop "-fuse-ld=gold" from the linker flags.

I thought it's due to different compiler during the installation, but I reinstall openmpi through yum and 

"module load mpi/openmpi_x86-64" so compilers should be consistent. 

I'm installing dealii on a cluster with candi. Install on my own virtual machine succeed.

I try to build deal.II directly without candi, and get "Insufficient gdf5 installation found". However HDF5 was 

installed by candi and PETSC & SLEPC are installed with no error, so HDF5 should be build with MPI.

[Wolfgang Bangerth]

> Thank you Wolfgang. I shift from mumps to superlu_dist. The trilinos is
> compiled with superlu_dist completely, but another error shows up. It says:
>

> * Compiler setup check for DEAL_II_HAVE_USABLE_FLAGS_DEBUG failed with:
> * Change Dir:
> /home/wyy/libs/candi/install/tmp/build/deal.II-master/check_compiler_setup/CheckCompilerSetupDEAL_II_HAVE_USABLE_FLAGS_DEBUG
> * Unable to compile a simple test program. Trying to drop "-fuse-ld=gold"
> from the linker flags.

The real error is actually above that:

/home/wyy/libs/candi/install/trilinos-release-13-2-0/lib/libstratimikosbelos.so:
error: undefined reference to 'Tpetra::MultiVector<float, int, long long,
Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>
>::get1dCopy(Teuchos::ArrayView<float> const&, unsigned long) const'
/home/wyy/libs/candi/install/trilinos-release-13-2-0/lib/libstratimikosbelos.so:
error: undefined reference to 'Tpetra::MultiVector<float, int, long long,
Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>
>::getLocalLength() const'
/home/wyy/libs/candi/install/trilinos-release-13-2-0/lib/libstratimikosbelos.so:
error: undefined reference to 'Tpetra::MultiVector<float, int, long long,
Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>
>::getGlobalLength() const'
/home/wyy/libs/candi/install/trilinos-release-13-2-0/lib/libstratimikosbelos.so:
error: undefined reference to 'Tpetra::MultiVector<float, int, long long,
Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>
>::getNumVectors() const'
/home/wyy/libs/candi/install/trilinos-release-13-2-0/lib/libstratimikosbelos.so:
error: undefined reference to 'Tpetra::MultiVector<float, int, long long,
Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace>
>::getVector(unsigned long) const'
/home/wyy/libs/candi/install/trilinos-release-13-2-0/lib/libstratimikosamesos2.so:
error: undefined reference to 'Amesos2::KLU2<Tpetra::CrsMatrix<float, int,
long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial,
Kokkos::HostSpace> >, Tpetra::MultiVector<float, int, long long,
Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >
>::KLU2(Teuchos::RCP<Tpetra::CrsMatrix<float, int, long long,
Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >
const>, Teuchos::RCP<Tpetra::MultiVector<float, int, long long,
Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >
>, Teuchos::RCP<Tpetra::MultiVector<float, int, long long,
Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >
const>)'
/home/wyy/libs/candi/install/trilinos-release-13-2-0/lib/libstratimikosamesos2.so:
error: undefined reference to 'Amesos2::TachoSolver<Tpetra::CrsMatrix<float,
int, long long, Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial,
Kokkos::HostSpace> >, Tpetra::MultiVector<float, int, long long,
Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >
>::TachoSolver(Teuchos::RCP<Tpetra::CrsMatrix<float, int, long long,
Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >
const>, Teuchos::RCP<Tpetra::MultiVector<float, int, long long,
Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >
>, Teuchos::RCP<Tpetra::MultiVector<float, int, long long,
Kokkos::Compat::KokkosDeviceWrapperNode<Kokkos::Serial, Kokkos::HostSpace> >
const>)'
collect2: error: ld returned 1 exit status

I must admit that I don't know enough about Trilinos and candi to say where
the problem is -- maybe others have seen this problem before.

Best
WB

[yy.wayne]

I ran into the wrong direction then... There are some trilinos warning on my vm machine 

but deal.ii install was success so I skip over them. Locating the error is very helpful. 

Thank you Wolfgang!

Best,

Wayne