BLAS and LAPACK in the MOOSE environment?

[john.m...@uconn.edu]

Hello,

I'm trying to compile an additional library ( in /contrib/scalfmm of https://github.com/xikaij/fredkin-koehler ) but only linking to the MOOSE environment. I hope to put this application on CIVET so it shouldn't have machine specific compile instructions -- we need it for a MultiApp transfer we'll be doing with the Ferret app.

Cmake wants

BLAS_LIBRARIES
LAPACK_LIBRARIES

It alleges that it finds BLAS_LIBRARIES are /opt/moose/gcc-6.2.0/lib64/libgomp.so, but then I get the following: Could NOT find BLAS (missing:  BLAS_INCLUDE_DIRS). A similar warning gets printed for LAPACK but I don't know where that library is.

I should note that this took me a bit of time to even get to this step because miniconda conflicts with conda, so I had to set -DCMAKE_IGNORE_PATH=/home/john/anaconda2/lib

There is also some "native" BLAS and LAPACK located in the scalFMM package when you set -SCALFMM_USE_BLAS=OFF but this ends up giving a compilation error of a conflict

John

[Miller, Jason M]

John,

While building the environment package, I strategically use BLAS/LAPACK libraries that come with what-ever-PETSc I am using at the time. For example; when building Trilinos, using mpich which was built with clang... I use the BLAS/LAPACK libraries already built while building PETSc, using mpich/clang.

So to answer your question, the BLAS_LAPACK libraries you are looking for should be at:

  $PETSC_DIR/lib/

     libflapack.a

     libfblas.a

Hope this helps!

Jason

--
You received this message because you are subscribed to the Google Groups "moose-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to moose-users+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/moose-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/47e867d8-0144-4ce1-ace0-7ca4117af033%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[john.m...@uconn.edu]

Excellent, that worked.

At least, -- SCALFMM_USE_BLA=ON is active and no warnings about missing includes for BLAS or LAPACK. Now I'm getting

undefined reference to symbol '_gfortran_transfer_character_write@@GFORTRAN_1.4'

which according to petsc mailing list means that I have multiple libgfortran.so and the wrong one is being linked. I looked through the cmake modules and there is no way to set the explicit linking. Currently the following are set as libraries:

-- SCALFMM_LIBRARIES   =    -Wl,-rpath  -Wl,/opt/moose/mpich/mpich-3.2/gcc-opt/lib  -Wl,--enable-new-dtags /opt/moose/mpich/mpich-3.2/gcc-opt/lib/libmpicxx.so /opt/moose/mpich/mpich-3.2/gcc-opt/lib/libmpi.so /opt/moose/petsc/mpich_petsc-3.7.4/gcc-opt-superlu/lib/libflapack.a /opt/moose/petsc/mpich_petsc-3.7.4/gcc-opt-superlu/lib/libfblas.a
-- SCALFMM_INCLUDES    =    /opt/moose/mpich/mpich-3.2/gcc-opt/include

Thanks
John

On Monday, February 26, 2018 at 4:38:42 PM UTC+1, jason.miller wrote:

John,

While building the environment package, I strategically use BLAS/LAPACK libraries that come with what-ever-PETSc I am using at the time. For example; when building Trilinos, using mpich which was built with clang... I use the BLAS/LAPACK libraries already built while building PETSc, using mpich/clang.

So to answer your question, the BLAS_LAPACK libraries you are looking for should be at:

  $PETSC_DIR/lib/

     libflapack.a

     libfblas.a

Hope this helps!

Jason

On Mon, Feb 26, 2018 at 7:02 AM, <john.m...@uconn.edu> wrote:

Hello,

I'm trying to compile an additional library ( in /contrib/scalfmm of https://github.com/xikaij/fredkin-koehler ) but only linking to the MOOSE environment. I hope to put this application on CIVET so it shouldn't have machine specific compile instructions -- we need it for a MultiApp transfer we'll be doing with the Ferret app.

Cmake wants

BLAS_LIBRARIES
LAPACK_LIBRARIES

It alleges that it finds BLAS_LIBRARIES are /opt/moose/gcc-6.2.0/lib64/libgomp.so, but then I get the following: Could NOT find BLAS (missing:  BLAS_INCLUDE_DIRS). A similar warning gets printed for LAPACK but I don't know where that library is.

I should note that this took me a bit of time to even get to this step because miniconda conflicts with conda, so I had to set -DCMAKE_IGNORE_PATH=/home/john/anaconda2/lib

There is also some "native" BLAS and LAPACK located in the scalFMM package when you set -SCALFMM_USE_BLAS=OFF but this ends up giving a compilation error of a conflict

John

--
You received this message because you are subscribed to the Google Groups "moose-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to moose-users...@googlegroups.com.

[Miller, Jason M]

Its possible you need to destroy your build directory... I've seen cmake keep old stuff around while I struggle to make things work (by constantly updating environment variables). I quickly learned to clean my out-of-tree-build with every cmake configure attempt (rm -rf *). Especially after each configure change (like you did with pointing cmake towards PETSC_DIR's blas/lapack libs).

So, perhaps cmake is instructing the build to linking to a different fortran, than the ones I used to build blas/lapack and thats the conflict.

To unsubscribe from this group and stop receiving emails from it, send an email to moose-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/moose-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/bad7c3d9-bfb6-4c0f-98ea-5f11396a227a%40googlegroups.com.

[John Mangeri]

Yeah, I have seen that too.

I tried that. Still get the linking conflict

John

--
You received this message because you are subscribed to a topic in the Google Groups "moose-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/moose-users/qn3Nh-4lMos/unsubscribe.
To unsubscribe from this group and all its topics, send an email to moose-users+unsubscribe@googlegroups.com.

Visit this group at https://groups.google.com/group/moose-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/CALgZTAeyaJXTrr8-CD7y0-fNfkp0YfszK1zVbeHGokJqrnP-0g%40mail.gmail.com.

[Miller, Jason M]

Hmmm well, I got nothing else at the moment. I'll give "https://github.com/xikaij/fredkin-koehler" a try and see if I can come up with any more suggestions...

What is your cmake configure line?

To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/CAJC7CwYrzHmym%2BdrhY51KhNQGaf0x2xqkyx5g42hOH_zL19xjQ%40mail.gmail.com.

[John Mangeri]

The ./build_scalfmm is

#!/bin/bash

cd scalfmm
mkdir build
cd build
cmake .. -DCMAKE_IGNORE_PATH=/home/bingus/anaconda2/lib -DBLAS_LIBRARIES=/opt/moose/petsc/mpich_petsc-3.7.4/gcc-opt-superlu/lib/libfblas.a -DLAPACK_LIBRARIES=/opt/moose/petsc/mpich_petsc-3.7.4/gcc-opt-superlu/lib/libflapack.a -DSCALFMM_USE_MPI=ON -DSCALFMM_USE_FFT=OFF -DSCALFMM_USE_SSE=OFF -DSCALFMM_USE_AVX=OFF -DCMAKE_INSTALL_PREFIX=../Install
make
make install

thanks!

To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/CALgZTAeSMAs0Avjy%3D_v9p0-oO5cU3mGzgoVkkH9xgE4YRY0tsg%40mail.gmail.com.

[Miller, Jason M]

Thanks...

Okay, good! I get the same error. I'll try and figure something out.

To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/CAJC7CwYf3sR2k52ZEi03uiXX2L7o-bPn-NyE-hkGCHg%3DTR0SzQ%40mail.gmail.com.

[Miller, Jason M]

Well... it looks like when PETSc builds BLAS/LAPACK, it uses the system compilers instead of the ones I told PETSc to build with. I am starting to remember running into this with something else...

In fact... you wouldn't happen to have a:

 /opt/moose/lapack-3.7.1

 would you? I started building my own blas/lapack (totally forgot about that sorry). If you have that path... use that one as I just did (without your DCMAKE_IGNORE_PATH of course):

  cmake .. -DBLAS_LIBRARIES=/opt/moose/lapack-3.7.1/lib/libblas.so -DLAPACK_LIBRARIES=/opt/moose/lapack-3.7.1/lib/liblapack.so -DSCALFMM_USE_MPI=ON -DSCALFMM_USE_FFT=OFF -DSCALFMM_USE_SSE=OFF -DSCALFMM_USE_AVX=OFF -DCMAKE_INSTALL_PREFIX=../Install

-- Build files have been written to: /tmp/fredkin-koehler/contrib/scalfmm/build

[/tmp/fredkin-koehler/contrib/scalfmm/build]> make -j 12

Scanning dependencies of target scalfmm

[  4%] Building CXX object Src/CMakeFiles/scalfmm.dir/Utils/FLog.cpp.o

[ 13%] Building CXX object Src/CMakeFiles/scalfmm.dir/Utils/FMemStats.cpp.o

[ 13%] Building CXX object Src/CMakeFiles/scalfmm.dir/Utils/FMpi.cpp.o

[ 17%] Building CXX object Src/CMakeFiles/scalf

...

...

...

[ 95%] Linking CXX executable Release/RotationMPIFMM

[ 95%] Built target RotationMPIFMM

[100%] Linking CXX executable Release/ChebyshevInterpolationMPIFMM

[100%] Built target ChebyshevInterpolationMPIFMM

[/tmp/fredkin-koehler/contrib/scalfmm/build]>

[John Mangeri]

I do not have  /opt/moose/lapack-3.7.1

Should I? Haven't done a clean install of the MOOSE environment in a while...

Does the application actually compile as well?

I used a LAPACK/BLAS in my conda environment and also got a "successful" scalFMM compile but then it failed the application build.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/CALgZTAfc2EHtAW6VNZjYDrohrcXSDwyL%2BE0qVbDx9NS2_Mi74g%40mail.gmail.com.

[Miller, Jason M]

Its a fairly recent addition (lapack was added Nov 8th to the package), so I am not surprised.

Ok, so I am cloning ferret now, (FYI looks to be a syntax error in the instructions for cloning using SSH protocols). How do I instruct Ferret to use the fredkin-koehler contrib I built? I'll give it a try and let you know if moose's liblas provides a solution or not...

To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/CAJC7CwbOHb0kpd%3DfJuOWhV%3DU7%2BUtgkrTO4rGvMoouwTuF7WmUA%40mail.gmail.com.

[John Mangeri]

Oh no, you are getting ahead of me here.

I haven't gotten to that step yet :)

I was just trying to get fredkin-koehler to compile for me today. I think to make sure that app works, just try to run fredkin-koehler.i that is sitting in the main directory.

I'll rebuild the MOOSE environment and try to see if it compiles for me.

Thanks

John

To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/CALgZTAfRAizrrdJjEiQiwRXyW05bF_HqFyU4OKdyBbFRq74nZQ%40mail.gmail.com.

[Miller, Jason M]

In that case... I am able to config/make/install using /opt/moose/lapack-3.7.1:

[/tmp/fredkin-koehler/contrib/scalfmm/Install/bin]> ls

total 1740

DirectComputation

changeFmaFormat

generateDistributions

statisticsOnOctree

RotationFMM

ChebyshevInterpolationFMM

ChebyshevInterpolationMPIFMMSplit

RotationMPIFMM

ChebyshevInterpolationMPIFMM

..

.

Hope you get the same results!

Jason

To view this discussion on the web visit https://groups.google.com/d/msgid/moose-users/CAJC7CwY4adKgO2gJOpwQoCbrj50QKHg1Lbg-KuBz1NAPhk1QEg%40mail.gmail.com.