Install error: missing cublas functions

[Pierson Guthrey]

I keep getting these issues in my installation

First I use 

make clean
echo -e 'BACKEND=cuda\nFORT=true' > make.inc
make generate -j

Then my cmake recipe is 

ml gcc/8.3.1
ml xl/2022.08.19-cuda-11.1.0
ml cmake/3.21.1
COMP_GCC_VER=8.3.1

cmake \
 -DCMAKE_C_COMPILER=/usr/tce/packages/gcc/gcc-${COMP_GCC_VER}/bin/gcc \
 -DCMAKE_CXX_COMPILER=/usr/tce/packages/gcc/gcc-${COMP_GCC_VER}/bin/g++ \
 -DGPU_TARGET:STRING=sm_70 \
 -DBUILD_SHARED_LIBS:BOOL=OFF \
 -DUSE_FORTRAN:BOOL=OFF \
 -DMAGMA_ENABLE_CUDA:BOOL=ON \
 '-DCMAKE_CXX_FLAGS:STRING=-L/usr/tce/packages/xl/xl-2022.08.19-cuda-11.1.1/lib_no_type_info -Wl,-rpath=/usr/tce/packages/xl/xl-2022.08.19-cuda-11.1.1/lib_no_type_info -L/usr/tce/packages/xl/xl-2022.08.19-cuda-11.1.1/alllibs -Wl,-rpath=/usr/tce/packages/xl/xl-2022.08.19-cuda-11.1.1/alllibs -L/usr/tce/packages/spectrum-mpi/ibm/spectrum-mpi-rolling-release/lib -Wl,-rpath=/usr/tce/packages/spectrum-mpi/ibm/spectrum-mpi-rolling-release/lib -L/usr/tcetmp/packages/lapack/lapack-3.6.0-xlf-15.1.5/lib -Wl,-rpath=/usr/tcetmp/packages/lapack/lapack-3.6.0-xlf-15.1.5/lib' \
 '-DLAPACK_LIBRARIES:STRING=/usr/tcetmp/packages/lapack/lapack-3.6.0-xlf-15.1.5/lib/liblapack.a;/usr/tcetmp/packages/blas/blas-3.6.0-xlf-15.1.5/lib/libblas.a;-lxlf90_r;-lxlfmath;-lxlopt;-lxl;-lomptarget-nvptx;-lxlcufdevice;-lxldevice;-libmc++;-lstdc++' \
 -DFORTRAN_CONVENTION:STRING=-DNOCHANGE \
 -DCMAKE_CUDA_COMPILER:PATH=/usr/tce/packages/cuda/cuda-11.1.1/bin/nvcc \
 -DCMAKE_CUDA_HOST_COMPILER:PATH=/usr/tce/packages/xl/xl-2022.08.19-cuda-11.1.1/bin/xlc++ \
 -DCMAKE_INSTALL_PREFIX="../../install/magma_cuda" \
 ../../magma

Note that there are some directory changes I haven't included.

The issue I get is:

/usr/workspace/boba_dev/boba_hip/magma/testing/testing_sgemv_batched.cpp: In function 'int main(int, char**)':
/usr/workspace/boba_dev/boba_hip/magma/testing/testing_sgemv_batched.cpp:166:17: error: 'cublasSgemvBatched' was not declared in this scope
                 cublasSgemvBatched(opts.handle, cublas_trans_const(opts.transA),
                 ^~~~~~~~~~~~~~~~~~
[ 71%] Building CXX object CMakeFiles/testing_zgesv_gpu.dir/testing/testing_zgesv_gpu.cpp.o
[ 71%] Linking CXX executable testing/testing_clacpy_batched
/usr/workspace/boba_dev/boba_hip/magma/testing/testing_sgemv_batched.cpp:166:17: note: suggested alternative: 'cublasSgemmBatched'
                 cublasSgemvBatched(opts.handle, cublas_trans_const(opts.transA),
                 ^~~~~~~~~~~~~~~~~~
                 cublasSgemmBatched
/usr/workspace/boba_dev/boba_hip/magma/testing/testing_sgemv_batched.cpp:185:17: error: 'cublasSgemvStridedBatched' was not declared in this scope
                 cublasSgemvStridedBatched(opts.handle, cublas_trans_const(opts.transA),
                 ^~~~~~~~~~~~~~~~~~~~~~~~~
[ 71%] Linking CXX executable testing/testing_dlacpy_batched
/usr/workspace/boba_dev/boba_hip/magma/testing/testing_sgemv_batched.cpp:185:17: note: suggested alternative: 'cublasSgemmStridedBatched'
                 cublasSgemvStridedBatched(opts.handle, cublas_trans_const(opts.transA),
                 ^~~~~~~~~~~~~~~~~~~~~~~~~
                 cublasSgemmStridedBatched

[Ahmad Abdelfattah]

Hi Pierson, 

This issue should’ve been resolved as of commit 2b60a6f. Are you sure you have a fresh clone from the repo? 

These cublas functions are fairly new, since cuda-11.7, I think. The tester files producing this error should now disable this call for versions older than 11.7.  

Let us know if you still have problems with the build. 

Thanks,

Ahmad Abdelfattah
Research Scientist
Innovative Computing Laboratory
University of Tennessee, USA
ah...@icl.utk.edu

--
You received this message because you are subscribed to the Google Groups "MAGMA User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to magma-user+...@icl.utk.edu.
To view this discussion on the web visit https://groups.google.com/a/icl.utk.edu/d/msgid/magma-user/480aab64-7d68-481f-b809-ec3253993efan%40icl.utk.edu.

[Pierson Guthrey]

A fresh clone got me past that error. 

Now I have undefined references at link time....

/usr/tce/packages/cuda/cuda-11.7.0/bin/nvcc -ccbin /usr/tce/packages/gcc/gcc-8.3.1/bin/g++ -x cu -std=c++17 -restrict -arch sm_70 --expt-extended-lambda --expt-relaxed-constexpr -DRAJA_CUDA -O3 -g -lineinfo -Iinstall/magma_cuda/include -DADD_ -Isource -isystem eigen -Iinstall/raja_cuda/include -Iinstall/camp_cuda/include -DBOBA_CAMP  -c examples/test_tensor_train_matrix.cpp -o test_tensor_train_matrix_cuda.o
/usr/tce/packages/cuda/cuda-11.7.0/bin/nvcc -ccbin /usr/tce/packages/gcc/gcc-8.3.1/bin/g++ -x cu -std=c++17 -restrict -arch sm_70 --expt-extended-lambda --expt-relaxed-constexpr -DRAJA_CUDA -O3 -g -lineinfo -Iinstall/magma_cuda/include -DADD_ -Isource -isystem eigen -Iinstall/raja_cuda/include -Iinstall/camp_cuda/include -DBOBA_CAMP  -c source/boba.cpp -o boba.o
/usr/tce/packages/cuda/cuda-11.7.0/bin/nvcc -ccbin /usr/tce/packages/gcc/gcc-8.3.1/bin/g++ -std=c++17 -restrict -arch sm_70 --expt-extended-lambda --expt-relaxed-constexpr -DRAJA_CUDA -O3 -g -lineinfo -o test_tensor_train_matrix_cuda.out test_tensor_train_matrix_cuda.o boba.o  -lcuda -lcusolver_static -lcudart_static -llapack_static -lmetis_static -lcublas_static -lcublasLt_static -lcusparse_static -lculibos -lnvToolsExt -lcuda -lcusolver_static -lcudart_static -llapack_static -lmetis_static -lcublas_static -lcublasLt_static -lcusparse_static -lculibos -lnvToolsExt -lgfortran -Linstall/magma_cuda/lib -lmagma -Linstall/raja_cuda/lib -lRAJA -Linstall/camp_cuda/lib -lcamp -ldl
install/magma_cuda/lib/libmagma.a(dgeqp3_gpu.cpp.o): In function `magma_dgeqp3_gpu':
dgeqp3_gpu.cpp:(.text+0x1e78): undefined reference to `dswap'
install/magma_cuda/lib/libmagma.a(dorgqr.cpp.o): In function `magma_dorgqr':
dorgqr.cpp:(.text+0x21bc): undefined reference to `dlacpy'
dorgqr.cpp:(.text+0x2208): undefined reference to `dlaset'
dorgqr.cpp:(.text+0x2250): undefined reference to `dlarft'
dorgqr.cpp:(.text+0x22e8): undefined reference to `dlarfb'
dorgqr.cpp:(.text+0x2478): undefined reference to `dlaset'
install/magma_cuda/lib/libmagma.a(dlaqps2_gpu.cu.o): In function `magma_dlaqps2_gpu':
/var/tmp/guthrey1/tmpxft_0001cd26_00000000-6_dlaqps2_gpu.cudafe1.cpp:(.text+0x4cc): undefined reference to `dlamch'
install/magma_cuda/lib/libmagma.a(interface.cpp.o): In function `magma_print_environment':
interface.cpp:(.text+0x204c): undefined reference to `GOMP_parallel'
install/magma_cuda/lib/libmagma.a(interface.cpp.o): In function `magma_print_environment._omp_fn.0':
interface.cpp:(.text+0x3794): undefined reference to `omp_get_num_threads'
collect2: error: ld returned 1 exit status
make: *** [Makefile:243: test_tensor_train_matrix_cuda.out] Error 1

[Stanimire Tomov]

That’s very good MAGMA compiled!

I see the new error relates to undefined LAPACK functions and is detected when linking your program.

Typically this is because of the naming - if LAPACK comes from Fortran the name is composed with

underscore, e.g., dlacpy_ instead of dlacpy.

I see in your Magma cmake options you have option

-DFORTRAN_CONVENTION:STRING=-DNOCHANGE

I am wondering if trying

-DFORTRAN_CONVENTION:STRING=-DADD_

will fix the problem.

Actually, you can also check what are the names in the lapack implementation, e.g.,

nm libopenblas.so | grep dlacpy 

where libopenblas.so should be replaced with the path and name to your lapack_static library.

Stan

To view this discussion on the web visit https://groups.google.com/a/icl.utk.edu/d/msgid/magma-user/e622d84d-03de-48f4-bdc7-0a3c89ca14dbn%40icl.utk.edu.