MAGMA A100 CUDA12 issue

[Michael Wall]

Hello,

I have an issue I'm seeing using MAGMA 2.7.2 on an NVIDIA A100 machine with CUDA 12.

When I try to run an example, I'm seeing the below error. Consultants are working with vendors but reportedly having difficulty pinning down the problem. I just thought I'd post here to see if anyone else has come across this before. Any suggestions from MAGMA developers/users?

Thank you,

Michael Wall

mewall@nid001388:/usr/projects/icapt/mewall/packages/gpmd/magma-2.7.2/example> ./example_f
using MAGMA CPU interface
CUDA runtime error: CUDA driver version is insufficient for CUDA runtime version (35) in magma_getdevice_arch at /usr/projects/icapt/mewall/packages/gpmd/magma-2.7.2/interface_cuda/interface.cpp:568
Error in magma_getdevice_arch: MAGMA not initialized (call magma_init() first) or bad device
CUDA runtime error: CUDA driver version is insufficient for CUDA runtime version (35) in magma_getdevice at /usr/projects/icapt/mewall/packages/gpmd/magma-2.7.2/interface_cuda/interface.cpp:631
CUDA runtime error: CUDA driver version is insufficient for CUDA runtime version (35) in magma_setdevice at /usr/projects/icapt/mewall/packages/gpmd/magma-2.7.2/interface_cuda/interface.cpp:650
CUDA runtime error: CUDA driver version is insufficient for CUDA runtime version (35) in magma_zgetrf at /usr/projects/icapt/mewall/packages/gpmd/magma-2.7.2/src/zgetrf.cpp:148
CUBLAS error: not initialized (1) in magma_zgetrf at /usr/projects/icapt/mewall/packages/gpmd/magma-2.7.2/src/zgetrf.cpp:148
CUBLAS error: not initialized (1) in magma_zgetrf at /usr/projects/icapt/mewall/packages/gpmd/magma-2.7.2/src/zgetrf.cpp:148
 ** On entry to cusparseCreate(): CUDA context cannot be initialized

MAGMA error: function-specific error, see documentation (1) in magma_zgetrf at /usr/projects/icapt/mewall/packages/gpmd/magma-2.7.2/src/zgetrf.cpp:148
 ** On entry to cusparseSetStream() parameter number 1 (handle) had an illegal value: NULL pointer

MAGMA error: function-specific error, see documentation (3) in magma_zgetrf at /usr/projects/icapt/mewall/packages/gpmd/magma-2.7.2/src/zgetrf.cpp:148

[AndrewC]

I suggest to upgrade your nVIDIA device drivers. In my experience this error can happen if you build MAGMA against a CUDA version that is newer that the actual drivers installed on your system.

I had the same issue on Windows which required me to install newer nVIDIA drivers.

Andrew

[Michael Wall]

Thank you, I've passed this on to the consultants.