Hi,
I am getting a strange error with numba/cuda on a GeForce GTX 1080 Ti GPU (with driver version 381.22):
Most of the time, my code compiles and runs fine. But sometimes (maybe 1 out of 6 ; apparently completely randomly) it does not and I get the following error:
Traceback (most recent call last):
File "/global/scratch/rlehe/miniconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 127
8, in add_ptx
ptxbuf, len(ptx), namebuf, 0, None, None)
File "/global/scratch/rlehe/miniconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 259
, in safe_cuda_api_call
self._check_error(fname, retcode)
File "/global/scratch/rlehe/miniconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 296
, in _check_error
raise CudaAPIError(retcode, msg)
numba.cuda.cudadrv.driver.CudaAPIError: [999] Call to cuLinkAddData results in CUDA_ERROR_UNKNOWN
(The error is then caught by some of the numba code and finally returns a numba.cuda.cudadrv.driver.LinkerError.)
Note that I also have access to K20 and K80 GPUs, and that the above error never occurred on these types of GPU. Any clue as to what is happening here?
Thanks for your help!
PS: For what it's worth, the above error occurs both with cudatoolkit 8.0-0 and cudatoolkit 7.5-1 (from the numba channel).