LinkerError with CUDA due to "Unsupported .version 4.3"

[remi...@gmail.com]

I recently tried to run numba with cuda on one of the GPUs of the Titan cluster (at Oakridge Computing Facility).

However, I got the following error:

LinkerError: [209] Call to cuLinkAddData results in CUDA_ERROR_NO_BINARY_FOR_GPU
ptxas application ptx input, line 9; fatal   : Unsupported .version 4.3; current version is '4.2'
ptxas fatal   : Ptx assembly aborted due to errors

Is this error known, and is there an easy fix to it ?

By the way, here is the complete stack:

build/bdist.linux-x86_64/egg/fbpic/particles/cuda_methods.py in <module>()
     16             float64[:], float64[:], float64[:], \
     17             float64[:], float64[:], float64[:], \
---> 18             float64, float64, int32, float64)')
     19 def push_p_gpu( ux, uy, uz, inv_gamma,
     20                 Ex, Ey, Ez, Bx, By, Bz,

/lustre/atlas/scratch/rlehe/aph103/anaconda2/lib/python2.7/site-packages/numba/cuda/decorators.pyc in kernel_jit(func)
     90             # Force compilation for the current context
     91             if bind:
---> 92                 kernel.bind()
     93
     94             return kernel

/lustre/atlas/scratch/rlehe/aph103/anaconda2/lib/python2.7/site-packages/numba/cuda/compiler.pyc in bind(self)
    501         Force binding to current CUDA context
    502         """
--> 503         self._func.get()
    504
    505     @property

/lustre/atlas/scratch/rlehe/aph103/anaconda2/lib/python2.7/site-packages/numba/cuda/compiler.pyc in get(self)
    386             # Link
    387             linker = driver.Linker()
--> 388             linker.add_ptx(ptx)
    389             for path in self.linking:
    390                 linker.add_file_guess_ext(path)

/lustre/atlas/scratch/rlehe/aph103/anaconda2/lib/python2.7/site-packages/numba/cuda/cudadrv/driver.pyc in add_ptx(self, ptx, name)
   1234                                  ptxbuf, len(ptx), namebuf, 0, None, None)
   1235         except CudaAPIError as e:
-> 1236             raise LinkerError("%s\n%s" % (e, self.error_log))
   1237
   1238     def add_file(self, path, kind):

[Stanley Seibert]

(Sorry this got stuck in the groups spam filter.  Just now found it.)

What version of CUDA is installed on the cluster?  We recently updated the cudatoolkit conda package from CUDA 7.0 to CUDA 7.5.

--
You received this message because you are subscribed to the Google Groups "Numba Public Discussion - Public" group.
To unsubscribe from this group and stop receiving emails from it, send an email to numba-users...@continuum.io.
To post to this group, send email to numba...@continuum.io.
To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/numba-users/bac9c580-464c-41a5-b990-65e5b539749d%40continuum.io.
For more options, visit https://groups.google.com/a/continuum.io/d/optout.

[rl...@lbl.gov]

Thanks for your answer (and sorry for my own delay)
The version of cuda on the cluster in 7.0 ("nvcc --version" returns "Cuda compilation tools, release 7.0, V7.0.27"). I am using numba 0.25.0 and accelerate 2.2.0

The error I am reporting is quite surprising, as I used to be able to run numba and accelerate on Titan without error a few months ago.
I will simultaneous ask the support team of Titan whether they updated something related to CUDA over the past few months.

[Stanley Seibert]

The issue would be caused by the cudatoolkit conda package being updated.  You can downgrade to the previous version with:

conda install cudatoolkit=7.0

Both version 7.0 and 7.5 should continue to work for the near future, but it would be nice if your GPU cluster could update to CUDA 7.5.

To view this discussion on the web visit https://groups.google.com/a/continuum.io/d/msgid/numba-users/b82652ce-0c15-452b-b5e8-6b112a794579%40continuum.io.

[rl...@lbl.gov]

Thanks for the advice! Using `cuda install cudatoolkit=7.0` solved the problem!