random.shuffle gives RuntimeError: radix_sort: failed on 1st step: cudaErrorInvalidDeviceFunction

Andras Balogh

unread,

Oct 2, 2020, 2:06:02 AM10/2/20

to CuPy User Group

cp.random.shuffle(a) results in error.

Other parts of cupy seem to work. I get this both under ubuntu 18.04 and CentOS 7.8

cuda -11.0 and cupy 8.0.0

This is the first time I tried shuffle.

File "/usr/local/lib/python3.6/dist-packages/cupy/random/_permutations.py", line 15, in shuffle

return rs.shuffle(a)

File "/usr/local/lib/python3.6/dist-packages/cupy/random/_generator.py", line 1090, in shuffle

a[:] = a[self._permutation(len(a))]

File "/usr/local/lib/python3.6/dist-packages/cupy/random/_generator.py", line 1125, in _permutation

array = cupy.argsort(sample)

File "/usr/local/lib/python3.6/dist-packages/cupy/_sorting/sort.py", line 106, in argsort

return a.argsort(axis=axis)

File "cupy/core/core.pyx", line 728, in cupy.core.core.ndarray.argsort

File "cupy/core/core.pyx", line 745, in cupy.core.core.ndarray.argsort

File "cupy/core/_routines_sorting.pyx", line 91, in cupy.core._routines_sorting._ndarray_argsort

File "cupy/cuda/thrust.pyx", line 117, in cupy.cuda.thrust.argsort

RuntimeError: radix_sort: failed on 1st step: cudaErrorInvalidDeviceFunction: invalid device function

------------

Thanks!

Kenichi Maehashi

unread,

Oct 2, 2020, 3:06:58 AM10/2/20

to CuPy User Group

What GPU are you using?

Do you install CuPy via wheel (cupy-cuda110) or built from source?

Thanks,

2020年10月2日金曜日 15:06:02 UTC+9 Andras Balogh:

Andras Balogh

unread,

Oct 2, 2020, 12:55:46 PM10/2/20

to CuPy User Group

The GPUs are Tesla K40c on one and Tesla K80 on the other.

CuPy was installed as cupy-cuda110.

Kenichi Maehashi

unread,

Oct 15, 2020, 1:59:34 AM10/15/20

to CuPy User Group

Thanks for the info! This issue was specific to CUDA 11.0+ with CC 3.5 hardware.

We will work on fixing the issue here: https://github.com/cupy/cupy/pull/4137

2020年10月3日土曜日 1:55:46 UTC+9 Andras Balogh:

Message has been deleted

Andras Balogh

unread,

Aug 14, 2021, 2:14:27 PM8/14/21

to CuPy User Group

Right now it works for me under cupy 8.3.0, cuda 11.2, Ubuntu 20.04.2.

On Friday, August 13, 2021 at 1:23:29 AM UTC-5 neelu patel wrote:

Good Morning,
i am also facing the same issue and i have cuda 11 into my system. Did you found any solution for this issue.

Thanks and Regards
Neelu Patel

Reply all

Reply to author

Forward