Running CuPy scripts from python concurrently on different GPUs with different parameters

[Andras Balogh]

I can run a CuPy script with different parameters on different GPUs concurrently very easily from the terminal of from a bash script as:

export CUDA_VISIBLE_DEVICES="0"; python3 mycode.py arg1 &

export CUDA_VISIBLE_DEVICES="0"; python3 mycode.py arg2 &

I can also do the same using subprocesses from a python scipt:

import subprocess 

subprocess.run('export CUDA_VISIBLE_DEVICES="0"; python3 mycode.py arg1 &
export CUDA_VISIBLE_DEVICES="0"; python3 mycode.py arg2 &', shell=True)

I tried to use function version with selecting device, but it does not run concurrently  

with cupy.cuda.Device(0):

     mycode(arg1)

with cupy.cuda.Device(1):

     mycode(arg2)

Is there a more elegant solution?

Andras

[Kenichi Maehashi]

Hi Andras,

As with the general Python program you can use parallelize the code using multiprocessing or threading.

https://docs.python.org/3/library/multiprocessing.html

https://docs.python.org/3/library/threading.html

Limitations specific to CUDA/CuPy are:

- When using multiprocessing, due to the limitation of CUDA, processes cannot be forked after CUDA initialization. You will need to use multiprocessing.set_start_method('spawn') (or forkserver), or avoid initializing CUDA (i.e., do not use CuPy API except import cupy) until you fork child processes.

- When using threading, you may need to create a stream for each thread instead of the default stream to get parallelism.

Thanks,

Kenichi

2021年5月12日水曜日 11:46:55 UTC+9 Andras Balogh:

[Andras Balogh]

Hi Kenichi,

This looks like a good solution.

Thank you very much. 

Andras