Calling magma from python multiprocessing

25 views
Skip to first unread message

Analabha Roy

unread,
Sep 19, 2020, 1:43:14 PM9/19/20
to MAGMA User
As I understand it, magma can be called in a multiprocessing environment as long as initialization and finalization are done after all processes have finished.

However, trying to call magma_zgeev for multiple processes (with locking) generates exceptions


Code is at pastebin: https://pastebin.com/phxii7mr

The relevant bits are:
  1. lock = Lock()
  2.     for h in ampls:
  3.         #Generate Floquet Matrix here. Random for now
  4.         floquet_matrix = np.random.random((size,size)) + (1j) * np.random.random((size,size))
  5.         p = Process(target=get_eigsys, args=(lock, floquet_matrix, tasks_to_accomplish, tasks_that_are_done))
  6.         processes.append(p)
  7.         p.start()
Now, the function "get_eigsys" creates workspaces and calls zgeev, but yields the following exception


Error in magma_getdevice_arch: MAGMA not initialized (call magma_init() first) or bad device
Process Process-2:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/home/daneel/PycharmProjects/floquet_theory/run_parallel_tasks.py", line 123, in get_eigsys
    w, v = gpu_eig(lock, mat, left=False, right=True)
  File "/home/daneel/PycharmProjects/floquet_theory/run_parallel_tasks.py", line 99, in gpu_eig
    work.ctypes.data, lwork, rwork.ctypes.data)
  File "/home/daneel/.local/lib/python3.6/site-packages/skcuda/magma.py", line 2699, in magma_zgeev
    magmaCheckStatus(status)
  File "/home/daneel/.local/lib/python3.6/site-packages/skcuda/magma.py", line 64, in magmaCheckStatus
    raise MagmaError(status)
skcuda.magma.MagmaError: b'cannot allocate memory on GPU device' (Code: -113)

Process finished with exit code 0

Mark Gates

unread,
Sep 19, 2020, 3:13:34 PM9/19/20
to Analabha Roy, MAGMA User
On Sat, Sep 19, 2020 at 1:43 PM Analabha Roy <harise...@gmail.com> wrote:
As I understand it, magma can be called in a multiprocessing environment as long as initialization and finalization are done after all processes have finished.

magma_init() and magma_finalize() are thread safe, so there are choices:

1) Call magma_init() before launching threads, launch threads to call magma functions in parallel, wait for all threads to exit, and call magma_finalize().
2) Every thread calls magma_init(), other magma functions, magma_finalize(), and the thread exits.


However, trying to call magma_zgeev for multiple processes (with locking) generates exceptions

Code is at pastebin: https://pastebin.com/phxii7mr

The relevant bits are:
  1. lock = Lock()
  2.     for h in ampls:
  3.         #Generate Floquet Matrix here. Random for now
  4.         floquet_matrix = np.random.random((size,size)) + (1j) * np.random.random((size,size))
  5.         p = Process(target=get_eigsys, args=(lock, floquet_matrix, tasks_to_accomplish, tasks_that_are_done))
  6.         processes.append(p)
  7.         p.start()
Now, the function "get_eigsys" creates workspaces and calls zgeev, but yields the following exception

Where are magma_init() and magma_finalize() called?

Analabha Roy

unread,
Sep 20, 2020, 1:08:40 AM9/20/20
to Mark Gates, MAGMA User
Hi,





On Sun, Sep 20, 2020, 00:43 Mark Gates <mga...@icl.utk.edu> wrote:
On Sat, Sep 19, 2020 at 1:43 PM Analabha Roy <harise...@gmail.com> wrote:
As I understand it, magma can be called in a multiprocessing environment as long as initialization and finalization are done after all processes have finished.

magma_init() and magma_finalize() are thread safe, so there are choices:

1) Call magma_init() before launching threads, launch threads to call magma functions in parallel, wait for all threads to exit, and call magma_finalize().
2) Every thread calls magma_init(), other magma functions, magma_finalize(), and the thread exits.


However, trying to call magma_zgeev for multiple processes (with locking) generates exceptions

Code is at pastebin: https://pastebin.com/phxii7mr

The relevant bits are:
  1. lock = Lock()
  2.     for h in ampls:
  3.         #Generate Floquet Matrix here. Random for now
  4.         floquet_matrix = np.random.random((size,size)) + (1j) * np.random.random((size,size))
  5.         p = Process(target=get_eigsys, args=(lock, floquet_matrix, tasks_to_accomplish, tasks_that_are_done))
  6.         processes.append(p)
  7.         p.start()
Now, the function "get_eigsys" creates workspaces and calls zgeev, but yields the following exception

Where are magma_init() and magma_finalize() called?



magma_init is called just before that loop, and magma_finalize is called just after the loop and after all thr processes are terminated. 

Analabha Roy

unread,
Sep 20, 2020, 11:54:29 AM9/20/20
to Mark Gates, MAGMA User
Hi all,

Nevermind, I figured it out.

For the record, the multiprocessing module in python for Linux starts a process with the "fork" method, which CUDA does not support. Instead, just do


>> import multiprocessing as mp
>> mp.set_start_method('spawn')

and it works!

Regards,
AR
--
Analabha Roy
Assistant Professor
Golapbag Campus, Barddhaman 713104
West Bengal, India
Reply all
Reply to author
Forward
0 new messages