I know that dense hybrid magma solvers like magma_cgetrf calls for the multiple GPU solver magma_cgetrf_m with ngpus when ngpu > 1. However, I observe the following problems when I start jobs that cannot fit in a GPU (say N > 50k) on a server with two GeForce GTX 1080s:
1. Only the first GPU with id 0 is used. i.e., I do not see anything on 2nd gpu when I query with nvidia-smi. I am sure that 2nd GPU is not utilized since no speedup is observed compared to 1 GPU.
2. When 2 or more GPU jobs (processes) started, sometimes GPU solver fails with segfault.
Does anyone have similar issues? Any suggestions? I am using magma-2.5.1.The server has plenty of RAM available.