cpu/gpu, mpi

[Tamas Hegedus]

Hi,

I do not understand, what could be a good running solution for gmx patched with plumed on a single node with 16 CPUs and 3 GPUs.

There is plumed 2.2 on our HPC.
Does PLUMED_NUM_THREADS exist in this version? It is only mentioned in 2.3

How should I distribute the tasks on CPU/GPU?
1 task for the node and 15 CPUs for the task? With 1 or 3 GPUs?
3 MPI tasks for the node and 5 CPUs/task with 3 GPUs?

How does these relate to PLUMED? Should I save CPUs for PLUMED?
E.g.
3 MPI tasks for the node and 4 CPUs/task for mdrun and 3-4 CPUs for PLUMED?

Thanks for your suggestions,
Tamas

[Giovanni Bussi]

PLUMED_NUM_THREADS is only available starting with v2.3, so you should upgrade it.

Then you have to experiment with settings until you find the best choice. Keep in mind that plumed only uses the CPU, and that the gromacs load balancing algorithm will typically shift more non-bonded interactions to the GPU while using plumed

Giovanni

--
You received this message because you are subscribed to the Google Groups "PLUMED users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to plumed-users+unsubscribe@googlegroups.com.
To post to this group, send email to plumed...@googlegroups.com.
Visit this group at https://groups.google.com/group/plumed-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/plumed-users/0366a560-c946-41fc-92bb-201af0524772%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tamas Hegedus]

Thanks.

A further question:
Is there any solution to run gmx+plumed on a single node and use several GPUs?

I try to compile gmx 5.1.4 +plumed 2.3 for single nodes, which have 16 CPU and 4 GPU. It seems for me that I can utilize only one GPU with gmx patched with plumed.
I think that I have to compile both gmx and plumed with MPI and run gmx with 1 MPI and 16 thread.
I think if I compile gmx without MPI and try to use more than one GPU, gmx will start thread-MPI processes that are not compatible with plumed compiled either with or without MPI.
Most likely I can start 1 thread-MPI process with 16 threads, but in this case I can also use only one GPU.

Thanks,
Tamas

On Wednesday, September 13, 2017 at 9:48:02 AM UTC+2, Giovanni Bussi wrote:

PLUMED_NUM_THREADS is only available starting with v2.3, so you should upgrade it.

Then you have to experiment with settings until you find the best choice. Keep in mind that plumed only uses the CPU, and that the gromacs load balancing algorithm will typically shift more non-bonded interactions to the GPU while using plumed

Giovanni

On Tue, Sep 12, 2017 at 7:42 PM, Tamas Hegedus <biohe...@gmail.com> wrote:

Hi,

I do not understand, what could be a good running solution for gmx patched with plumed on a single node with 16 CPUs and 3 GPUs.

There is plumed 2.2 on our HPC.
Does PLUMED_NUM_THREADS exist in this version? It is only mentioned in 2.3

How should I distribute the tasks on CPU/GPU?
1 task for the node and 15 CPUs for the task? With 1 or 3 GPUs?
3 MPI tasks for the node and 5 CPUs/task with 3 GPUs?

How does these relate to PLUMED? Should I save CPUs for PLUMED?
E.g.
3 MPI tasks for the node and 4 CPUs/task for mdrun and 3-4 CPUs for PLUMED?

Thanks for your suggestions,
Tamas

--
You received this message because you are subscribed to the Google Groups "PLUMED users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to plumed-users...@googlegroups.com.

[Giovanni Bussi]

You should disable thread-mpi in gromacs and use a real mpi library (same library for both gromacs and plumed)

Giovanni

To unsubscribe from this group and stop receiving emails from it, send an email to plumed-users+unsubscribe@googlegroups.com.

To post to this group, send email to plumed...@googlegroups.com.
Visit this group at https://groups.google.com/group/plumed-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/plumed-users/cc1b4fa8-86ea-443f-9284-decc5bcd4ced%40googlegroups.com.

[Tamas Hegedus]

Hi,

It seems for me that on a single node with GPUs the performance is best with threaded gromacs and plumed without mpi.

- if you use a real MPI then you have to set the mpi rank to 1 and set the gpu_id to one gpu; in this case my given simulations was estimated to be finished in 14 days
- when I disabled mpi in plumed and compiled gromacs without a real mpi, then I set the -ntmpi 1 -ntomp 16; in this case the same job is estimated to be completed in 10 days
- in both case you can only use one GPU

Tamas

[Carlo Camilloni]

Hi

You can use -ntomp also with mpi. Indeed this is the number of openmp thread that is unrelated with gromacs multi-threading that is an alternative to mpi for single workstations (not supported by plumed). Furthermore if you compile with mpi you can use multiple gpus by using multiple mpi rank, multiple gpus and multiple openmp threads (mpiexec -np 2 gmx_mpi mdrun -ntomp 8 -gpu_id 01) 

Carlo

To view this discussion on the web visit https://groups.google.com/d/msgid/plumed-users/7cbaa1f6-efbe-4b89-a060-4d8499d87385%40googlegroups.com.

[Tamas Hegedus]

Dear Carlo,

Yes, I thought that I could use multiple GPUs this way. I have tried.
Now I realized that the problem could be using Radeon GPUs thus I had to compile gromacs with OpenCL
My error: "The OpenCL implementation only supports using exactly one PP rank per node"

Can you confirm that this is OpenCL and Radeon issue?
Could I use multiple MPI ranks and GPUs on a single node, if I have Nvidia GPUs with CUDA?
Or do I miss something else in my setup?

Thanks,
Tamas

[Carlo Camilloni]

Hi

Yes this is an OpenCL issue, and yes it would  work with nvidia/cuda ones

Carlo

To view this discussion on the web visit https://groups.google.com/d/msgid/plumed-users/26818ce1-29cb-42ce-99ac-8b17618e10a4%40googlegroups.com.