Running i-pi with in multiple nodes at a supercomputer

[Marc Riera]

Hello all!

As you may know, we developed a C++ library (MBX, http://paesanigroup.ucsd.edu/software/mbx.html) that allows to use MB-pol (J. Chem. Theory Comput.2013,9, 12, 5395-5403, J. Chem. Theory Comput.2014,10,4,1599-1607) to get energies and gradients of a water system. In order to allow MD simulations, we have developed an interface with i-pi that uses i-pi as the MD engine. The library itself is parallelized with OpenMP, and it scales decently in a single node. However, we would like to be able to run, a part from classical MD, also REMD, PIMD, and REPIMD simulations with i-pi, and the only way to efficiently run this would be to use multiple instances of the driver in multiple nodes of a supercomputer.

Right now, the driver that interfaces i-pi with MBX assumes single node simulations (inet is set to 0). I would appreciate a little bit of guidance on how to set this up for supercomputers in multiple nodes (fyi, we can run on a supercomputer in a single node; we have already tested and done it!). Please correct me if I am wrong in any step, and if what I am saying below is correct, I might recontact you if there is any problem or strange behavior. Let's say I want to run an REMD simulation for a system with 2 replicas, using 1 driver per node (2 nodes + i-pi total)

1. First, I should submit an instance of i-pi to the supercomputer (1 core should be OK?) The xml file will contain 2 instances for the driver:

   <ffsocket name="driver1" mode="unix">

   <address>inethost1</address>

   </ffsocket>

   <ffsocket name="driver2" mode="unix">

   <address>inethost2</address>

   </ffsocket>

and the corresponding drivers:

      <forces>

         <force forcefield ="driver1" > </force>

         <force forcefield ="driver2" />

      </forces>

Do we 2 driver instances or just one?

2. Once i-pi is running, I need to submit the multiple instances of the driver. Here I have a couple questions:

a) I do need to use inet = 1 (to use the internet ports), right?

b) Each driver need the information of the address, port and inet information. Does each driver need to have a different port number, or just a different host?

c) How will the driver know where to connect to contact i-pi, if it is not in the same physical machine?

After this, is a matter of setting the input properly.

I would appreciate if you guys can let me know if this is correct, or if I have any missconception or missunderstanding about how to set this up.

thanks a lot!

Marc

[Michele Ceriotti]

Hi, this is discussed quite in detail in the manual. 

Submitting calculations on HPC is a bit of a pain. You typically need a bit of bash magic in the submit script. 

Fitst, you need to determine the node on which i-PI will run, and pick the IP of the interface of that node. This is made harder by the fact you often have multiple network cards, but usually it's not too bad to understand which is the right one. 

Then, you need to make sure that within i-PI the socket will be open on an inet port, using that IP address. So you typically sed -i 's/address> .* </address> $LOCALIP </' input.xml, or similar.

Finally, you need to launch the driver code (LAMMPS, MB-POL, whatever) specifying that it should connect to said IP and on the appropriate port.

Never found a case in which I couldn't get it to work, but it's far from ideal, which is why we're working on a MPI communicator mechanism.