RMSD Collective variable using PLUMED2: Performance

[sap...@udel.edu]

Dear users,

we are trying to run plumed2/gromacs4.6.1 with rmsd umbrella restraint for a cg protein model (we want pmf versus rmsd).  Surprisingly the performance with this collective variable dramatically deteriorated (about 2ns / day on a system of 93000 atoms). Without the rmsd restraint, this coarse-grained system gives about 100ns/day. 

I am using the standard lapack/blas libraries (lapack3.4.2). Is there an issue with RMSD in plumed2 that i am not understanding that is leading to the deterioration of performance.

Thanks in advance for any advice / suggestions/ information..


best,
Sandeep

[davide branduardi]

Hi Sandeep,

The performance of MSD calculation strongly depends on how many atoms
you are constraining.
Try reducing them to a minimal number and the performance could improve.

Davide

[Giovanni Bussi]

Hi,

as Davide suggested the performance scales linearly with the number of atoms. Moreover, computing the average displacement of 93000 likely do not make a lot of sense. Either their movement is highly correlated (and many less atoms would suffice) or taking the mean displacement is meaningless.

Giovanni

--
You received this message because you are subscribed to the Google Groups "PLUMED users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to plumed-users...@googlegroups.com.
To post to this group, send email to plumed...@googlegroups.com.
Visit this group at http://groups.google.com/group/plumed-users?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.

[sap...@udel.edu]

Thanks Davide and Giovanni,

I miscommunicated the system specifics. We have a total of 93K atoms (cg water, cg lipid bilayer, and cg peptide), and only 26 atoms (cg beads) of a peptide helix segment are used for the restraint. We are not using pme, this is compiled with mpi support and we are using openmpi, and on compiliation I did not observe any warnings/messages indicating issues with performance. This is strange, but we will try to keep searching, though I agree that with only 26 atoms in the restraint, there should not be a big problem.

Thanks for any other suggestions there might be!

Sandeep

[Giovanni Bussi]

OK so the slow down is a nonsense.

A list of things to check first:

1. Are you using homogeneous weights in the pdb? If weights are not all equal to one, a slower implementation of rmsd is used.

2. At the end of the md.log file you should find a report from plumed concerning timings. Can you paste it here? Please start from the line where gromacs dumps the elapsed wallclock time.

Giovanni

[sap...@udel.edu]

Thanks Giovanni,

To respond to your questions:

1. So, initially, we had not been using 1 for the masses of the beads involved. We changed to a mass of 1 (set in the pdb file) for the affected beads, and we observed an improvement in performance from 1.9 ns/day (mass = 72) to 14 ns/day (mass = 1). by the way, the mass 72 is for martini cg bead.

2. Attached is the end of the md.log files for runs with mass=72 and mass=1

2A. mass = 72 (1.9 ns/day)
     R E A L   C Y C L E   A N D   T I M E   A C C O U N T I N G

 Computing:                 Nodes   Th.     Count  Wall t (s)     G-Cycles       %
-----------------------------------------------------------------------------
 Domain decomp.         8           1        505     174.834     4092.459     3.7
 DD comm. load           8          1         52       5.489      128.486     0.1
 Vsite constr.                8          1       5051       0.031        0.733     0.0
 Neighbor search         8          1        506      10.404      243.532     0.2
 Comm. coord.            8          1       5051    1441.899    33751.571    30.2
 Force                          8         1       5051      52.273     1223.600     1.1
 Wait + Comm. F         8         1       5051       6.351      148.654     0.1
 Vsite spread              8          1       5051       0.018        0.420     0.0
 Write traj.                  8          1          8       2.075       48.572     0.0
 Update                      8         1       5051       1.329       31.100     0.0
 Constraints                8        1       5051      71.778     1680.151     1.5
 Comm. energies        8        1         506     163.628     3830.157     3.4
 Rest                          8                    2844.407    66581.083    59.6
-----------------------------------------------------------------------------
 Total                  8                    4774.516   111760.517   100.0
-----------------------------------------------------------------------------

               Core t (s)   Wall t (s)        (%)
       Time:    38189.760     4774.516      799.9
                         1h19:34
                 (ns/day)    (hour/ns)
Performance:        1.828       13.129
Finished mdrun on node 0 Wed Jun  5 12:59:52 2013
PLUMED:                                                                Cycles        Total             Average           Minumum      Maximum
PLUMED:                                                                   1        2250.932467     2250.932467  2250.932467  2250.932467
PLUMED: 1 Prepare dependencies                          5051     0.024430        0.000005         0.000002     0.000034
PLUMED: 2 Sharing data                                         5051    91.313519        0.018078        0.012201     0.023680
PLUMED: 3 Waiting for data                                    5051    43.261075         0.008565         0.005207     0.471730
PLUMED: 4 Calculating (forward loop)                    5051  2092.131796       0.414202       0.263449     0.510323
PLUMED: 5 Applying (backward loop)                      5051    20.836353        0.004125        0.001912     0.640848



2B. mass=1 (14 ns/day)
     R E A L   C Y C L E   A N D   T I M E   A C C O U N T I N G

 Computing:                   Nodes   Th.     Count  Wall t (s)     G-Cycles       %
-----------------------------------------------------------------------------
 Domain decomp.                  8    1        507      16.152      378.092     2.6
 DD comm. load                    8    1         52       0.454       10.627     0.1
 Vsite constr.                        8    1       5071       0.030        0.705     0.0
 Neighbor search                 8    1        508      10.457      244.766     1.7
 Comm. coord.                    8    1       5071     147.589     3454.712    23.6
 Force                                  8    1       5071      52.685     1233.243     8.4
 Wait + Comm. F                 8    1       5071       5.110      119.615     0.8
 Vsite spread                      8    1       5071       0.019        0.441     0.0
 Write traj.                          8    1          3       0.089        2.088     0.0
 Update                             8    1       5071       1.363       31.909     0.2
 Constraints                      8    1       5071      12.885      301.612     2.1
 Comm. energies              8    1        508      11.581      271.089     1.9
 Rest                                   8                     367.016     8591.004    58.7
-----------------------------------------------------------------------------
 Total                  8                     625.431    14639.904   100.0
-----------------------------------------------------------------------------

               Core t (s)   Wall t (s)        (%)
       Time:     5002.640      625.431      799.9
                            (ns/day)    (hour/ns)
Performance:       14.011        1.713
Finished mdrun on node 0 Wed Jun  5 11:38:54 2013
PLUMED:                                                                Cycles        Total               Average           Minumum      Maximum
PLUMED:                                                                 1           295.566503      295.566503        295.566503   295.566503
PLUMED: 1 Prepare dependencies                          5071     0.024927        0.000005              0.000003     0.000013
PLUMED: 2 Sharing data                                         5071    103.670268        0.020444          0.017778     0.023465
PLUMED: 3 Waiting for data                                   5071      54.195365        0.010687         0.007146     0.024436
PLUMED: 4 Calculating (forward loop)                    5071      116.477575     0.022969          0.016686     0.031247
PLUMED: 5 Applying (backward loop)                      5071    17.925737        0.003535           0.002704     0.038111


Thanks
Sandeep

[sap...@udel.edu]

Dear users,

we have isolated our problem. we were unclear about what information is supposed to be in the ref.pdb file. we had initially all atoms in our system in the ref.pdb with only the atoms involved with the colvar having nonzero masses. we changed to having only the atoms of interest, so all is good now.

apologies for the false alarm..

Sandeep




On Tuesday, June 4, 2013 7:04:37 PM UTC-4, sap...@udel.edu wrote:

[Giovanni Bussi]

Hi,

a few clarifications (might be useful in the future).

1. Try to use the minimal amount of atoms in collective variables. Many atoms add overheads to obtain them from the MD engine (gromacs). As you correctly did, you should remove from the pdb of the reference structure atoms with weight=0

2. There are two routines for RMSD in plumed2, one faster and one slower (but more general). Currently the fast version is only used when all weights=1. The slow version is used in all other cases. When weights for alignment and weights for displacement are equal (but different from 1) you can enforce the fast version with TYPE=OPTIMAL-FAST. This will become the default very soon (before stable release) as we are fixing some detail. So, if it's fine for you to use align weights = displace weights but want to weight atoms with their mass, you can try that (hidden) option (TYPE=OPTIMAL-FAST). Also notice that in this case the result with fast and slow version *will be slightly different* (indeed they use a different definition for weights). The fast one should be consistent with the way gromacs does weighted RMSD (but if this is a strict condition for you I suggest you to compare with the results of g_rms).

3. Comment 2 will become obsolete as soon as possible (we are working on that). In the next beta snapshot, RMSD will give consistent results with both routines, and plumed will automatically choose the correct version (fast for align=displace weights, slow in the other case).

Ciao

Giovanni

On Wed, Jun 5, 2013 at 11:32 PM, <sap...@udel.edu> wrote:

Dear users,

we have isolated our problem. we were unclear about what information is supposed to be in the ref.pdb file. we had initially all atoms in our system in the ref.pdb with only the atoms involved with the colvar having nonzero masses. we changed to having only the atoms of interest, so all is good now.

apologies for the false alarm..

Sandeep




On Tuesday, June 4, 2013 7:04:37 PM UTC-4, sap...@udel.edu wrote:

Dear users,

we are trying to run plumed2/gromacs4.6.1 with rmsd umbrella restraint for a cg protein model (we want pmf versus rmsd).  Surprisingly the performance with this collective variable dramatically deteriorated (about 2ns / day on a system of 93000 atoms). Without the rmsd restraint, this coarse-grained system gives about 100ns/day. 

I am using the standard lapack/blas libraries (lapack3.4.2). Is there an issue with RMSD in plumed2 that i am not understanding that is leading to the deterioration of performance.

Thanks in advance for any advice / suggestions/ information..


best,
Sandeep

[davide branduardi]

Hi,
just to add on that,
the only thing that is important is the relative change in the
occupancy and beta columns of
pdb you provide.
This means that if all the weights are 72 this is the same of having
all the weights to 1.
Nevertheless plumed takes the safe way if it sees weights different
from 1 which is also more computationally
expensive. You should have no difference in the quantity you are calculating.
CiaoCiao
D

[sap...@udel.edu]

Hi Davide and Giovanni,
thanks for the follow up. That is useful to know. we'll check the source code as well, and look forward to the upcoming releases of PLUMED. It is a very useful
piece of work!

-S

On Tuesday, June 4, 2013 7:04:37 PM UTC-4, sap...@udel.edu wrote: