special tpr for RDF calculations

[Andrey Brukhno]

Hello,

I encountered a very strange (looks like legacy) issue with v1.4.1.

In my earlier IBI iterations (with v1.3) I normally used special topol-rdf.tpr which was different from normal CG simulation topol.tpr by not including the intra-molecular pairs in the RDF stats (done by csg_stat). I wanted to make sure my previous results are reproduced by v1.4.1, so copied all the initialisation files into a new directory and tried to run once again. After the very first iteration I checked the obtained RDFs and they were all wrong - corresponding to normal topol.tpr (including intra-pairs) and not topol-rdf.tpr (excluding intra-pairs)!

I guess there might have been changes made to locations or names of tags in settings.xml between the VOTCA versions, but I could not identify those changes by referring to the most recent manual (both v1.4.1 and development versions).

Below is the relevant extract from my settings.xml file.

Any clue, hint, advice are most welcome!

Thanks in advance.

Andrey

-----------

    <gromacs>

      <grompp>

        <bin>gmx grompp</bin>

        <!--opts></opts-->

      </grompp>

      <mdrun>

        <!--command>grmx-vtc-rerun.sh</command-->

        <!--command>mpirun -lsf mdrun_mpi</command-->

        <!--opts>-multidir sim_0 sim_1 sim_2 sim_3</opts-->

        <command>mpirun -n 4 mdrun_mpi</command>

        <opts>-multidir sim_0 sim_1 sim_2 sim_3 -tablep table.xvg -tableb table_b?.xvg table_a?.xvg table_d?.xvg</opts>

      </mdrun>

      <!--mdrun><opts>-nt 2</opts></mdrun-->

      <!-- trash so many frames at the beginning -->

      <equi_time>0</equi_time>

      <!-- grid for table*.xvg !-->

      <table_bins>0.001</table_bins>

      <!-- cut the potential at this value (gromacs bug) -->

      <pot_max>10000</pot_max>

      <!-- extend the tables to this value -->

      <table_end>2.5</table_end>

      <initial_configuration>maindir</initial_configuration>

      <rdf>

        <block_length><DESC>Length of the block for the error analysis</DESC>

          1000000

        </block_length>

        <map><DESC>Special mapping file for rdf calculations needed for bonded interactions</DESC>

          dopc_cg-map11.xml

        </map>

        <topol><DESC> Gromacs binary topol (tpr) file to be used for csg_stat</DESC>

          topol-rdf.tpr

        </topol>

        <with_errors>yes<DESC>calculate error on the rdf: yes/no</DESC>

        </with_errors>

      </rdf>

    </gromacs>

-----------

[Christoph Junghans]

On Sat, Jul 21, 2018 at 9:01 PM, 'Andrey Brukhno' via votca
<vo...@googlegroups.com> wrote:
> Hello,
>
> I encountered a very strange (looks like legacy) issue with v1.4.1.
> In my earlier IBI iterations (with v1.3) I normally used special
> topol-rdf.tpr which was different from normal CG simulation topol.tpr by not
> including the intra-molecular pairs in the RDF stats (done by csg_stat). I
> wanted to make sure my previous results are reproduced by v1.4.1, so copied
> all the initialisation files into a new directory and tried to run once
> again. After the very first iteration I checked the obtained RDFs and they
> were all wrong - corresponding to normal topol.tpr (including intra-pairs)
> and not topol-rdf.tpr (excluding intra-pairs)!
>
> I guess there might have been changes made to locations or names of tags in
> settings.xml between the VOTCA versions, but I could not identify those
> changes by referring to the most recent manual (both v1.4.1 and development
> versions).
>
> Below is the relevant extract from my settings.xml file.
>
> Any clue, hint, advice are most welcome!

We haven't changes any tags in IBI since v1.4.1 and the special
topology for IBI can be set by cg.inverse.<sim_prog>.rdf.topol, which
you did below.
You could try "grep csg_stat inverse.log" to figure out if the right
topology got appended to the command line of csg_stat.

Christoph

> --
> You received this message because you are subscribed to the Google Groups
> "votca" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to votca+un...@googlegroups.com.
> To post to this group, send email to vo...@googlegroups.com.
> Visit this group at https://groups.google.com/group/votca.
> For more options, visit https://groups.google.com/d/optout.



--
Christoph Junghans
Web: http://www.compphys.de

[Andrey Brukhno]

Thanks for your reply Christoph.

I did check it, but inverse.log does not contain sufficient info, see the extract below.

I will be investigating it further to make sure I did not do anything stupid myself, although I haven't changed much for the test iteration except recompiling the tpr files for the newer version of Gromacs).

Andrey

-----

$ grep rdf inverse.log

...

cp_from_main_dir: 'grompp.mdp* topol.top* topol-rdf.tpr table_??.xvg index.ndx* conf.gro*'

cp_from_main_dir: 'grompp.mdp* topol.top* topol-rdf.tpr table_??.xvg index.ndx* conf.gro*'

cp_from_main_dir: 'grompp.mdp* topol.top* topol-rdf.tpr table_??.xvg index.ndx* conf.gro*'

cp_from_main_dir: 'grompp.mdp* topol.top* topol-rdf.tpr table_??.xvg index.ndx* conf.gro*'

cp_from_main_dir: 'grompp.mdp* topol.top* topol-rdf.tpr table_??.xvg index.ndx* conf.gro*'

‘sim_0/topol-rdf.tpr’ -> ‘./topol-rdf.tpr’

Calculating rdfs with csg_stat using 4 tasks

Calculating average rdfs and its errors for interaction TCH-TCH

rdf calculation is already done

Calculating average rdfs and its errors for interaction TCH-TCO

rdf calculation is already done

Calculating average rdfs and its errors for interaction TCO-TCO

rdf calculation is already done

Calculating average rdfs and its errors for interaction TCH-PO

rdf calculation is already done

Calculating average rdfs and its errors for interaction TCH-NH

rdf calculation is already done

Calculating average rdfs and its errors for interaction TCO-PO

rdf calculation is already done

Calculating average rdfs and its errors for interaction TCO-NH

rdf calculation is already done

Calculating average rdfs and its errors for interaction PO-PO

rdf calculation is already done

Calculating average rdfs and its errors for interaction PO-NH

rdf calculation is already done

Calculating average rdfs and its errors for interaction NH-NH

> email to votca+unsubscribe@googlegroups.com.

> To post to this group, send email to vo...@googlegroups.com.
> Visit this group at https://groups.google.com/group/votca.
> For more options, visit https://groups.google.com/d/optout.

--
Christoph Junghans
Web: http://www.compphys.de

--
You received this message because you are subscribed to a topic in the Google Groups "votca" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to votca+unsubscribe@googlegroups.com.

[Christoph Junghans]

On Sun, Jul 22, 2018 at 1:27 PM, 'Andrey Brukhno' via votca
<vo...@googlegroups.com> wrote:
> Thanks for your reply Christoph.
>
> I did check it, but inverse.log does not contain sufficient info, see the
> extract below.
> I will be investigating it further to make sure I did not do anything stupid
> myself, although I haven't changed much for the test iteration except
> recompiling the tpr files for the newer version of Gromacs).

The " rdf calculation is already done" tells me that this is not a run
from scratch.
Try a fresh run, because some chance in the xml don't get populated
until the next iteration step.
And grep for "csg_stat" (not rdf), you should see actually command,
which got run, e.g.:
"Running critical command 'csg_stat --nt 4 --options settings.xml
--top topol.tpr --trj traj.xtc --begin 20 --first-frame 0' "
From that you can see if all the options are read as excepted.

Christoph

>> > email to votca+un...@googlegroups.com.

>> > To post to this group, send email to vo...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/votca.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>> --
>> Christoph Junghans
>> Web: http://www.compphys.de
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "votca" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to

>> votca+un...@googlegroups.com.

>> To post to this group, send email to vo...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/votca.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --

> You received this message because you are subscribed to the Google Groups
> "votca" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to votca+un...@googlegroups.com.

[Andrey Brukhno]

On Mon, Jul 23, 2018 at 2:22 AM, Christoph Junghans <jung...@votca.org> wrote:

On Sun, Jul 22, 2018 at 1:27 PM, 'Andrey Brukhno' via votca
<vo...@googlegroups.com> wrote:
> Thanks for your reply Christoph.
>
> I did check it, but inverse.log does not contain sufficient info, see the
> extract below.
> I will be investigating it further to make sure I did not do anything stupid
> myself, although I haven't changed much for the test iteration except
> recompiling the tpr files for the newer version of Gromacs).
The " rdf calculation is already done" tells me that this is not a run
from scratch.
Try a fresh run, because some chance in the xml don't get populated
until the next iteration step.
 And grep for "csg_stat" (not rdf), you should see actually command,
which got run, e.g.:
"Running critical command 'csg_stat --nt 4 --options settings.xml
--top topol.tpr --trj traj.xtc --begin 20 --first-frame 0' "
From that you can see if all the options are read as excepted.

I knew you were going to say this...

As a matter of fact, there is no such message from calc_rdf_generic.sh, but instead:

  msg "Calculating rdfs with csg_stat using $tasks tasks"
  critical csg_stat --nt $tasks --options "$CSGXMLFILE" --top "$topol" --trj "$traj" --begin $equi_time --first-frame $first_frame ${error_opts} ${maps:+--cg ${maps}}
  mark_done "rdf_calculation${suffix}"

so I had to add this after the msg and before the csg_stat call:

  echo "Doing: critical csg_stat --nt $tasks --options "$CSGXMLFILE" --top "$topol" --trj "$traj" --begin $equi_time --first-frame $first_frame ${error_opts} ${maps:+--cg ${maps}}"

This shows me that topol-rdf.tpr is actually used (extract from inverse.log):

Make update for ibi

Calculating rdfs with csg_stat using 4 tasks

Doing: critical csg_stat --nt 4 --options /home/andrey/Work/Models/DOPC-long-wtip4p-new/ibi-Rc1.65nm-freeN-dihs_CG-fits1-151cT/dopc_cg-int-map11.xml --top topol-rdf.tpr --trj traj.xtc --begin 0 --first-frame 0 --block-length 1000000 --ext dist.block --cg /home/andrey/Work/Models/DOPC-long-wtip4p-new/ibi-Rc1.65nm-freeN-dihs_CG-fits1-151cT/dopc_cg-map11.xml;
begin to calculate distribution functions

However, the distributions are still wrong, and I am lost now as to which program to blame (again all the initial files are the same as in my previous iteration).

I compared the outputs of `gmx dump topol-rdf.tpr' and `gmx dump topol.tpr' generated with the new Gromacs and everything seems alright, the relevant differences in exclusion lists are there.

The problem is also that the original calculation was done on a cluster more than a year ago, so I cannot reproduce exactly all the installation setup, not on my desktop anyway. But I know that those calculations were correct and well converged.

So something should have changed somewhere within the newly installed Gromacs/VOTCA combination.

Andrey

>> > email to votca+unsubscribe@googlegroups.com.

>> > To post to this group, send email to vo...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/votca.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>> --
>> Christoph Junghans
>> Web: http://www.compphys.de
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "votca" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to

>> votca+unsubscribe@googlegroups.com.

>> To post to this group, send email to vo...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/votca.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "votca" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to votca+unsubscribe@googlegroups.com.

> To post to this group, send email to vo...@googlegroups.com.
> Visit this group at https://groups.google.com/group/votca.
> For more options, visit https://groups.google.com/d/optout.

--
Christoph Junghans
Web: http://www.compphys.de

--
You received this message because you are subscribed to a topic in the Google Groups "votca" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to votca+unsubscribe@googlegroups.com.

[Christoph Junghans]

On Mon, Jul 23, 2018 at 2:39 AM, 'Andrey Brukhno' via votca

Well the "critical" function includes the print statement:
critical() {
[[ $quiet = "no" ]] && echo "Running critical command '$*'" >&2
"$@" || die "${FUNCNAME[0]}: '$*' failed"
}
So I am not sure when that got lost in your case.

You can see what VOTCA has with:
csg_dump --top topol-rdf.tpr --cg dopc_cg-map11.xml

Remember, if you are using a mapping file ("--cg" option) exclusions
from the topology file are ignored and the ones from the mapping file
are used instead.

Christoph

>> >> > email to votca+un...@googlegroups.com.

>> >> > To post to this group, send email to vo...@googlegroups.com.
>> >> > Visit this group at https://groups.google.com/group/votca.
>> >> > For more options, visit https://groups.google.com/d/optout.
>> >>
>> >>
>> >>
>> >> --
>> >> Christoph Junghans
>> >> Web: http://www.compphys.de
>> >>
>> >> --
>> >> You received this message because you are subscribed to a topic in the
>> >> Google Groups "votca" group.
>> >> To unsubscribe from this topic, visit
>> >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> >> To unsubscribe from this group and all its topics, send an email to

>> >> votca+un...@googlegroups.com.

>> >> To post to this group, send email to vo...@googlegroups.com.
>> >> Visit this group at https://groups.google.com/group/votca.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "votca" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an

>> > email to votca+un...@googlegroups.com.

>> > To post to this group, send email to vo...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/votca.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>> --
>> Christoph Junghans
>> Web: http://www.compphys.de
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "votca" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to

>> votca+un...@googlegroups.com.

>> To post to this group, send email to vo...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/votca.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "votca" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to votca+un...@googlegroups.com.

[Andrey Brukhno]

OK, but its output does not contain any info about the exclusions.

 

Remember, if you are using a mapping file ("--cg" option) exclusions
from the topology file are ignored and the ones from the mapping file
are used instead.

Alright, this must be the root of my problem!

Apparently, in v1.3 csg_stat would take the exclusions from tpr, not xml,

because I never provided any exclusion list in any xml file, but it did work correctly!

I will look into this again.

Andrey

 

>> >> > email to votca+unsubscribe@googlegroups.com.

>> >> > To post to this group, send email to vo...@googlegroups.com.
>> >> > Visit this group at https://groups.google.com/group/votca.
>> >> > For more options, visit https://groups.google.com/d/optout.
>> >>
>> >>
>> >>
>> >> --
>> >> Christoph Junghans
>> >> Web: http://www.compphys.de
>> >>
>> >> --
>> >> You received this message because you are subscribed to a topic in the
>> >> Google Groups "votca" group.
>> >> To unsubscribe from this topic, visit
>> >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> >> To unsubscribe from this group and all its topics, send an email to

>> >> votca+unsubscribe@googlegroups.com.

>> >> To post to this group, send email to vo...@googlegroups.com.
>> >> Visit this group at https://groups.google.com/group/votca.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "votca" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an

>> > email to votca+unsubscribe@googlegroups.com.

>> > To post to this group, send email to vo...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/votca.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>> --
>> Christoph Junghans
>> Web: http://www.compphys.de
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "votca" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to

>> votca+unsubscribe@googlegroups.com.

>> To post to this group, send email to vo...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/votca.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "votca" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to votca+unsubscribe@googlegroups.com.

> To post to this group, send email to vo...@googlegroups.com.
> Visit this group at https://groups.google.com/group/votca.
> For more options, visit https://groups.google.com/d/optout.

--
Christoph Junghans
Web: http://www.compphys.de

--
You received this message because you are subscribed to a topic in the Google Groups "votca" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to votca+unsubscribe@googlegroups.com.

[Andrey Brukhno]

Sorry, I forgot - below is the output of `grep critical inverse.log' for one iteration:

Running critical command 'gmx grompp -n index.ndx -f grompp.mdp -p topol.top -o topol.tpr -c conf.gro'

Running critical command 'gmx grompp -n index.ndx -f grompp.mdp -p topol.top -o topol.tpr -c conf.gro'
Running critical command 'gmx grompp -n index.ndx -f grompp.mdp -p topol.top -o topol.tpr -c conf.gro'
Running critical command 'gmx grompp -n index.ndx -f grompp.mdp -p topol.top -o topol.tpr -c conf.gro'
Running critical command 'mpirun -n 4 mdrun_mpi -s topol.tpr -c confout.gro -o traj.trr -x traj.xtc -cpi state.cpt -maxh 47.8289 -multidir sim_0 sim_1 sim_2 sim_3 -tablep table.xvg -tableb table_b1.xvg table_b2.xvg table_b3.xvg table_b4.xvg table_b5.xvg table_a1.xvg table_a2.xvg table_a3.xvg table_a4.xvg table_a5.xvg table_d1.xvg table_d2.xvg table_d3.xvg table_d4.xvg table_d5.xvg table_d6.xvg'
Doing: critical csg_stat --nt 4 --options /home/andrey/Work/Models/DOPC-long-wtip4p-new/ibi-Rc1.65nm-freeN-dihs_CG-fits1-151cNT/dopc_cg-int-map11.xml --top topol-rdf.tpr --trj traj.xtc --begin 0 --first-frame 0 --block-length 1000000 --ext dist.block --cg /home/andrey/Work/Models/DOPC-long-wtip4p-new/ibi-Rc1.65nm-freeN-dihs_CG-fits1-151cNT/dopc_cg-map11.xml;

As you can see, the only mention of `critical csg_stat' is due to my addition of echo.

Andrey

[Christoph Junghans]

On Mon, Jul 23, 2018 at 6:39 AM, 'Andrey Brukhno' via votca

This behavior hasn't changed for long time and was certainly the same
in v1.3 as well.
When using a mapping file, exclusions get generated for each bond,
angle and dihedral interaction in the mapping file.

Why are using a mapping file to begin with?

Christoph

>> >> >> > email to votca+un...@googlegroups.com.

>> >> >> > To post to this group, send email to vo...@googlegroups.com.
>> >> >> > Visit this group at https://groups.google.com/group/votca.
>> >> >> > For more options, visit https://groups.google.com/d/optout.
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Christoph Junghans
>> >> >> Web: http://www.compphys.de
>> >> >>
>> >> >> --
>> >> >> You received this message because you are subscribed to a topic in
>> >> >> the
>> >> >> Google Groups "votca" group.
>> >> >> To unsubscribe from this topic, visit
>> >> >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> >> >> To unsubscribe from this group and all its topics, send an email to

>> >> >> votca+un...@googlegroups.com.

>> >> >> To post to this group, send email to vo...@googlegroups.com.
>> >> >> Visit this group at https://groups.google.com/group/votca.
>> >> >> For more options, visit https://groups.google.com/d/optout.
>> >> >
>> >> >
>> >> > --
>> >> > You received this message because you are subscribed to the Google
>> >> > Groups
>> >> > "votca" group.
>> >> > To unsubscribe from this group and stop receiving emails from it,
>> >> > send
>> >> > an

>> >> > email to votca+un...@googlegroups.com.

>> >> > To post to this group, send email to vo...@googlegroups.com.
>> >> > Visit this group at https://groups.google.com/group/votca.
>> >> > For more options, visit https://groups.google.com/d/optout.
>> >>
>> >>
>> >>
>> >> --
>> >> Christoph Junghans
>> >> Web: http://www.compphys.de
>> >>
>> >> --
>> >> You received this message because you are subscribed to a topic in the
>> >> Google Groups "votca" group.
>> >> To unsubscribe from this topic, visit
>> >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> >> To unsubscribe from this group and all its topics, send an email to

>> >> votca+un...@googlegroups.com.

>> >> To post to this group, send email to vo...@googlegroups.com.
>> >> Visit this group at https://groups.google.com/group/votca.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "votca" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an

>> > email to votca+un...@googlegroups.com.

>> > To post to this group, send email to vo...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/votca.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>> --
>> Christoph Junghans
>> Web: http://www.compphys.de
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "votca" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to

>> votca+un...@googlegroups.com.

>> To post to this group, send email to vo...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/votca.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "votca" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to votca+un...@googlegroups.com.

[Christoph Junghans]

Well, the other lines with "Running critical command" show that the
logging actually works.
Did you overload calc_rdf_generic.sh with your own version?
Otherwise, I would say your installation is bourked up.

Christoph

[Andrey Brukhno]

I am only reporting my experience: version 1.3 (installed on our cluster 8 Feb 2015) did not need any extra exclusion definitions (nrex) nor list and I had no problem obtaining correct distributions with topol-rdf.tpr only (the number provided by [moleculetype] is 10).

But now, as of v1.41, the distributions correspond to exclusions starting with 1-4 intra-molecular pairs (the number provided by [moleculetype] is 3).

Since I did not have any issue before, I did not care about generating exclusion lists in addition to tpr files.

 

When using a mapping file, exclusions get generated for each bond,
angle and dihedral interaction in the mapping file.

Why are using a mapping file to begin with?

Good question!

Well, the manual advises to use cg.inverse.map tag with a one-to-one mapping xml file for iterative methods even when optimising only non-bonded interactions (and I recall having problems without this tag!), so I have:

     <map>
      <DESC>Special mapping file for rdf calculations needed for bonded interactions</DESC>
      dopc_cg-map11.xml
    </map>

and of course I do not do anything special about the csg_stat call in calc_rdf_generic.sh, so it does what it does.

Andrey

>> >> >> > email to votca+unsubscribe@googlegroups.com.

>> >> >> > To post to this group, send email to vo...@googlegroups.com.
>> >> >> > Visit this group at https://groups.google.com/group/votca.
>> >> >> > For more options, visit https://groups.google.com/d/optout.
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Christoph Junghans
>> >> >> Web: http://www.compphys.de
>> >> >>
>> >> >> --
>> >> >> You received this message because you are subscribed to a topic in
>> >> >> the
>> >> >> Google Groups "votca" group.
>> >> >> To unsubscribe from this topic, visit
>> >> >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> >> >> To unsubscribe from this group and all its topics, send an email to

>> >> >> votca+unsubscribe@googlegroups.com.

>> >> >> To post to this group, send email to vo...@googlegroups.com.
>> >> >> Visit this group at https://groups.google.com/group/votca.
>> >> >> For more options, visit https://groups.google.com/d/optout.
>> >> >
>> >> >
>> >> > --
>> >> > You received this message because you are subscribed to the Google
>> >> > Groups
>> >> > "votca" group.
>> >> > To unsubscribe from this group and stop receiving emails from it,
>> >> > send
>> >> > an

>> >> > email to votca+unsubscribe@googlegroups.com.

>> >> > To post to this group, send email to vo...@googlegroups.com.
>> >> > Visit this group at https://groups.google.com/group/votca.
>> >> > For more options, visit https://groups.google.com/d/optout.
>> >>
>> >>
>> >>
>> >> --
>> >> Christoph Junghans
>> >> Web: http://www.compphys.de
>> >>
>> >> --
>> >> You received this message because you are subscribed to a topic in the
>> >> Google Groups "votca" group.
>> >> To unsubscribe from this topic, visit
>> >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> >> To unsubscribe from this group and all its topics, send an email to

>> >> votca+unsubscribe@googlegroups.com.

>> >> To post to this group, send email to vo...@googlegroups.com.
>> >> Visit this group at https://groups.google.com/group/votca.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "votca" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an

>> > email to votca+unsubscribe@googlegroups.com.

>> > To post to this group, send email to vo...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/votca.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>> --
>> Christoph Junghans
>> Web: http://www.compphys.de
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "votca" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to

>> votca+unsubscribe@googlegroups.com.

>> To post to this group, send email to vo...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/votca.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "votca" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to votca+unsubscribe@googlegroups.com.

> To post to this group, send email to vo...@googlegroups.com.
> Visit this group at https://groups.google.com/group/votca.
> For more options, visit https://groups.google.com/d/optout.

--
Christoph Junghans
Web: http://www.compphys.de

--
You received this message because you are subscribed to a topic in the Google Groups "votca" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to votca+unsubscribe@googlegroups.com.

[Andrey Brukhno]

Nope! It is what spack installed for me, except I recently added that `echo' for `critical csg_stat ...'

I do use my run_gromacs.sh and run_gromacs_tasks.sh scripts but these do not affect RDF calculations.

 

Otherwise, I would say your installation is bourked up.

Spack to blame then?

 

Andrey

> email to votca+unsubscribe@googlegroups.com.

> To post to this group, send email to vo...@googlegroups.com.
> Visit this group at https://groups.google.com/group/votca.
> For more options, visit https://groups.google.com/d/optout.

--
Christoph Junghans
Web: http://www.compphys.de

--
You received this message because you are subscribed to a topic in the Google Groups "votca" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to votca+unsubscribe@googlegroups.com.

[Christoph Junghans]

On Mon, Jul 23, 2018 at 7:02 AM, 'Andrey Brukhno' via votca

Well then don't specify a mapping or a secondary tpr file. Both
together don't make much sense.

>
>>
>> When using a mapping file, exclusions get generated for each bond,
>> angle and dihedral interaction in the mapping file.
>>
>> Why are using a mapping file to begin with?
>
>
> Good question!
> Well, the manual advises to use cg.inverse.map tag with a one-to-one mapping
> xml file for iterative methods even when optimising only non-bonded
> interactions (and I recall having problems without this tag!), so I have:

I guess you are referring to section 7.3.2, but that is referring to
IMC (not IBI).
And even then, that has already been improved a while (pre-v1.3) ago,
so that section should probably get removed unless you remember a
reason why it is needed for IMC.

>> >> >> >> > email to votca+un...@googlegroups.com.

>> >> >> >> > To post to this group, send email to vo...@googlegroups.com.
>> >> >> >> > Visit this group at https://groups.google.com/group/votca.
>> >> >> >> > For more options, visit https://groups.google.com/d/optout.
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> Christoph Junghans
>> >> >> >> Web: http://www.compphys.de
>> >> >> >>
>> >> >> >> --
>> >> >> >> You received this message because you are subscribed to a topic
>> >> >> >> in
>> >> >> >> the
>> >> >> >> Google Groups "votca" group.
>> >> >> >> To unsubscribe from this topic, visit
>> >> >> >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> >> >> >> To unsubscribe from this group and all its topics, send an email
>> >> >> >> to

>> >> >> >> votca+un...@googlegroups.com.

>> >> >> >> To post to this group, send email to vo...@googlegroups.com.
>> >> >> >> Visit this group at https://groups.google.com/group/votca.
>> >> >> >> For more options, visit https://groups.google.com/d/optout.
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > You received this message because you are subscribed to the Google
>> >> >> > Groups
>> >> >> > "votca" group.
>> >> >> > To unsubscribe from this group and stop receiving emails from it,
>> >> >> > send
>> >> >> > an

>> >> >> > email to votca+un...@googlegroups.com.

>> >> >> > To post to this group, send email to vo...@googlegroups.com.
>> >> >> > Visit this group at https://groups.google.com/group/votca.
>> >> >> > For more options, visit https://groups.google.com/d/optout.
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Christoph Junghans
>> >> >> Web: http://www.compphys.de
>> >> >>
>> >> >> --
>> >> >> You received this message because you are subscribed to a topic in
>> >> >> the
>> >> >> Google Groups "votca" group.
>> >> >> To unsubscribe from this topic, visit
>> >> >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> >> >> To unsubscribe from this group and all its topics, send an email to

>> >> >> votca+un...@googlegroups.com.

>> >> >> To post to this group, send email to vo...@googlegroups.com.
>> >> >> Visit this group at https://groups.google.com/group/votca.
>> >> >> For more options, visit https://groups.google.com/d/optout.
>> >> >
>> >> >
>> >> > --
>> >> > You received this message because you are subscribed to the Google
>> >> > Groups
>> >> > "votca" group.
>> >> > To unsubscribe from this group and stop receiving emails from it,
>> >> > send
>> >> > an

>> >> > email to votca+un...@googlegroups.com.

>> >> > To post to this group, send email to vo...@googlegroups.com.
>> >> > Visit this group at https://groups.google.com/group/votca.
>> >> > For more options, visit https://groups.google.com/d/optout.
>> >>
>> >>
>> >>
>> >> --
>> >> Christoph Junghans
>> >> Web: http://www.compphys.de
>> >>
>> >> --
>> >> You received this message because you are subscribed to a topic in the
>> >> Google Groups "votca" group.
>> >> To unsubscribe from this topic, visit
>> >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> >> To unsubscribe from this group and all its topics, send an email to

>> >> votca+un...@googlegroups.com.

>> >> To post to this group, send email to vo...@googlegroups.com.
>> >> Visit this group at https://groups.google.com/group/votca.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "votca" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an

>> > email to votca+un...@googlegroups.com.

>> > To post to this group, send email to vo...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/votca.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>> --
>> Christoph Junghans
>> Web: http://www.compphys.de
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "votca" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to

>> votca+un...@googlegroups.com.

>> To post to this group, send email to vo...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/votca.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "votca" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to votca+un...@googlegroups.com.

[Andrey Brukhno]

Christoph,

I think I figured why there is no log msg for critical csg_stat in calc_rdfgeneric.sh.

The fact is I override the inverse.sh script with the one having this (to avoid all the frame numbers when reading trajectory files):

   log="${PWD}/${log##*/}"
   export CSGLOG="$log"
   if [[ -f $CSGLOG ]]; then
     exec 3>&1 4>&2 >> "$CSGLOG" 2>&1
     echo -e "\n\n##################################"
 #    echo "# Appending to existing logfile #"
     echo    "# Appending to logfile (+errors) #"
     echo -e "##################################\n\n"
     msg --color blue "Appending to existing logfile (+errors) ${CSGLOG##*/}"
   else
     exec 3>&1 4>&2 >> "$CSGLOG" 2>/dev/null
     echo -e "\n\n##################################"
     echo    "# Creating new logfile (-errors) #"
     echo -e "##################################\n\n"
     msg "For a more verbose log see: ${CSGLOG##*/} (or csg_inverse.log)"
   fi

This worked before and did not affect the workflow.

Andrey

> email to votca+unsubscribe@googlegroups.com.

> To post to this group, send email to vo...@googlegroups.com.
> Visit this group at https://groups.google.com/group/votca.
> For more options, visit https://groups.google.com/d/optout.

--
Christoph Junghans
Web: http://www.compphys.de

--
You received this message because you are subscribed to a topic in the Google Groups "votca" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to votca+unsubscribe@googlegroups.com.

[Christoph Junghans]

Honestly, I don't think so. Spack is just calling CMake for the installation.

I don't understand why it worked in one instance of calling
"critical()", but not in the other, my guess is your system is just
special as I have never seen that problem before.

Christoph

[Andrey Brukhno]

>> >> > So something should have changed somewhere within the newly installed

>> >> > Gromacs/VOTCA combination.
>> >> You can see what VOTCA has with:
>> >> csg_dump --top topol-rdf.tpr --cg dopc_cg-map11.xml
>> >
>> >
>> > OK, but its output does not contain any info about the exclusions.
>> >
>> >>
>> >>
>> >> Remember, if you are using a mapping file ("--cg" option) exclusions
>> >> from the topology file are ignored and the ones from the mapping file
>> >> are used instead.
>> >
>> >
>> > Alright, this must be the root of my problem!
>> > Apparently, in v1.3 csg_stat would take the exclusions from tpr, not
>> > xml,
>> > because I never provided any exclusion list in any xml file, but it did
>> > work
>> > correctly!
>> This behavior hasn't changed for long time and was certainly the same
>> in v1.3 as well.

Do you mean the cg.inverse.map tag is not needed anymore in .xml for both IBI and IMC nowadays?

Or should I still include the tag for IMC? Because as of now IMC (and also RE) produces wrong distributions too.

 

Andrey

>> >> >> >> > email to votca+unsubscribe@googlegroups.com.

>> >> >> >> > To post to this group, send email to vo...@googlegroups.com.
>> >> >> >> > Visit this group at https://groups.google.com/group/votca.
>> >> >> >> > For more options, visit https://groups.google.com/d/optout.
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> Christoph Junghans
>> >> >> >> Web: http://www.compphys.de
>> >> >> >>
>> >> >> >> --
>> >> >> >> You received this message because you are subscribed to a topic
>> >> >> >> in
>> >> >> >> the
>> >> >> >> Google Groups "votca" group.
>> >> >> >> To unsubscribe from this topic, visit
>> >> >> >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> >> >> >> To unsubscribe from this group and all its topics, send an email
>> >> >> >> to

>> >> >> >> votca+unsubscribe@googlegroups.com.

>> >> >> >> To post to this group, send email to vo...@googlegroups.com.
>> >> >> >> Visit this group at https://groups.google.com/group/votca.
>> >> >> >> For more options, visit https://groups.google.com/d/optout.
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > You received this message because you are subscribed to the Google
>> >> >> > Groups
>> >> >> > "votca" group.
>> >> >> > To unsubscribe from this group and stop receiving emails from it,
>> >> >> > send
>> >> >> > an

>> >> >> > email to votca+unsubscribe@googlegroups.com.

>> >> >> > To post to this group, send email to vo...@googlegroups.com.
>> >> >> > Visit this group at https://groups.google.com/group/votca.
>> >> >> > For more options, visit https://groups.google.com/d/optout.
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Christoph Junghans
>> >> >> Web: http://www.compphys.de
>> >> >>
>> >> >> --
>> >> >> You received this message because you are subscribed to a topic in
>> >> >> the
>> >> >> Google Groups "votca" group.
>> >> >> To unsubscribe from this topic, visit
>> >> >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> >> >> To unsubscribe from this group and all its topics, send an email to

>> >> >> votca+unsubscribe@googlegroups.com.

>> >> >> To post to this group, send email to vo...@googlegroups.com.
>> >> >> Visit this group at https://groups.google.com/group/votca.
>> >> >> For more options, visit https://groups.google.com/d/optout.
>> >> >
>> >> >
>> >> > --
>> >> > You received this message because you are subscribed to the Google
>> >> > Groups
>> >> > "votca" group.
>> >> > To unsubscribe from this group and stop receiving emails from it,
>> >> > send
>> >> > an

>> >> > email to votca+unsubscribe@googlegroups.com.

>> >> > To post to this group, send email to vo...@googlegroups.com.
>> >> > Visit this group at https://groups.google.com/group/votca.
>> >> > For more options, visit https://groups.google.com/d/optout.
>> >>
>> >>
>> >>
>> >> --
>> >> Christoph Junghans
>> >> Web: http://www.compphys.de
>> >>
>> >> --
>> >> You received this message because you are subscribed to a topic in the
>> >> Google Groups "votca" group.
>> >> To unsubscribe from this topic, visit
>> >> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> >> To unsubscribe from this group and all its topics, send an email to

>> >> votca+unsubscribe@googlegroups.com.

>> >> To post to this group, send email to vo...@googlegroups.com.
>> >> Visit this group at https://groups.google.com/group/votca.
>> >> For more options, visit https://groups.google.com/d/optout.
>> >
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "votca" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an

>> > email to votca+unsubscribe@googlegroups.com.

>> > To post to this group, send email to vo...@googlegroups.com.
>> > Visit this group at https://groups.google.com/group/votca.
>> > For more options, visit https://groups.google.com/d/optout.
>>
>>
>>
>> --
>> Christoph Junghans
>> Web: http://www.compphys.de
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "votca" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to

>> votca+unsubscribe@googlegroups.com.

>> To post to this group, send email to vo...@googlegroups.com.
>> Visit this group at https://groups.google.com/group/votca.
>> For more options, visit https://groups.google.com/d/optout.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "votca" group.
> To unsubscribe from this group and stop receiving emails from it, send an

> email to votca+unsubscribe@googlegroups.com.

> To post to this group, send email to vo...@googlegroups.com.
> Visit this group at https://groups.google.com/group/votca.
> For more options, visit https://groups.google.com/d/optout.

--
Christoph Junghans
Web: http://www.compphys.de

--
You received this message because you are subscribed to a topic in the Google Groups "votca" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to votca+unsubscribe@googlegroups.com.

[Christoph Junghans]

Glad, you found it and it is not VOTCA's original code.
I remember now that you did this modification for a reason, which I
didn't like exactly because error message will get lost.

Anyhow, well in conclusion if you send the errors to dev/null, you
won't see some messages (like critical) in the log file.

Christoph

[Andrey Brukhno]

In this case, it's not an error message which is lost ;) Why does it go to stderr at all?

Moreover, if I try to run the iteration a second time, everything including errors is reported back

Anyway, my primary issue is not about messages, but to make it work correctly for distributions.

Andrey

[Christoph Junghans]

On Mon, Jul 23, 2018 at 7:41 AM, 'Andrey Brukhno' via votca
<vo...@googlegroups.com> wrote:
> In this case, it's not an error message which is lost ;) Why does it go to
> stderr at all?

The code has the reason right there:
#print this message to stderr because $(critical something) is used very often

> Moreover, if I try to run the iteration a second time, everything including
> errors is reported back

Well that is good, nevertheless we just spend 5 E-Mails back and
forth, hunting down a bug you introduced yourself.
And I now recall you made that change to filter gromacs "Read frame
XXX" message, which is fixed when using a newer gromacs version
(>=2016.3), see:
https://github.com/votca/csg/commit/155eb770873a6f65bac372c9f8c5efc9f3a4be75

>
> Anyway, my primary issue is not about messages, but to make it work
> correctly for distributions.

Like I said in the other email if section 7.3.2 is right, a mapping
file might be needed only for IMC, but even that I think is not true
anymore in v1.3.1.
So you need to figure out, which exclusions you want to read the ones
from the mapping file or the ones the tpr.
For the former specify a mapping file or the latter don't.

And then you can use csg_dump with or without --cg option to check the
exclusions!
$ csg_dump --top topol.tpr --excl
or
$ csg_dump --top topol.tpr --excl --cg mapping.xml.

[Andrey Brukhno]

On Mon, 23 Jul 2018 15:17 Christoph Junghans, <jung...@votca.org> wrote:

On Mon, Jul 23, 2018 at 7:41 AM, 'Andrey Brukhno' via votca
<vo...@googlegroups.com> wrote:
> In this case, it's not an error message which is lost ;) Why does it go to
> stderr at all?
The code has the reason right there:
#print this message to stderr because $(critical something) is used very often

> Moreover, if I try to run the iteration a second time, everything including
> errors is reported back
Well that is good, nevertheless we just spend 5 E-Mails back and
forth, hunting down a bug you introduced yourself.

As  a way to excuse myself, I did reprort the cause straight away when I recalled my overriding the logging, to stop you from any further search.

The problem was that I have not used VOTCA for a lomg while, so it takes time to refresh myself. Sorry, if reflects on you too. Should get better soon.

And I now recall you made that change to filter gromacs "Read frame
XXX" message, which is fixed when using a newer gromacs version
(>=2016.3), see:
https://github.com/votca/csg/commit/155eb770873a6f65bac372c9f8c5efc9f3a4be75

>
> Anyway, my primary issue is not about messages, but to make it work
> correctly for distributions.
Like I said in the other email if section 7.3.2 is right, a mapping
file might be needed only for IMC, but even that I think is not true
anymore in v1.3.1.
So you need to figure out, which exclusions you want to read the ones
from the mapping file or the ones the tpr.
For the former specify a mapping file or the latter don't.

And then you can use csg_dump with or without --cg option to check the
exclusions!
$ csg_dump --top topol.tpr --excl
or
$ csg_dump --top topol.tpr --excl --cg mapping.xml.

Thanks for clarifying this!

Andrey

[Andrey Brukhno]

On Monday, July 23, 2018 at 3:17:49 PM UTC+1, Christoph Junghans wrote:

On Mon, Jul 23, 2018 at 7:41 AM, 'Andrey Brukhno' via votca
<vo...@googlegroups.com> wrote:
> In this case, it's not an error message which is lost ;) Why does it go to
> stderr at all?
The code has the reason right there:
#print this message to stderr because $(critical something) is used very often

> Moreover, if I try to run the iteration a second time, everything including
> errors is reported back
Well that is good, nevertheless we just spend 5 E-Mails back and
forth, hunting down a bug you introduced yourself.
And I now recall you made that change to filter gromacs "Read frame
XXX" message, which is fixed when using a newer gromacs version
(>=2016.3), see:
https://github.com/votca/csg/commit/155eb770873a6f65bac372c9f8c5efc9f3a4be75

>
> Anyway, my primary issue is not about messages, but to make it work
> correctly for distributions.
Like I said in the other email if section 7.3.2 is right, a mapping
file might be needed only for IMC, but even that I think is not true
anymore in v1.3.1.

My test IMC iteration went through without tagging the 1:1 CG map file.

 

So you need to figure out, which exclusions you want to read the ones
from the mapping file or the ones the tpr.
For the former specify a mapping file or the latter don't.

OK, I preferred the exclusions from topol-rdf.tpr (which appears to work now for IBI), so I skipped the map tags in my settings file for IMC too. However, the distributions are still wrong in the case of IMC.

[Andrey Brukhno]

Well, I can of course, but v1.4.1 does not use any special rdf.topol tag (so I have to introduce it myself):

sim_prog="$(csg_get_property cg.inverse.program)"

topol=$(csg_get_property cg.inverse.$sim_prog.topol)

[[ -f $topol ]] || die "${0##*/}: topol file '$topol' not found, possibly you have to add it to cg.inverse.filelist"

traj=$(csg_get_property cg.inverse.$sim_prog.traj)

[[ -f $traj ]] || die "${0##*/}: traj file '$traj' not found"

equi_time="$(csg_get_property cg.inverse.$sim_prog.equi_time)"

if [[ ${CSG_RUNTEST} ]] && csg_calc "$equi_time" ">" "0"; then

  msg --color blue --to-stderr "Automatically setting equi_time to 0, because CSG_RUNTEST was set"

  equi_time=0

fi

first_frame="$(csg_get_property cg.inverse.$sim_prog.first_frame)"

tasks=$(get_number_tasks)

msg "Calculating IMC statistics using $tasks tasks"

if is_done "imc_analysis"; then

  echo "IMC analysis is already done"

else

  #copy+resample all target dist in $this_dir

  for_all "non-bonded bonded" do_external resample target '$(csg_get_interaction_property inverse.target)' '$(csg_get_interaction_property name).dist.tgt'

  critical csg_stat --do-imc --options "$CSGXMLFILE" --top "$topol" --trj "$traj" --begin $equi_time --first-frame $first_frame --nt $tasks

  mark_done "imc_analysis"

fi

[Andrey Brukhno]

As a matter of fact including a 1:1 topology xml where all the intra-molecular pairs are counted as bonded does not affect the exclusion list (so only tpr file matters) for csg_dump:

1) the case where topol.tpr has nrexcl = 3 and  topol-map11-rdf.xml defines all pairs as bonded

> csg_dump --top topol.tpr --map topol-map11-rdf.xml --excl

Reading file topol.tpr, VERSION 2018.2 (single precision)

I have 12 beads in 1 molecule

List of exclusions:

1 2 3 4

2 3 4 9

3 4 9 10 11

4 8 9 10 11 12

5 6 7 8

6 7 8 10

7 8 9 10

8 9 10 11

9 10 11 12

10 11 12

11 12

2) the case where topol.tpr has nrexcl = 3 and no topol-map11-rdf.xml is used

> csg_dump --top topol.tpr  --excl

Reading file topol.tpr, VERSION 2018.2 (single precision)

I have 12 beads in 1 molecule

List of exclusions:

1 2 3 4

2 3 4 9

3 4 9 10 11

4 8 9 10 11 12

5 6 7 8

6 7 8 10

7 8 9 10

8 9 10 11

9 10 11 12

10 11 12

11 12

13 14 15 16

14 15 16 21

15 16 21 22 23

16 20 21 22 23 24

17 18 19 20

18 19 20 22

19 20 21 22

20 21 22 23

21 22 23 24

22 23 24

23 24

3) the case where topol-rdf.tpr has nrexcl = 10 so all pairs within a chain are excluded from RDF calculation

> csg_dump --top topol-rdf.tpr  --excl

Reading file topol-rdf.tpr, VERSION 2018.2 (single precision)

I have 12 beads in 1 molecule

List of exclusions:

1 2 3 4 5 6 7 8 9 10 11 12

2 3 4 5 6 7 8 9 10 11 12

3 4 5 6 7 8 9 10 11 12

4 5 6 7 8 9 10 11 12

5 6 7 8 9 10 11 12

6 7 8 9 10 11 12

7 8 9 10 11 12

8 9 10 11 12

9 10 11 12

10 11 12

11 12

 

[Christoph Junghans]

https://github.com/votca/csg-manual/pull/8

[Christoph Junghans]

This is interesting, too! csg_stat for IMC is always called without
mapping files.

Christoph

[Christoph Junghans]

I am quite sure what you want to tell us with that comparison! If you
want to achieve the same as nrexcl=10 with a mapping file, you will
need to add 12*11/2=66 bonds between all possible pairs to that
mapping file.

Christoph

[Andrey Brukhno]

This is why I preferred to use topol-rdf.tpr and not the mapping xml file. But since you suggested that a mapping file might be needed in some cases, like IMC with bonds, I tried that as well. But apparently mapping file does not affect the exclusions at all (there is no difference for my two mapping files)! 

The bottom line here: adding bonds for all pairs in mapping file does not affect the exclusion list at all - it does look like exclusion list comes from tpr only.

I am stopping now testing this, because the workflow works now for me, with my modifications in mc_stat_generic.sh

Andrey

>>
>>
>>>
>>>

[Andrey Brukhno]

It seems that mapping files are only used for IBI.

Andrey

> email to votca+unsubscribe@googlegroups.com.

> To post to this group, send email to vo...@googlegroups.com.
> Visit this group at https://groups.google.com/group/votca.
> For more options, visit https://groups.google.com/d/optout.

--
Christoph Junghans
Web: http://www.compphys.de

--
You received this message because you are subscribed to a topic in the Google Groups "votca" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.

To unsubscribe from this group and all its topics, send an email to votca+unsubscribe@googlegroups.com.

[Christoph Junghans]

On Tue, Jul 24, 2018 at 2:33 AM, 'Andrey Brukhno' via votca
<vo...@googlegroups.com> wrote:
> It seems that mapping files are only used for IBI.

This absolutely correct, mapping files are currently only implemented
for IBI and not for IMC, RE.

Christoph

[Andrey Brukhno]

Not only that (mapping) but also calculation of distributions with errors seems to be only supported for IBI.

("seems" because I only checked calc_rdf_generic.sh, imc_stat_generic.sh and update_re.sh)

Andrey

[Christoph Junghans]

On Tue, Jul 24, 2018 at 2:31 AM, 'Andrey Brukhno' via votca

Just to clarify, the section 7.3.2 of the manual, which suggest you
need a mapping file, is obsolete and got removed in:
https://github.com/votca/csg-manual/pull/8.
Mapping files are only implemented for RDF calculations for IBI (and
the optimizer) at this point and aren't used for methods like IMC and
RE.
It would be easy to add, but nobody has asked for it yet.

>
> The bottom line here: adding bonds for all pairs in mapping file does not
> affect the exclusion list at all - it does look like exclusion list comes
> from tpr only.

Sorry, but that statement is just wrong, but must have made a mistake
in your mapping file.

VOTCA reads the exclusions from the topology file (tpr, xml or
whatever), but if a mapping file is specified a mapping is done (1:1
or not) and after the mapping exclusions get re-generated from the
bonded interactions in the mapping with an "nrexcl=1"-like algorithm,
i.e. beads in a bond, angle or dihedral are excluded.

One can test that very easily using the "hexane/ibi_all" example:
These are the exclusions in the tpr (from step_001):
$ csg_dump --top topol.tpr --excl | head -5
I have 3000 beads in 1000 molecules

List of exclusions:
1 2 3

2 3

Using the included mapping file with 2 bonds and 1 angle, one can get
the same exclusions:
$ csg_dump --top topol.tpr --excl --cg ../hexane_cg.xml | head -6
I have 3000 beads in 1000 molecules
I have 3000 beads in 1000 molecules for the coarsegraining

List of exclusions:
1 2 3

2 3

When removing the angle from the mapping file, one get less exclusions:
$ csg_dump --top topol.tpr --excl --cg ./hexane_noangle.xml | head -6
I have 3000 beads in 1000 molecules
I have 3000 beads in 1000 molecules for the coarsegraining

List of exclusions:
1 2

2 3

When removing the bonds and angle from the mapping file, one get no exclusions:
csg_dump --top topol.tpr --excl --cg ./hexane_cg.xml | head -6
I have 3000 beads in 1000 molecules
I have 3000 beads in 1000 molecules for the coarsegraining

List of exclusions:
<nothing>

So everything works like expected. However, you won't be able to use
exclusions from mapping files out side of IBI anyhow.

> I am stopping now testing this, because the workflow works now for me, with
> my modifications in mc_stat_generic.sh

The modification you did is already part of the current development
branch since Aug 2017 and hence will be part of VOTCA v1.5.

Christoph
>
> Andrey

[Christoph Junghans]

On Tue, Jul 24, 2018 at 5:48 AM, 'Andrey Brukhno' via votca
<vo...@googlegroups.com> wrote:
> Not only that (mapping) but also calculation of distributions with errors
> seems to be only supported for IBI.
> ("seems" because I only checked calc_rdf_generic.sh, imc_stat_generic.sh and
> update_re.sh)

Yes, that is correctly only the rdf calculation (used in IBI and the
optimizer) has error estimation.
In addition, the kbibi and cibi post-update and the density
calculation (for the optimizer) has error estimation.

If isn't hard to implement for IMC using csg_stat's --block-length
option, but you will need a bunch more statistics to do it.
Pull requests are always welcome.

Christoph

[Andrey Brukhno]

On Tuesday, July 24, 2018 at 12:48:40 PM UTC+1, Christoph Junghans wrote:

On Tue, Jul 24, 2018 at 2:31 AM, 'Andrey Brukhno' via votca
<vo...@googlegroups.com> wrote:
>
>
> On Tue, Jul 24, 2018 at 1:34 AM, Christoph Junghans <jung...@votca.org>
> wrote:
>>
>> >
>> > 1) the case where topol.tpr has nrexcl = 3 and  topol-map11-rdf.xml
>> > defines
>> > all pairs as bonded
>> >
>> >> csg_dump --top topol.tpr --map topol-map11-rdf.xml --excl
>> > Reading file topol.tpr, VERSION 2018.2 (single precision)
>> > I have 12 beads in 1 molecule
>> >
>> > List of exclusions:
>> > 1 2 3 4
>> > 2 3 4 9
>> > 3 4 9 10 11
>> > 4 8 9 10 11 12
>> > 5 6 7 8
>> > 6 7 8 10
>> > 7 8 9 10
>> > 8 9 10 11
>> > 9 10 11 12
>> > 10 11 12
>> > 11 12
>> >

> This is why I preferred to use topol-rdf.tpr and not the mapping xml file.
> But since you suggested that a mapping file might be needed in some cases,
> like IMC with bonds, I tried that as well. But apparently mapping file does
> not affect the exclusions at all (there is no difference for my two mapping
> files)!
Just to clarify, the section 7.3.2 of the manual, which suggest you
need a mapping file, is obsolete and got removed in:
https://github.com/votca/csg-manual/pull/8.
Mapping files are only implemented for RDF calculations for IBI (and
the optimizer) at this point and aren't used for methods like IMC and
RE.
It would be easy to add, but nobody has asked for it yet.

For my current purposes (and I think generally) taking exclusions from a special tpr file is more convenient.

It is also much less hassle to prepare, as well as to implement in the workflows.

 

>
> The bottom line here: adding bonds for all pairs in mapping file does not
> affect the exclusion list at all - it does look like exclusion list comes
> from tpr only.
Sorry, but that statement is just wrong, but must have made a mistake
in your mapping file.

OK, I have to admit to using a wrong flag `--map' instead of `--cg' above.

With the correct flag the exclusions are calculated correctly based on mapping xml file.

 

Andrey

[Andrey Brukhno]

I guess the mapping xml for special exclusions might be needed with packages other than Gromacs.

E.g. DL_POLY certainly does not provide an easy way of altering exclusion lists.

This remark is just for completeness of discussion...

Andrey

--
You received this message because you are subscribed to a topic in the Google Groups "votca" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/votca/l42VuhKqQb4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to votca+unsubscribe@googlegroups.com.

[Christoph Junghans]

On Tue, Jul 24, 2018 at 6:52 AM, 'Andrey Brukhno' via votca
<vo...@googlegroups.com> wrote:
> I guess the mapping xml for special exclusions might be needed with packages
> other than Gromacs.
> E.g. DL_POLY certainly does not provide an easy way of altering exclusion
> lists.

Another options to use an xml topology, which Jakub has implemented
(see end of section 3.3), for the rdf calculation. This works with any
simulation backend even when Gromacs support isn't compiled in.

Christoph