enquiry on FES_from_Reweighting.py and FES_from_State.py in OPES_METAD Tutorial

[Ming Tang]

Dear support,

I am trying OPES_META using two torsion as CVs following this tutorial https://www.plumed.org/doc-master/user-doc/html/opes-metad.html

But I got huge FES as attached using FES_from_Reweighting.py with the below command

./FES_from_Reweighting.py --kt 0.616033 --sigma 0.01129792112539913  --blocks 20

sigma is the sigma value of last line in KERNELS, kt is for 310 K. The three parameters defined in plumed.dat are as TEMP=310 BARRIER=15 PACE=500

Are there anything wrong with my parameters?

I also got below problems when using ./FES_from_State.py --kt 0.616033 --all_stored

  File "./FES_from_State.py", line 69, in <module>
    data=pd.read_table(filename,sep='\s+',header=None)
  File "/home/tm/anaconda3/lib/python3.8/site-packages/pandas/io/parsers.py", line 689, in read_table
    return _read(filepath_or_buffer, kwds)
  File "/home/tm/anaconda3/lib/python3.8/site-packages/pandas/io/parsers.py", line 468, in _read
    return parser.read(nrows)
  File "/home/tm/anaconda3/lib/python3.8/site-packages/pandas/io/parsers.py", line 1057, in read
    index, columns, col_dict = self._engine.read(nrows)
  File "/home/tm/anaconda3/lib/python3.8/site-packages/pandas/io/parsers.py", line 2061, in read
    data = self._reader.read(nrows)
  File "pandas/_libs/parsers.pyx", line 756, in pandas._libs.parsers.TextReader.read
  File "pandas/_libs/parsers.pyx", line 771, in pandas._libs.parsers.TextReader._read_low_memory
  File "pandas/_libs/parsers.pyx", line 827, in pandas._libs.parsers.TextReader._read_rows
  File "pandas/_libs/parsers.pyx", line 814, in pandas._libs.parsers.TextReader._tokenize_rows
  File "pandas/_libs/parsers.pyx", line 1951, in pandas._libs.parsers.raise_parser_error
pandas.errors.ParserError: Error tokenizing data. C error: Expected 6 fields in line 71454, saw 8

I checked line 71454, it is the headline which is the same (with 6 fields ) as headline of other states. I tried to prosess the first several state, the script works fine. But when I increase the number of state a bit, it gives me errors. The script works fine for the tutorial process on my comptuer. Does anyone have any suggestions for me?

Thanks,

Ming

[Michele Invernizzi]

Dear Ming,

High values in the FES are generally not a problem. It just means that you did not sample that CV region, thus the reweighting is telling you that the FES there is extremely high. In the limit of very narrow bandwidth (or if you use a histogram instead of KDE) you would get that in CV regions never explored FES=infinite.

These regions of very high FES will not have any influence when you calculate physical quantities such as the free energy difference between two metastable minima. In this sense it is not a problem and the precise reweight estimate there has no importance. 

OPES typically gives a reliable estimate of the FES only a few kbt above the chosen BARRIER, thus in your case I would simply plot the FES in the y range [0:30] and not care about anything above that.

It's not clear to me why kt=0.616033 corresponds to TEMP=310, which units are you using in your plumed.dat? With default units one would have kt=2.577483 kj/mol for TEMP=310

Finally, I am sorry the FES_from_State.py script is failing. It seems that pandas has problems with big files. I suggest two possible solutions:

1. split the STATE file into multiple files, so that each one is smaller than 71454 lines, and run the script on each of these files separately. You should split the file so that each file starts with the correct header.
2. you can install the datatable package, and use the latest version of the FES_from_State.py, that can be found here. This version uses datatable and falls back on pandas only if datatable is not installed. Datatable should be faster than pandas and more robust with big files.

Hope this helps.

Best,

Michele

--
You received this message because you are subscribed to the Google Groups "PLUMED users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to plumed-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/plumed-users/0da85c51-20f5-400f-81b3-9b703e204d2dn%40googlegroups.com.

[Ming Tang]

Dear Michele,

Thanks for your precious support and help.

Oh I see. The plot was from first several state, which matches your explanation that a lot of the regions have not been explored.

I use these units in my plumed.dat: UNITS LENGTH=A TIME=ps ENERGY=kcal/mol

Following the tutorial, I used kt command to convert K into kcal/mol. The energy value seems normal.

I used the second solution, which works and is very convenient. while processing state file with more states, the energy values look normal now. Thanks for the great support.

I want to plot a figure for my system similar to Fig. 6: Free energy estimate obtained from the instantaneous bias in your opes tutorial. Is there a script available in the plumed website? The script will save me a lot of time.

Many thanks,

Ming

[Michele Invernizzi]

Dear Ming,

Now I understand why you were using that value for --kt, thanks!

Regarding the figure in the tutorial, you can obtain it in gnuplot with something like this:

shift=2.5;gamma=10;p [-3:3][:20]'opes/COLVAR'u 2:(-1/(1-1./gamma)*$4+shift):($1/1000) pt 7pal,'ref-fes.dat'w l dt 2 lw 3 lc 0

then I think I used inkscape to put together the opes and the metad figure.

Best,

Michele

To view this discussion on the web visit https://groups.google.com/d/msgid/plumed-users/5b9b6062-0763-4b92-bc23-fb4932ba8183n%40googlegroups.com.

[Ming Tang]

Dear Michele,

Thanks for your precious gnuplot command, great help. 

Using your opes_meta approach in plumed, I set two torsions in a flexible loop as CVs. After some time, the conformation is locked although the CV values do change. Do this mean that my CVs are in your word "degenerated"?

Thanks,

Ming

[Michele Invernizzi]

Dear Ming,

It's hard to say without being an expert of the details of the system. One thing you can try is to run OPES_METAD_EXPLORE with the same input as OPES_METAD, it is more forgiving with the choice of the CV, and by default it can also cross higher barriers. This can help you decide if to change CV and how.

Best, 

Michele

To view this discussion on the web visit https://groups.google.com/d/msgid/plumed-users/e07d72d2-17f6-4310-bd90-281537888a77n%40googlegroups.com.

[Ming Tang]

Dear Michele,

Thanks for your precious suggestion. I tried  OPES_METAD_EXPLORE and found that the pose/conformation that was stuck in  OPES_METAD is not stuck now. I also tried conventional MD, the stuck pose is not stuck.

Cheers,

Ming

[Ming Tang]

Hi Michele,

I am reading your paper here https://pubs.acs.org/doi/full/10.1021/acs.jctc.9b00032. Your free energy surface in Figure 5 looks cool. I am trying to make a similar one. did you do it in Gnuplot? I could not rotate the left figure (attached) as yours. do you still remember how you did it?

thanks,

ming

[Michele Invernizzi]

Hi Ming,

at this link https://archive.materialscloud.org/record/2019.0004/v4 in the figures.zip file you can find all the gnuplot inputs for the figures of the paper.

I personally moved from gnuplot to matplotlib, because I find it easier to make complex figures such as this one.

Best,

Michele

To view this discussion on the web visit https://groups.google.com/d/msgid/plumed-users/2360c4e2-d7ad-4191-93a9-2dac74c43ed3n%40googlegroups.com.

[Ming Tang]

Hi Michele,

Thanks a lot. This is great help!  

Thanks for sharing your experience. I have not tried ploting complicated figures in python yet. But I will follow your advice and try...

Thanks,

Ming