Why the Err values of fullRMC v3.3.0 and v4.1.0 are so different with the same structure and settings?

[Leon Wang]

Hi, Bachior,

I used the same initial structure, experimental data, settings, run file to refine the structure. The initial error is so different.

v3.3

2020-02-06 12:06:50 - fullrmc <INFO> Engine started 17280 steps, total standard error is: 33188.789062
2020-02-06 12:06:50 - fullrmc <INFO> Gen:1 - Tr:1(100.000%) - Acc:1(100.000%) - Rem:0(0.000%) - Err:33186.058594
2020-02-06 12:06:50 - fullrmc <INFO> Gen:2 - Tr:2(100.000%) - Acc:2(100.000%) - Rem:0(0.000%) - Err:33180.695312

2020-02-06 12:06:50 - fullrmc <INFO> Gen:4 - Tr:3(75.000%) - Acc:3(75.000%) - Rem:0(0.000%) - Err:33179.082031
2020-02-06 12:06:50 - fullrmc <INFO> Gen:5 - Tr:4(80.000%) - Acc:4(80.000%) - Rem:0(0.000%) - Err:33177.281250

v4.1

2020-02-17 20:20:33 - fullrmc <INFO> @1 Gen:1 - Tr:1(100.000%) - Acc:1(100.000%) - Rem:0(0.000%) - Err:33323900.000000
2020-02-17 20:20:33 - fullrmc <INFO> @1 Gen:3 - Tr:3(100.000%) - Acc:2(66.667%) - Rem:0(0.000%) - Err:33323896.000000
2020-02-17 20:20:33 - fullrmc <INFO> @1 Gen:4 - Tr:4(100.000%) - Acc:3(75.000%) - Rem:0(0.000%) - Err:33323892.000000
2020-02-17 20:20:33 - fullrmc <INFO> @1 Gen:5 - Tr:5(100.000%) - Acc:4(80.000%) - Rem:0(0.000%) - Err:33323892.000000

after running for one night, I noticed that the Err fitted by V4.1 is not convergent, but V3.3 has changed a lot. Which  one should I trust?

My job is still running (more than 14.5hours). I chose 6h46m08s, which is the time costed by v3.3, as a comparing point.

V3.3

2020-02-06 18:52:57 - fullrmc <INFO> Gen:10022926 - Tr:7598475(75.811%) - Acc:1209074(12.063%) - Rem:100(50.761%) - Err:5171.506348
2020-02-06 18:52:57 - fullrmc <INFO> Gen:10022937 - Tr:7598486(75.811%) - Acc:1209075(12.063%) - Rem:100(50.761%) - Err:5171.505859
2020-02-06 18:52:57 - fullrmc <INFO> Gen:10022938 - Tr:7598487(75.811%) - Acc:1209076(12.063%) - Rem:100(50.761%) - Err:5171.504395

V4.1

2020-02-18 03:06:41 - fullrmc <INFO> @1 Gen:682588 - Tr:484611(70.996%) - Acc:      336559(49.306%) - Rem:0(0.000%) - Err:33310052.000000
2020-02-18 03:06:41 - fullrmc <INFO> @1 Gen:682589 - Tr:484612(70.996%) - Acc:      336560(49.306%) - Rem:0(0.000%) - Err:33310052.000000
2020-02-18 03:06:41 - fullrmc <INFO> @1 Gen:682590 - Tr:484613(70.996%) - Acc:      336561(49.306%) - Rem:0(0.000%) - Err:33310052.000000

Here are my questions:

1. What is the primary difference of fullRMC v4.1 and v3.3 when running PDF simulations? Did the algorithm has been changed or improved?

2. Why the V4.1 costs much longer time than V3.3?

3. Which one should I trust?

4. How can I increase the simulating speed if it is possible?

5. Any further suggestions?

Appreciatively,

Leon

[Leon Wang]

Hi, Bachir,

After running around 18.5hours, the job stopped with some error messages and the Err value is still very big. Attached are the error messages, one of log files, run.py (created by Peter), initial pdb, and gr files.

Following

Best,

Leon

[Bachir Aoun]

Leon,

I am really glad you took upon yourself comparing both versions.

Both simulations are good !! But make sure you use the latest version. Here's what's actually happening:

• First please lose your window function. It's tempting to try to reproduce the exact experimental data 'error function' but that's not always practical. Attached is a plot of your window function ... 
• Your simulation box is considered isolated because your boundary conditions are infinite (see below). I visualized your structure and it's a normal cubic simulation box. you must set up normal boundary conditions
• In fullrmc v 3.3 when boundary conditions are infinite, fullrmc automatically fits a shape function which is not the case anymore in fullrmc 4.1. You must set your shape function parameters.  You can use default values by doing the following
  PDF_CONSTRAINT.set_shape_function_parameters({})
  
    
• Your pdf covers  ridiculously large R values (see attached) you must set your limits to lower values
  PDF_CONSTRAINT.set_limits((None, 45))
  
  
• Once you fix all of that. After 2 min in fullrmc 4.1 you will get the following result. see below

Hope that helps

[Leon Wang]

Hi, Bachir,

Thanks for your suggestion and comments.

1. I did test it and received error messages. Finally, job died. Attached is the bottom records in the last log file. There are WARNING and ERROR messages included. Any suggestions? Thanks.

2. As for your 2nd bullets above, I have set boundary conditions with limited value in the pdb file. I do not understand why you saw my boundary is infinite. If I am wrong, how to set the boundary conditions in my pdb file? According to your documents for fullRMC v4.1, if I add

ENGINE.set_boundary_conditions(np.array([60.37 0.00 0.00 0.12 60.35 0.00 0.00 0.00 52.84])

do you think it will work?

3. I tested the PDF_CONSTRAINT.export(...) function like below:

PDF_CONSTRAINT.export(fileName="constrain_track.log",frame=None, format='%s',   delimiter='\t', comments='#')

It only shows PDF when frame = 0. Your documents does not explain what the frame means. Is it same to the frame or trajectory in MD methods? Would you please explain it? If it is like the frame defined in MD, what do you suggest to extract the frames?

4. I checked the last figure you gave to me. Do you think the density is reasonable based on the figure you gave to me? In my previous questions, you educated me that the slope of simulation at the very beginning should agree with the experimental one. Any comments or suggestion?

Attached run.wly4.1.py is the run file I improved. Would you please look at it and provide me your comments? I sincerely appreciate your time and help.

Thanks for your help.

Appreciatively,

Leon