Resume from previous run in e2spt_refine.py

55 views
Skip to first unread message

tofusha...@g2.nctu.edu.tw

unread,
Nov 1, 2020, 10:43:02 AM11/1/20
to EMAN2
Thanks for all the developers in EMAN2! This is really a good software processing cryo-EM images. 

I'm wondering how to use the resume option in e2spt_refine.py to continue the refinement with the last iteration as the starting point. It seems like the program would get the path in the --reference and initiate the process within that folder without the setting --path option. 

The reason why I do this is to use the initial parameters in the particles_parms_01.json and continuing locally refining the particles based on the initial parameters. I want to skip the random orientational search which is supposed to be done in the first iteration. 

Thank you, 
Hsuan Fu


Muyuan Chen

unread,
Nov 1, 2020, 10:50:52 AM11/1/20
to em...@googlegroups.com
If your version is new enough (probably >2.31), you can use the previous particle_parms_xx.json file as input, instead of the lst file, so the program will read the orientations. You can also give it a —refine option so it only searches around the given orientation.
--
--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2

---
You received this message because you are subscribed to the Google Groups "EMAN2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/36f9dac0-dc7d-45f6-a81e-ccdf3214db78n%40googlegroups.com.

tofusha...@g2.nctu.edu.tw

unread,
Nov 1, 2020, 3:48:22 PM11/1/20
to EMAN2

Thanks for your reply. 

I ran with the command but I got some errors. 
e2spt_refine.py spt_02/particle_parms_01.json --path spt_02 --refine --niter=2 --sym=c1 --mass=500.0 --goldstandard=30 --pkeep=0.5 --maxtilt=90.0 --parallel=mpi:140:/scratch

Traceback (most recent call last):
  File "/nfs/bartesaghilab/hl325/programs/miniconda2/envs/eman2/bin/e2spt_refine.py", line 295, in <module>
    main()
  File "/nfs/bartesaghilab/hl325/programs/miniconda2/envs/eman2/bin/e2spt_refine.py", line 131, in main
    er=EMData(ref,refn,True)
  File "/nfs/bartesaghilab/hl325/programs/miniconda2/envs/eman2/lib/python3.7/site-packages/EMAN2db.py", line 400, in db_emd_init
    self.read_image_c(*parms)
RuntimeError: FileAccessException at /nfs/bartesaghilab/hl325/programs/eman2/src/libEM/emutil.cpp:420: 'cannot access file ''' caught


g5v...@gmail.com 在 2020年11月1日 星期日上午10:50:52 [UTC-5] 的信中寫道:

MuyuanChen

unread,
Nov 1, 2020, 3:51:30 PM11/1/20
to em...@googlegroups.com
That is why I said you need to have a relatively new version (>2.31). The functionality is only introduced recently. You can switch to the continuous build and try this again.

tofusha...@g2.nctu.edu.tw

unread,
Nov 1, 2020, 3:54:09 PM11/1/20
to EMAN2
I check the version using e2version.py 

EMAN 2.39 ( GITHUB: 2020-11-01 15:25 - commit: 4b9eee8 )
Your Python version is: 3.7.9

Do I use the e2spt_refine.py correctly? 


g5v...@gmail.com 在 2020年11月1日 星期日下午3:51:30 [UTC-5] 的信中寫道:

MuyuanChen

unread,
Nov 1, 2020, 3:56:02 PM11/1/20
to em...@googlegroups.com
Sorry… I just noticed that you did not give it an reference… 

tofusha...@g2.nctu.edu.tw

unread,
Nov 1, 2020, 4:03:52 PM11/1/20
to EMAN2
Thank you! It works.

g5v...@gmail.com 在 2020年11月1日 星期日下午3:56:02 [UTC-5] 的信中寫道:

digvij...@gmail.com

unread,
Jul 28, 2021, 12:43:16 PM7/28/21
to EMAN2
Hi Muyuan

A question related to this thread: 

I get the way to continue from an existing spt_n refinement. I could just use its last particles_parms_m.json file as the particle input, mask.hdf as the mask, model_input as the reference, along with the -refine option. 

But I guess the e2spt_align would still start with default --minres and --maxres values. Or it would be able to decide on the appropriate --minres and --maxres value by parsing the resolution of the threed_m.hdf structure? 
BTW, what is the formula for determining the --minres and --maxres value, given R being the latest resolution of the threed_m.hdf structure?

Thanks,

Muyuan Chen

unread,
Jul 28, 2021, 12:53:57 PM7/28/21
to em...@googlegroups.com
No it won't handle minres/maxres automatically. Over iterations, maxres will be set automatically by measuring the FSC of the last iteration, I think it is currently using the resolution where FSC crosses 0.2 multiplied by 0.75. It works reasonably well but it may be underestimating if you provide a large mask and there is a drastic change in local resolution. I don't have a good way to decide minres so far, and I think it has a lot to do with your imaging conditions and the shape of the protein. I would just take an unmasked map and go to filtertool, apply a highpass filter and see at which resolution the protein features start to disappear. 

digvij...@gmail.com

unread,
Jul 28, 2021, 1:35:39 PM7/28/21
to EMAN2
Cool,

[1] So while re-starting the e2spt_refine.py, I could give an additional flag of --maxres from 0.2 FSC*0.75 of the last fsc_masked_m.txt file. I could also give the --minres by perusing the structure and processing it with filter in the e2filtertool.py.

[2] During the normal e2spt_refine.py, how does the program automatically decide on the --minres value? 

Muyuan Chen

unread,
Jul 28, 2021, 1:44:21 PM7/28/21
to em...@googlegroups.com
Yes.
It does not. I think the default is just the first two Fourier pixels, which shouldn’t hurt (but not necessarily optimal) in most cases.

digvij...@gmail.com

unread,
Jul 28, 2021, 2:44:50 PM7/28/21
to EMAN2
Cool. Thank you
Reply all
Reply to author
Forward
0 new messages