e2spt_refine_new.py - how many iteration? very slow after 3 iterations

103 views
Skip to first unread message

Maia Azubel

unread,
Sep 21, 2021, 3:01:53 PM9/21/21
to EMAN2
I'm using
EMAN 2.99 ( GITHUB: 2021-09-13 10:49 - commit: ba39969d8 )
Your Python version is: 3.7.9

I ran
e2spt_refine_new.py --ptcls sets/ncp.lst --ref old_spt_97_threed_01.hdf --iters p,p,t,r,p,d --parallel=thread:72 --threads=72 --tophat=localwiener

In the help I didn't find an optional argument for number of iterations. Is it niter=5 by default? Is it possible to use --niter? 
My dataset is very large. The first 3 iterations worked fine, though. (~15 hours/iter more or less). However, now it has been about 4 days and it seems that the program is stuck on

######## iter 4 ##########
### Subtilt rotational refinement....
e2spt_subtlt_local.py --ref spt_00/threed_03.hdf --path spt_00 --iter 4 --maxres 4.706352988240001 --parallel thread:72 --refine_trans --aliptcls3d spt_00/aliptcls3d_02.lst --smooth 100 --smoothN 15 --refine_rot
4256615 jobs on 72 CPUs

According to Task Manager
09/14 16:39 inco(45059) e2spt_refine_new.py
09/16 09:17     0%(90282) e2spt_subtlt_local.py

aliptcls3d_03.lst hasn't been created yet.

If I >tail .eman2log.txt
...
2021/09/16 09:17:35 2021/09/16 09:17:35 90279/ 90113 sr20-395c92b4ea.stanford.edu /data/maia2/miniconda3/envs/eman2/bin/e2proc3d.py spt_00/threed_03_odd.hdf spt_00/threed_03_odd.hdf --multfile spt_00/mask.hdf --process normalize.edgemean --ppid=90113
2021/09/16 09:17:35  0%               90282/ 45059 sr20-395c92b4ea.stanford.edu /data/maia2/miniconda3/envs/eman2/bin/e2spt_subtlt_local.py --ref spt_00/threed_03.hdf --path spt_00 --iter 4 --maxres 4.706352988240001 --parallel thread:72 --refine_trans --aliptcls3d spt_00/aliptcls3d_02.lst --smooth 100 --smoothN 15 --refine_rot --ppid=45059

I don't know if it is so slow because the resolution is much better (which has nicely improved from iteration to iteration) or got stuck and I should kill it  (if I top, it looks it's still going)

If the recommendation is to kill it, is there a way to restart from where it stops?

Thanks,
Maia


MuyuanChen

unread,
Sep 21, 2021, 3:14:33 PM9/21/21
to em...@googlegroups.com
Some information can be found at

 * Control the iterative refinement with the --iters option, where different characters denote different types of refinement. `p` for 3D particle orienation, `t` for 2D subtilt translation, `r` for subtilt translation & rotation, and `d` for subtilt defocus refinement.  

So —niter no longer applies, but you can control it in a similar way with —iters. For example, if you want to run five iterations of subtomogram refinement and three iterations of subtilt refinement, you can specify —iters p5,t3. By default the fourth iteration will be subtilt rotational refinement, which should be significant slower than the translation only one. So it is possible the program is still running properly, just have not finished yet. Right now, the program only reports the percentage in the mpi mode (—parallel=mpi:xxx), so the 0% finish isn’t actually wrong. 

Muyuan

On Sep 21, 2021, at 2:01 PM, Maia Azubel <maz...@stanford.edu> wrote:

I'm using 
EMAN 2.99 ( GITHUB: 2021-09-13 10:49 - commit: ba39969d8 )
Your Python version is: 3.7.9

I ran
e2spt_refine_new.py --ptcls sets/ncp.lst --ref old_spt_97_threed_01.hdf --iters p,p,t,r,p,d --parallel=thread:72 --threads=72 --tophat=localwiener

In the help I didn't find an optional argument for number of iterations. Is it niter=5 by default? Is it possible to use --niter?  
My dataset is very large. The first 3 iterations worked fine, though. (~15 hours/iter more or less). However, now it has been about 4 days and it seems that the program is stuck on

######## iter 4 ##########
### Subtilt rotational refinement....
e2spt_subtlt_local.py --ref spt_00/threed_03.hdf --path spt_00 --iter 4 --maxres 4.706352988240001 --parallel thread:72 --refine_trans --aliptcls3d spt_00/aliptcls3d_02.lst --smooth 100 --smoothN 15 --refine_rot
4256615 jobs on 72 CPUs

According to Task Manager 
09/14 16:39 inco(45059) e2spt_refine_new.py
09/16 09:17     0%(90282) e2spt_subtlt_local.py

aliptcls3d_03.lst hasn't been created yet. 

If I >tail .eman2log.txt
...
2021/09/16 09:17:35 2021/09/16 09:17:35 90279/ 90113 sr20-395c92b4ea.stanford.edu/data/maia2/miniconda3/envs/eman2/bin/e2proc3d.py spt_00/threed_03_odd.hdf spt_00/threed_03_odd.hdf --multfile spt_00/mask.hdf --process normalize.edgemean --ppid=90113
2021/09/16 09:17:35  0%               90282/ 45059 sr20-395c92b4ea.stanford.edu/data/maia2/miniconda3/envs/eman2/bin/e2spt_subtlt_local.py --ref spt_00/threed_03.hdf --path spt_00 --iter 4 --maxres 4.706352988240001 --parallel thread:72 --refine_trans --aliptcls3d spt_00/aliptcls3d_02.lst --smooth 100 --smoothN 15 --refine_rot --ppid=45059

I don't know if it is so slow because the resolution is much better (which has nicely improved from iteration to iteration) or got stuck and I should kill it  (if I top, it looks it's still going)

If the recommendation is to kill it, is there a way to restart from where it stops?

Thanks, 
Maia


-- 
-- 
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2

--- 
You received this message because you are subscribed to the Google Groups "EMAN2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/BY5PR02MB6657441ABD1CA04F031B3350DFA19%40BY5PR02MB6657.namprd02.prod.outlook.com.

Maia Azubel

unread,
Sep 21, 2021, 3:18:42 PM9/21/21
to em...@googlegroups.com
Thank you so much for your prompt response, Muyuan.

Maia
From: em...@googlegroups.com <em...@googlegroups.com> on behalf of MuyuanChen <g5v...@gmail.com>
Sent: Tuesday, September 21, 2021 12:14 PM
To: em...@googlegroups.com <em...@googlegroups.com>
Subject: Re: [EMAN2] e2spt_refine_new.py - how many iteration? very slow after 3 iterations
 

MuyuanChen

unread,
Sep 21, 2021, 3:22:58 PM9/21/21
to em...@googlegroups.com
I would like to add that it would be safer to add the —goldstandard option (which is somehow not enabled by default) if you are running on a large dataset. Due to the difference in implementation, overfitting can be a bigger problem here comparing to e2spt_refine.py. 

Muyuan

Maia Azubel

unread,
Sep 22, 2021, 2:18:56 PM9/22/21
to em...@googlegroups.com
Thanks, Muyuan.
First, as you said the 4th iteration (r) was just slower. It has finished now.

I can run it again adding --goldstandard. I noticed, though, that the even without specifying it, the --startres for first iteration was 50A
Are any clear indications of overfitting? The threed_xx.hdf and fsc curves I'm getting look fine.
I was planning to do multi reference alignment next, and don't see either --goldstandard or --goldcontinue for e2spt_refinemulti_new.py

Thanks again,
Maia
Sent: Tuesday, September 21, 2021 12:22 PM

Muyuan Chen

unread,
Sep 22, 2021, 2:28:38 PM9/22/21
to em...@googlegroups.com
You can sometimes tell the overfitting effect from the unrealistic features of the density map, but it is not always obvious. As I said, it is more aggressive than the e2spt_refine routine, but probably still not as subjective to model bias as the single particle routines, since you still have 3D information of individual particles from tomograms. I would do another round of gold standard refinement after the non-gold standard finishes, with --startres set at double the resolution it achieves in the first time. So if your non gold standard refinement get to 10A, do another run with --goldstandard and --startres=20 and see if it can get back to 10A.

There is no resolution calculation in e2spt_refinemulti_new. The resolution range of the comparison is controlled by --minres and --maxres. It seems safer to do the classification at a lower resolution, then push to higher resolution using the single model refinement. You can also using an existing single model refinement and use --loadali3d so it will only do local search while doing classification. 

Muyuan

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/BY5PR02MB665720F441F4F0A97E7AB724DFA29%40BY5PR02MB6657.namprd02.prod.outlook.com.

Maia Azubel

unread,
Sep 22, 2021, 2:34:13 PM9/22/21
to em...@googlegroups.com
Thanks! I was planning to do that (i.e. single model refinement after classification) but will also do as you suggest regarding --goldstandard. 
In comparison to e2refine, my maps actually look better now (no weird/random spikes)

Maia
Sent: Wednesday, September 22, 2021 11:28 AM

Maia Azubel

unread,
Sep 24, 2021, 12:37:06 PM9/24/21
to em...@googlegroups.com
Hi Muyuan,

I have a follow up question, to the thread below.
As mentioned
version
EMAN 2.99 ( GITHUB: 2021-09-13 10:49 - commit: ba39969d8 )
Your Python version is: 3.7.9
Command
e2spt_refine_new.py --ptcls sets/ncp.lst --ref old_spt_97_threed_01.hdf --iters p,p,t,r,p,d --parallel=thread:72 --threads=72 --tophat=localwiener

I was previously using EMAN2.91. I had to move data to a new computer to run EMAN2.99. I transferred from the old eman directory the following directories
titlseries/
tomograms/
info/

As per advice I re-extracted the particles using EMAN2.99
Whe running e2spt_refine_new.py, the first 5 iteration run with no errors but for the last one (d) I got

Error executing:  <e2spt_subtlt_local.SptAlignTask object at 0x7f8de81a6a10>
Traceback (most recent call last):
  File "/data/maia2/miniconda3/envs/eman2/bin/e2parallel.py", line 87, in main
    ret=task.execute(empty_func)
  File "/data/maia2/miniconda3/envs/eman2/bin/e2spt_subtlt_local.py", line 306, in execute
    ctf=m["ctf"]
RuntimeError: NotExistingObjectException at /data/maia2/eman2/libEM/emdata_metadata.cpp:1142: error with 'ctf': 'The requested key does not exist' caught ...
.....
(full error message attached)

Does it mean that the ctf info was not properly transfered? Where is the ctf info stored? Is it that I need to run ctf again? Or is this a completely different issue?

Thanks,
Maia
From: em...@googlegroups.com <em...@googlegroups.com> on behalf of Maia Azubel <maz...@stanford.edu>
Sent: Wednesday, September 22, 2021 11:34 AM
errorMessage_spt_00.txt

Muyuan Chen

unread,
Sep 24, 2021, 12:44:32 PM9/24/21
to em...@googlegroups.com
As long as you ran ctf estimation before particle extraction, the "ctf" key should be saved in the header of each 2d particle. Before particle extraction, check if ctf info exists for each tomogram by looking at the defocus column in e2tomo_eval. You can also check whether the ctf key exist in the 2d particles by clicking "Info" in e2display browser when selecting a 2d particle stack or set, and look at the header of each particle. If the ctf key does not exist in the header of particles, you may need to rerun ctf estimation and particle extraction. 

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/BY5PR02MB6657FCFFD05D5FE9BDF8B068DFA49%40BY5PR02MB6657.namprd02.prod.outlook.com.

Maia Azubel

unread,
Sep 24, 2021, 1:06:24 PM9/24/21
to em...@googlegroups.com
ctf was run before particle extraction using EMAN2.91
  • ctf info exists for each tomogram, there is a value for the defocus column in e2tomo_eval
  • checking the ctf key via "Info" in e2display
  • (example)
  • ctf                   EMAN2Ctf().from_dict({"defocus":1.56756,"bfactor":0.0,"ampcont":98.48,"apix":1.340,"voltage":300.00,"cs":2.70})
Any suggestion?
Thanks,
Maia

From: em...@googlegroups.com <em...@googlegroups.com> on behalf of Muyuan Chen <g5v...@gmail.com>
Sent: Friday, September 24, 2021 9:43 AM

Muyuan Chen

unread,
Sep 24, 2021, 1:14:10 PM9/24/21
to em...@googlegroups.com
The error can happen when there are only a few particles that do not have ctf information for some reason. It is quite hard to tell why, since strange things can happen during file transfer (one corrupted file for example). However I don't think it matters much. The last iteration ('d') does defocus refinement, while it functions correctly, it rarely cause significant improvement to the resolution of the structure. So skipping it won't really hurt. If you want to dig into the issue, you can also just write a small script to loop over all particles in the set and check if the ctf information exist for each particle. 

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/BY5PR02MB66570AFCAE2D6DA8EDBE689FDFA49%40BY5PR02MB6657.namprd02.prod.outlook.com.

Maia Azubel

unread,
Sep 24, 2021, 1:16:55 PM9/24/21
to em...@googlegroups.com
Thanks, as always for your prompt response and help!
Maia
Sent: Friday, September 24, 2021 10:13 AM

Ludtke, Steven J.

unread,
Sep 24, 2021, 1:18:47 PM9/24/21
to em...@googlegroups.com
e2iminfo.py -C file.hdf ...

will check all each image in each listed file to insure that it can be read and has CTF parameters. 

Corruption during file transfer really should not be "a thing". Modern file transfer mechanisms should either produce identical files bit-for-bit or an error message. If you get an error and ignore it, well, that's a problem, of course. The only existing "problem" tool would be using ftp/sftp in text mode, but it's been years since I've heard of someone doing that or having a problem with it....

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine 
Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem)
Academic Director, CryoEM Core                                        (cryoem.bcm.edu)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr)




Maia Azubel

unread,
Sep 24, 2021, 1:29:14 PM9/24/21
to em...@googlegroups.com
I transferred using rsync -av and didn't get any error message 
Is file.hdf the tomogram?
If yes, I get the following
e2iminfo.py -C tomograms/aligned_105k_CHR_Ab_AuL_vpp_tomo003__bin4.hdf
tomograms/aligned_105k_CHR_Ab_AuL_vpp_tomo003__bin4.hdf 1 images in HDF5 format (swap) 1024 x 1024 x 312
Error with CTF on  tomograms/aligned_105k_CHR_Ab_AuL_vpp_tomo003__bin4.hdf 0
representing 0 particles

Maia
From: em...@googlegroups.com <em...@googlegroups.com> on behalf of Ludtke, Steven J. <slu...@bcm.edu>
Sent: Friday, September 24, 2021 10:18 AM

Ludtke, Steven J.

unread,
Sep 24, 2021, 1:32:05 PM9/24/21
to em...@googlegroups.com
No, file.hdf would be 2-D or 3-D particle files. You could give a .lst file as well.

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine 
Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem)
Academic Director, CryoEM Core                                        (cryoem.bcm.edu)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr)



Maia Azubel

unread,
Sep 24, 2021, 1:33:47 PM9/24/21
to em...@googlegroups.com
Also, error
Here's when I use .lst
...
Error with CTF on  sets/ncp.lst 59
Error with CTF on  sets/ncp.lst 60
Error with CTF on  sets/ncp.lst 61
Error with CTF on  sets/ncp.lst 62
Error with CTF on  sets/ncp.lst 63
Error with CTF on  sets/ncp.lst 64
...
From: em...@googlegroups.com <em...@googlegroups.com> on behalf of Ludtke, Steven J. <slu...@bcm.edu>
Sent: Friday, September 24, 2021 10:31 AM

Maia Azubel

unread,
Sep 24, 2021, 1:39:00 PM9/24/21
to em...@googlegroups.com
And if I check "Info" for the .lst on e2display (before I checked 2d particles)
there is no ctf key.

Neither on 3d particles.
Sent: Friday, September 24, 2021 10:33 AM

Ludtke, Steven J.

unread,
Sep 24, 2021, 1:39:45 PM9/24/21
to em...@googlegroups.com
Well, clearly you don't have CTF info in the header of some of the particles. You have ...'s does it start with the first particle in the file and go through the entire set?  Did you remember to do the CTF determination step?

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine 
Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem)
Academic Director, CryoEM Core                                        (cryoem.bcm.edu)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr)



Maia Azubel

unread,
Sep 24, 2021, 1:44:49 PM9/24/21
to em...@googlegroups.com
Which file are you asking if it starts with the first particle in the file and goes through the entire set?

As for ctf
I run ctf estimation on one tiltseries to check the hand, and the run it for all tiltseries, before I transferred.

Maia
From: em...@googlegroups.com <em...@googlegroups.com> on behalf of Ludtke, Steven J. <slu...@bcm.edu>
Sent: Friday, September 24, 2021 10:39 AM

Maia Azubel

unread,
Sep 24, 2021, 1:48:03 PM9/24/21
to em...@googlegroups.com
It'd be easy enough to run ctf again.

I'll have to then re-extract the particles, correct?

Maia
Sent: Friday, September 24, 2021 10:44 AM

Ludtke, Steven J.

unread,
Sep 24, 2021, 1:48:39 PM9/24/21
to em...@googlegroups.com
Did you do the CTF estimation before extracting the particles? If you do it after, then the particles clearly won't have CTF...

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine 
Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem)
Academic Director, CryoEM Core                                        (cryoem.bcm.edu)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr)



Maia Azubel

unread,
Sep 24, 2021, 1:51:14 PM9/24/21
to em...@googlegroups.com
I did.
I did the reconstruction of the tomograms and the ctf estimation with previous versions, then transferred and extracted the particles using EMAN2.99

Maia
From: em...@googlegroups.com <em...@googlegroups.com> on behalf of Ludtke, Steven J. <slu...@bcm.edu>
Sent: Friday, September 24, 2021 10:48 AM

Ludtke, Steven J.

unread,
Sep 24, 2021, 2:25:39 PM9/24/21
to em...@googlegroups.com
The only thing that comes to mind is that possibly when you ran the CTF the second time after checking the hand, you might have still had the handedness option enabled?  Anyway, rerunning the CTF and re-extracting the particles, then doing the e2iminfo test again would be my suggestion.

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine 
Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem)
Academic Director, CryoEM Core                                        (cryoem.bcm.edu)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr)



Maia Azubel

unread,
Sep 24, 2021, 2:27:17 PM9/24/21
to em...@googlegroups.com
Thanks! That's what I'm doing. 
I can update when it's done.
Maia
From: em...@googlegroups.com <em...@googlegroups.com> on behalf of Ludtke, Steven J. <slu...@bcm.edu>
Sent: Friday, September 24, 2021 11:25 AM

Maia Azubel

unread,
Sep 24, 2021, 3:32:56 PM9/24/21
to em...@googlegroups.com
Ran
e2spt_tomoctf.py --alltiltseries --dfrange=0.5,2.0,0.02 --psrange=60,120,2 --tilesize=256 --voltage=300 --cs=2.7 --nref=15 --stepx=20 --stepy=40 --threads=1
Still running
e2spt_extract.py --boxsz_unbin=180 --label=ncp --threads=72 --maxtilt=100 --padtwod=2.0 --alltomograms --shrink=1.0 --tltkeep=1.0 --rmbeadthr=-1.0 --curves=-1 --curves_overlap=0.5 --compressbits=8
Attaching screen shot of one example of info for 2d particles and particles3d.
Because it's still running, I haven't built the set yet. But if I
e2iminfo.py -C particles/aligned_105k_CHR_Ab_AuL_vpp_tomo003__ncp.hdf
particles/aligned_105k_CHR_Ab_AuL_vpp_tomo003__ncp.hdf 76000 images in HDF5 format (swap) 360 x 360
Error with CTF on  particles/aligned_105k_CHR_Ab_AuL_vpp_tomo003__ncp.hdf 0
Error with CTF on  particles/aligned_105k_CHR_Ab_AuL_vpp_tomo003__ncp.hdf 1
Error with CTF on  particles/aligned_105k_CHR_Ab_AuL_vpp_tomo003__ncp.hdf 2
Error with CTF on  particles/aligned_105k_CHR_Ab_AuL_vpp_tomo003__ncp.hdf 3...



Sent: Friday, September 24, 2021 11:27 AM
Screenshot info 2d particles.png
Screenshot info paticles3d.png

Muyuan Chen

unread,
Sep 24, 2021, 3:40:29 PM9/24/21
to em...@googlegroups.com
CTF for 2d particles from tilt series does not have individual SNR curve, which may cause the e2iminfo.py -C to return an error. This may not be the best way to check CTF information for those particles...
Also run particle extraction only after the CTF estimation finishes. In theory it should not be a big problem since particle extraction is much slower than CTF estimation, but things might still go wrong.

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/BY5PR02MB66574E1270BBB9DFDDE3E0E8DFA49%40BY5PR02MB6657.namprd02.prod.outlook.com.

Maia Azubel

unread,
Sep 24, 2021, 3:47:42 PM9/24/21
to em...@googlegroups.com
CTF estimation was finished before I started with extraction. 
Does the same apply for 3d particles?
because for
e2iminfo.py -C particles3d/aligned_105k_CHR_Ab_AuL_vpp_tomo003__ncp.hdf
I get the same error message.

Maia
Sent: Friday, September 24, 2021 12:39 PM

Muyuan Chen

unread,
Sep 24, 2021, 3:51:53 PM9/24/21
to em...@googlegroups.com
3d particles should not have ctf information (since they come from 2d particles with different defocus). I guess you can make it work by commenting out line 52 in e2iminfo.py (snr=ctf.snr[0]) and run on the 2d particles..

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/BY5PR02MB6657F8A9631176E5B272C770DFA49%40BY5PR02MB6657.namprd02.prod.outlook.com.

Ludtke, Steven J.

unread,
Sep 24, 2021, 4:49:20 PM9/24/21
to em...@googlegroups.com
Yes, of course, clearly it was a problem with the 2-D, not 3-D particles. No need to edit anything, though. If you need to check for the presence of all readable images in a stack just run e2iminfo.py with the -as option.


--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine 
Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem)
Academic Director, CryoEM Core                                        (cryoem.bcm.edu)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr)



Maia Azubel

unread,
Sep 24, 2021, 7:11:57 PM9/24/21
to em...@googlegroups.com
Thanks so much!!

with e2iminfo.py -as particles/aligned_105k_CHR_Ab_AuL_vpp_tomo003__ncp.hdf
I get
0. apix=1.34  min=-3.268     max=3.078      mean=-0.004594  sigma=0.733     skewness=-0.01057  kurtosis=0.4456    moment_inertia=6.949e+08 radius_gyration=    137.4    defocus=1.69   B=0
1. apix=1.34  min=-3.183     max=3.26       mean=-0.01113   sigma=0.7363    skewness=-0.002676 kurtosis=0.5383    moment_inertia= 6.73e+08 radius_gyration=    136.1    defocus=1.47   B=0
2. apix=1.34  min=-3.232     max=3.305      mean=-0.01803   sigma=0.7401    skewness=-0.0294   kurtosis=0.5874    moment_inertia=6.617e+08 radius_gyration=    135.6    defocus=1.35   B=0
....

Does the appearance of the defocus value indicate that is correct?
If I were to run again defocus refinement (which I understand that might not be needed) and get the same error (i.e. ...RuntimeError: NotExistingObjectException at /data/maia2/eman2/libEM/emdata_metadata.cpp:1142: error with 'ctf': 'The requested key does not exist' caught ...) would my option then be, as Muyuan suggested, write a script to see if there is for some reason information for some particles missing?
Maia
From: em...@googlegroups.com <em...@googlegroups.com> on behalf of Ludtke, Steven J. <slu...@bcm.edu>
Sent: Friday, September 24, 2021 1:48 PM

Steve Ludtke

unread,
Sep 27, 2021, 10:00:58 AM9/27/21
to em...@googlegroups.com
Hi Maia,
sorry for the delay. Yes, having a defocus indicates CTF info in the header. The -C option does basically the same thing but without reporting for particles which are ok. You should be able to use -C with particles/ but not with particles3d/

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine 
Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem)
Academic Director, CryoEM Core                                        (cryoem.bcm.edu)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr)


Reply all
Reply to author
Forward
0 new messages