To extract particles from self reconstructed tomograms

[王渊浩]

Hi Steven,

I found that with eman2, I could locate the particle with the refered sample using my own reconstructed tomograms. But when I was tring to extract the particles using eman2, failed, error read 'ttparams=np.array(js["tlt_params"])'. It seems this value exists in the json file. How to get this file? How should it look? I tried to  generate using eman2 and get a tlt_params with 5 rows, and the 4-th row seems to be the tilt angle, what are other row refer?

Regards,

[Steve Ludtke]

Hi, you're going to need to provide more details. It is impossible to follow what you did and exactly what error you got.

1) What is the full output of e2version.py

2) Which program were you using to select particles?

3) What do you mean "my own reconstructed tomograms" ? Tomograms reconstructed in another program?

4) What is the complete error message you got?

--
--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2

---
You received this message because you are subscribed to the Google Groups "EMAN2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/c8ea3b2f-4d99-4b7b-83f0-e14254cd7358n%40googlegroups.com.

[王渊浩]

Hi Steven,

Thank you for your kind reply. 

1. The eman version is 2.9 final ( GITHUB: 2021-02-13 20:31 - commit: 006911ea3 )

2. I am using the reference-based boxing with a known map.

3. Yes the tomograms was reconstructed in another program.

4. The complete message shows like this:

Reading from tomograms/*denoised.hdf...

Traceback (most recent call last):

  File "*/anaconda3/envs/eman2/bin/e2spt_extract.py", line 978, in <module>

    main()

  File "*/anaconda3/envs/eman2/bin/e2spt_extract.py", line 144, in main

    do_extraction(a, options)

  File "*/anaconda3/envs/eman2/bin/e2spt_extract.py", line 166, in do_extraction

    ttparams=np.array(js["tlt_params"])

  File "*/anaconda3/envs/eman2/lib/python3.7/site-packages/EMAN2jsondb.py", line 867, in get

    raise KeyError(key)

KeyError: 'tlt_params'

It seems there is a missing in the json files.

[Steve Ludtke]

The extraction process extracts both 3-D particles as well as per-particle tilt-series, so it cannot work with tomograms reconstructed in other software. It isn't just metadata that it's missing, it is missing the tilt series, and alignment parameters as well. This was just the first of many errors which would have occurred. 

While there are mechanisms you can use to just extract the 3-D particles alone, then perform the first stage of subtomogram averaging, given the different assumptions made by different software, even with this, results may be suboptimal. Further, if you get a decent map doing that, you would then be unable to proceed with the PPPT refinement which is necessary to achieve good resolution. If you decided then to reconstruct the tomograms in EMAN2, to do PPPT, the box locations you determined would be wrong since the reconstructed tomogram in EMAN2 would almost certainly be in a slightly different overall orientation than the one done in other software.

That is to say, we strongly suggest starting with the tilt-series. The entire alignment/reconstruction process in EMAN2 normally only takes 5-10 minutes per tomogram (depending on your computer).

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/0bd32768-2887-4c67-843e-f28021c3fc02n%40googlegroups.com.

[Steve Ludtke]

Muyuan, is there still a functional mechanism for extracting just the 3-D particles for situations like this?

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/0bd32768-2887-4c67-843e-f28021c3fc02n%40googlegroups.com.

[Muyuan Chen]

To extract from tomograms, simply click 'save' from e2spt_boxer window. The subtomogram refinement will still work, but the subtilt and a few other functions wont.

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/304D200A-8FC7-4D5E-AFE4-543C2F0E8CFB%40gmail.com.

[王渊浩]

Hi Steven,

Thank you for this information. In fact, I am considering to add the alignment and tilt series in to the input of the particle extraction, but unfortunately have no much idea how the tlt_params are formualated in the json file, or what else are needed for the resumed process. I was trying to evaluate the reconstructions, but not do the whole procedure. So maybe there is a little difference.

Regards,

Yuanhao

[王渊浩]

Thank you so much, I will try.

Regards,

[Colin Gauvin]

Hi,

I realize this is an old thread, but I had a question re: this topic.

If I save the particles from the manual boxer window, will that save ctf-corrected particles?

Thanks

[MuyuanChen]

No. CTF correction only happens when you go through the extract particle process. 

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/ad6dbb18-0c2e-45ba-8dc9-3d1d329a0a4fn%40googlegroups.com.

[Ludtke, Steven J.]

Muyuan,

What about the new method you added which would produce ctf corrected whole tomograms?

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine 

Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem)

Academic Director, CryoEM Core                                        (cryoem.bcm.edu)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr)

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/8165420A-5B20-4AC9-966D-DCB6EC66B3ED%40gmail.com.

[MuyuanChen]

Yes, Steve is correct. I only recalled this after I clicked send on the first email, and it took a bit long to write this one.

Now I think about this, I remember that I actually implemented something for that lately. So if you have a new enough EMAN2 (after 2021-11-30 from Github), it can actually make CTF corrected tomograms. If you run tomogram reconstruction once, then run CTF estimation, then run the tomogram reconstruction again, with —load and —niter=0 and —bytile, it will skip the alignment, redo the reconstruction, and take the existing CTF information to phase flip the tilt series by tile to reconstruct the tomogram. The reconstruction in the first run is actually not necessary, and you can add —dryrun for the first tomogram reconstruction to only write metadata without writing the tomogram. 

Then you should be able to save particle from boxer. Saying that, this seems to be more complicated than (and probably not as good as) just extract particles… Personally I don’t really see a good reason to go through this protocol. I don’t think the ability to CTF corrected full tomogram is that useful either, even though I thought there was a good reason at the time I wrote it…

Muyuan

[Colin Gauvin]

Thanks for the quick response.

Is that because the average defocus of the tomogram varies so much, that without a 3d model to align to, there's no way to CTF correct as small portion of it?

My plan was to extract subtomograms of viruses from a tomogram, use the IMOD tool "seedSpikes" to place points on the surface, and search around the surface for surface proteins. Is there a recommended workflow to approximate that in Eman2?

[MuyuanChen]

No, even for the full tomogram CTF correction, it is still done in a per-tilt-per-tile bases, instead of just using the average defocus. The difference is when you do particle extraction it also consider the depth of particles in ice (location along z). The defocus estimation seems quite accurate in my tests. It won’t change much even when you refine defocus with a good 3d model. 

I am not familiar with IMOD functionalities. I played with spike proteins of a few types of viruses, and I can get it to work automatically with just the CNN based particle picking. (https://blake.bcm.edu/emanwiki/EMAN2/e2tomo_more#Automated_particle_selection) So I have never wrote a specialized algorithm to search for them...

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/758fc3ed-e520-456c-a23c-6a9c2b9f09fan%40googlegroups.com.

[Colin Gauvin]

OK, I will start playing around with that. Thank you very much.