Extract particles from multilevel annotations

[huang....@gmail.com]

Hello,

I merged several segmentations of the same tomogram together using mergetomoseg.py to 'compete' the neural networks on areas where regions were misassigned. I was wondering if it is possible to extract particles from a particular level in the mask for subtomogram averaging. 

Thank you!

Judy Huang

PhD Student 

University of Massachusetts Medical School

[MuyuanChen]

Somehow I never implemented this…

Although you can do 

e2proc3d.py a.hdf b.hdf —process threshold.binaryrange:low=x:high=y 

to take a certain level out, then extract particles from that.

--
--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2

---
You received this message because you are subscribed to the Google Groups "EMAN2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/57e632a3-77d4-4e26-a663-09a99b7a1851n%40googlegroups.com.

[Judy Huang]

Thank you so much!

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/65B0AC9D-E109-4241-A160-5D7F86A5320A%40gmail.com.

[huang....@gmail.com]

Hi Muyuan, 

I have a follow up question on this topic regarding inconsistent particle extractions. A couple weeks ago, I extracted particles from one single annotation and it returned about 2000 boxes, which was in the range of what I expected (the particles of interest are influenza glycoproteins). I then decided to merge multiple annotations together and extract several levels using e2proc3d.py. While I was able to successfully merge levels and extract particles, now I am getting in the range of 200 thousand boxes, which is quite baffling. I went back and tried extracting using single annotations with the exact segmentation and tomogram I was using before on the exact same workstation, and I got more than 150000 boxes. I am wondering what could have caused this 75-fold increase in the number of particles extracted. 

I'm working with a binary installation on a Linux workstation. e2version.py returns: 

EMAN 2.31 final (GITHUB: 2019-09-30 10:41 - commit: 194df95 )

Your EMAN2 is running on: Linux-3.10.0-1127.13.1.el7.x86_64-x86_64-with-centos-7.8.2003-Core 3.10.0-1127.13.1.el7.x86_64

Your Python version is: 2.7.14

This was the command I used for extracting: 

e2spt_extractfromseg.py segmentations/s41_bin2__bin4_preproc_good_glyco_seg.hdf tomograms/s41_bin2__bin4_preproc.hdf --thresh=1.0 --featurename=glyco --boxsz=32 --random=-1 --massthresh=20.0 --edge=4 --sort

156842 boxes found..

I've attached screenshots that correspond to the segmentation output used to extract particles (visualized in UCSF Chimera), as well as the particle extracted when I attempted last time. I tried to see which particles were chosen for my current extractions, but the massive number of boxes caused my workstation to freeze.

Hopefully this problem isn't too strange!

Thank you for all your help.

Judy

[Muyuan Chen]

I suspect there are some tiny debris of a few voxels in the segmentation. Although in theory the massthresh option should get rid of that so I am not quite sure why this is happening. Maybe lowpass filtering the annotation a bit and pick a new threshold will help. 

Just do "e2proc3d.py x_seg.hdf x_seglp.hdf --process filter.lowpass.gauss:cutoff_abs=.3" and see if it works better.

p.s. I have recently updated a new program specifically for particle picking in tomograms. This is using the same concept but is probably more convenient than the segmentation - extract approach. It can be found in later versions of EMAN. Some documentation here

https://blake.bcm.edu/emanwiki/EMAN2/e2tomo_more#Automated_particle_selection

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/fe2a2d6d-ed48-4cc2-8249-5b4330c2fb69n%40googlegroups.com.

[huang....@gmail.com]

Hi Muyuan,

Unfortunately that didn't work. I'm honestly really confused to why it worked last time but not now! I'll try the new program out tonight, one of the reasons I wanted to stick with the segmentations was that I was able to merge several segmentations for the best results, but hopefully the particle picker would work well also.

Best,

Judy

[Muyuan Chen]

That is quite odd...I am confused as well.  can you share me the segmentation file privately so I may take a look? 

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/6af545b1-b7c5-4e9f-afdf-7f2cec6c6fd4n%40googlegroups.com.

[huang....@gmail.com]

Hi Muyuan,

I figured it out! There may be a bug in the project manager where the first two arguments were switched? So I got the input command of:

e2spt_extractfromseg.py segmentations/s41_bin2__bin4_preproc_good_glyco_seg.hdf tomograms/s41_bin2__bin4_preproc.hdf --args

but 

e2spt_extractfromseg.py tomograms/s41_bin2__bin4_preproc.hdf segmentations/s41_bin2__bin4_preproc_good_glyco_seg.hdf

would have been the right command. Of course extracting particles from the segmentation based on the tomogram would not give you the best results.

Thanks again for your help!

Judy

[huang....@gmail.com]

Do you still want the segmentation file?

[Muyuan Chen]

No. I am glad it works. 

I see... the bug happens when you specify the files in the GUI tab, then switch to command tab and switch back... So programs in the project manager cannot have more than one positional argument I guess... I need to think about how to fix this while maintain backward compatibility...

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/28ac9cf8-1401-40e5-bf64-e65cb52cb8bdn%40googlegroups.com.

[huang....@gmail.com]

Hi Muyuan,

Very sorry to restart this old question....but I do have a follow-up question regarding extracting particles from multilevel annotations. 

I successfully segmented, extracted a single level, and extracted subtomograms from that level for averaging. However, I only realized recently that the particles extracted from multilevel annotations were very different from those extracted from a single annotation. I chose to do the multilevel annotation as there were a lot of off-target particles when only one neural net is applied. Would you mind if I send you a few images privately? 

Best,

Judy

P.S. I have tried the CNN-guided particle picker, but unfortunately it has not worked well for my project. 

[Muyuan Chen]

I suspect there are more small pieces of debris left in the multi level map that does not affect visualization but affect particle picking. It is a bit odd that the new particles picker fails. Yes you are welcome to send me a tomogram or two so I may be able to figure out what’s wrong.

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/6822859b-6a0b-4a4d-ae75-1b6512439045n%40googlegroups.com.