Importing coordinates automatically

[Jacob Croft]

Hello,

I have particle coordinates that were picked in imod that I would like to import into EMAN2.  Currently, the only way I know how to do this is to open boxer for each tomogram, click "Read box coordinates" then open the coordinate file and click save.  It's a bit tedious doing this for every tomogram, so I was wondering if there is a way to perform these actions from the command line so I could write a script to import each coordinate file?

Thanks,

Jake Croft

University of Washington (Kelly Lee Lab)

[Steve Ludtke]

Hi Jacob,

so, there are two parts to the answer. First, there is some reasoning behind not having that functionality:

While it is possible to use the 'old' subtomogram averaging strategy from 5-10 years ago in EMAN2, and align/average 3-D subtomograms from IMOD, the resolution of that approach is likely to max out at 18-25 Å resolution, even if you did IMOD CTF correction. With the new strategy (and good data) subnanometer resolution can readily be achieved. In this new scheme in EMAN2, particles are individually reconstructed from the subtilts (a per particle tilt series) over and over again, and the main purpose of the full tomogram is getting box locations so the subtilt series can be extracted (from the tilt series).

All of that is to say that there is virtually no point in trying to import tomograms from IMOD. If you are going to use the old resolution-limited EMAN2 approach, then it will generally take at most a few hundred particles to achieve maximum resolution so, you normally only need a few tomograms, and can do it manually.  If we make that step trivial, then we will run into people who want to do a comparative refinement in EMAN2, and they will use the outdated tools because they are easier, and they will get lousy results, and decide EMAN2 doesn't work well.

But wait, what if you are redoing the tilt series alignment/reconstruction in EMAN2? Can't you import the box locations then, so you get the same set of particles you used in IMOD?  Unfortunately, there is no absolute 3-D geometry defined by the tilt series. Different software will produce tomograms in different overall orientations. These differences can be subtle, or they can be quite significant. So, to do this, you don't just need to read the boxes from IMOD, you also need to realign the boxes to the EMAN2 tomogram. We haven't put an automated tool together to do that at this point.

Second:

if you want to ignore all of that background and do the import anyway, and aren't afraid of a little python, it wouldn't take much effort to write a little script to import .box files into 3-D box locations. If you look at e2import.py, it has an "import boxes" method which is designed to read 2-D box locations for single particle analysis. This starts at line 222, and is only a few lines of code. The 3-D box information is stored in info/*.json as described:

http://eman2.org/Eman2InfoMetadata

This is very similar to 2-D boxes, but the name is boxes_3d instead of boxes, and there are obviously 3 coordinates, not 2. Just keep in mind that importing box locations from IMOD and doing tilt series reconstructions in EMAN2 will produce misaligned boxes without further action.

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine 
Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem)
Academic Director, CryoEM Core                                        (cryoem.bcm.edu)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr)

--
--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2

---
You received this message because you are subscribed to the Google Groups "EMAN2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/c28ccf15-aa91-474d-9795-ec5193f01191n%40googlegroups.com.

[Jacob Croft]

Dear Steven,

Thanks for the detailed response!  To clarify - I'm aware of the requirement of using tomograms reconstructed within EMAN2 to perform subtilt refinement, which I think is a great approach and is specifically why I'm using EMAN2.  However, I find it easier in some cases to reconstruct tomograms in EMAN2 and then pick the particles from those tomograms in IMOD rather than using boxer and import the coordinates back into EMAN2 to continue (Since IMOD has tools like "meshinit" and "spikeinit" for spacing particle coordinates along membranes, and also I find it easier to distinguish particles in lower defocus tomograms while manual picking using IMODs slicer interface rather than boxer).  So I don't think this should cause any problems, because IMOD seems to use the same coordinate system as EMAN2 and the particle locations look fine when I have re-imported them into EMAN2.

Thank you for pointing me to the import boxes method, this should be what I'm looking for.  I'll get back to you if I have any more confusion about implementing it.

Best,

Jake

You received this message because you are subscribed to a topic in the Google Groups "EMAN2" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/eman2/P1sxOta620w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to eman2+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/DB788ABD-8B38-404A-8D53-F6D5075D6302%40gmail.com.

[Ludtke, Steven J.]

Ok, sounds good.

Out of curiousity, could you show some screenshots showing how it's easier to distinguish particles in slicer?  Always room for improvement...

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine 

Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem)

Academic Director, CryoEM Core                                        (cryoem.bcm.edu)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr)

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/CAOa2YHDA%3DnChjATVfRejKaifNYff17f1HA%3D7xysN%2BsDutidBTg%40mail.gmail.com.

[Jacob Croft]

Sure thing, I've included a screenshot of an example of an HIV-1 virion displaying Env proteins on its surface.  I added a few white arrowheads to point out Env particles that I think are easier to distinguish in IMOD.  It's generally not that I can't see them in boxer, it just seems to be a little harder/takes more time to pick them out.  I think the main thing that helps is being able to control the slice thickness in slicer (Please let me know if this can be done in boxer - maybe I missed it but couldn't find this option when I looked in eman 2.91). 

If I may suggest another thing to consider for improvement of the boxer interface (and sorry if somehow I missed that these things are possible), I like that in slicer you can press the page up and page down keys to scroll through the tomogram rather than having to use the slider on the side, as you have to move the mouse back and forth less and have more control over how many slices you go up and down.  Is there a hotkey for scrolling up and down in boxer?

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/EBFDA669-7C1D-4EDE-AD70-D657A91AA5EA%40bcm.edu.

[Ludtke, Steven J.]

Hi Jacob,

In the lower left corner of the boxer main window, there is a thickness widget. This is expressed in +- layers, so 1 corresponds to summing 3 voxels thick, 2 -> 5, etc. 

If the last thing you selected was the slider, ie - if you move the slider with the mouse, then as long as it has focus you can use up/down arrow and pg up/down to move with the KB instead, but it doesn't seem to carry through if you have changed the mouse focus to the main window. That's something we could add pretty easily though. Will have to add that to the TODO list.

--------------------------------------------------------------------------------------
Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine 

Charles C. Bell Jr., Professor of Structural Biology
Dept. of Biochemistry and Molecular Biology                      (www.bcm.edu/biochem)

Academic Director, CryoEM Core                                        (cryoem.bcm.edu)
Co-Director CIBR Center                                    (www.bcm.edu/research/cibr)

On Nov 5, 2021, at 1:12 AM, Jacob Croft <jac...@uw.edu> wrote:

Sure thing, I've included a screenshot of an example of an HIV-1 virion displaying Env proteins on its surface.  I added a few white arrowheads to point out Env particles that I think are easier to distinguish in IMOD.  It's generally not that I can't see them in boxer, it just seems to be a little harder/takes more time to pick them out.  I think the main thing that helps is being able to control the slice thickness in slicer (Please let me know if this can be done in boxer - maybe I missed it but couldn't find this option when I looked in eman 2.91). 

If I may suggest another thing to consider for improvement of the boxer interface (and sorry if somehow I missed that these things are possible), I like that in slicer you can press the page up and page down keys to scroll through the tomogram rather than having to use the slider on the side, as you have to move the mouse back and forth less and have more control over how many slices you go up and down.  Is there a hotkey for scrolling up and down in boxer?

<Manual_picking.png>

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/CAOa2YHAX07poGa%2BuORqSquCMhdGqUeOFVj61nm41%3DNerK%2BkTew%40mail.gmail.com.

[Muyuan Chen]

you can scroll up and down through z using the "`" and "1" buttons on the keyboard (the two buttons below Esc). I think holding shift and using the scroll of the mouse on individual panels also moves the plane of that panel.

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/CAOa2YHAX07poGa%2BuORqSquCMhdGqUeOFVj61nm41%3DNerK%2BkTew%40mail.gmail.com.

[Jacob Croft]

Hi Steven and Muyuan,

Thank you for the tips.  I will try these out!

Best,

Jake

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/CAO_0xqHFCiprWoBJX1_acoGcrtX3udV66GKJXtMmV6DF7cGQvg%40mail.gmail.com.