Placing subtomogram averages back in tomogram

[Matthias Vorländer]

Hi all, 

while this is admittedly not directly a Warp/M question, I wondered if anyone knew how to use the relion star file to place copies of the refined subtomogram average map back into the tomogram, preferably in ChimeraX. I managed to apply the shifts alright, but I am struggling to rotate the map properly using the euler angles from relion. 

Thanks in advance.

Matthias 

[Alister Burt]

Hi Matthias,

Are you trying to 

1) generate a tomogram (i.e. one volume file) with the density map for the subtomograms placed in the tomogram

2) apply the shifts/rotations to copies of the reconstruction in Chimera directly? 

3) or rather just a mesh representation of each particle in the correct pose with respect to the tomogram?

For the interpretation of Euler angles - the convention used is a set of ZYZ intrinsic right-handed rotations in a right handed coordinate system. This transforms the reference onto the particle -> you need to invert the matrix if you want the inverse transformation.

Here’s an example of getting the rotation matrix using the eulerangles package in Python

https://gist.github.com/alisterburt/15fda2431760da76ac6a09d2967afc67#file-tomo_vis_napari_demo-py-L36-L38

If you’re looking for (1) you’ll likely have to write something yourself. If you’re looking for (3) there are some useful tools in Dynamo for generating sets of meshes based on reconstructions https://wiki.dynamo.biozentrum.unibas.ch/w/index.php/Walkthrough_on_placement_of_averages_on_table_positions

If you want to get a dynamo table file from your relion star file you could base yourself on this https://github.com/alisterburt/dynamo2m/blob/master/dynamo2m/warp2dynamo.py#L20-L52

The only thing missing from the script above is handling of the shifts in the star file, an example of which can be seen here

https://github.com/alisterburt/dynamo2m/blob/33d76b5ea444e53a4877aa2e342c9f147dee8c99/dynamo2m/relion_star_downgrade.py#L11-L38

Cheers,

Alister

--
You received this message because you are subscribed to the Google Groups "Warp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to warp-em+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/warp-em/82fed9a4-2bc2-42c4-9756-585dbea24797n%40googlegroups.com.

[Matthias Vorländer]

Hi Alister, 

thanks a lot for the reply! What I am aiming to do is) 2) apply the shifts/rotations to copies of the reconstruction in Chimera directly. After your your reply, I tried the following:

import numpy as np

from eulerangles import euler2matrix

from eulerangles import invert_rotation_matrices

#Use euler angles from STAR file as arguments 

eulers = np.array([-35.625440, 162.841315, 87.283656])

rotation_matrix = euler2matrix(eulers,

                               axes='zyz',

                               intrinsic=True,

                               right_handed_rotation=True)

 

inverse_matrix = invert_rotation_matrices(rotation_matrix)

print(inverse_matrix)      

If I am not mistaken, the output should contain the rotation matrix that I need to apply to the subtomogram average to match the particle pose (after applying the shifts). To convert this matrix to axis-angle values that are required for the ChimeraX "turn" command, I used this website https://www.andre-gaschler.com/rotationconverter/, setting the output angle format to Degrees. Finally, I use these values in ChimeraX as the argument for the "turn" command (specifying the model that I turn as the center of rotation). Unfortunately, is is obvious that the transformed map does not fit into the original tomogram density.  

Is there something that I am obviously doing wrong? I am relatively confident that I apply the shifts correctly. 

I will also try to achieve the same with Dynamo.

Thanks a lot for your help,

Best,

Matthias

[Liang Xue]

Hi Matthias, 

Nice to know you are also doing tomography stuff now. We do the mapping using the Matlab TOM toolbox, which is slow but works well. If you have MATLAB running, I can send you the scripts we used to convert the shifts and rotations (TOM format is different from RELION format). 

Best,

Liang

Matthias Vorländer <mvorl...@gmail.com> 于2021年5月27日周四 下午3:46写道：

To view this discussion on the web visit https://groups.google.com/d/msgid/warp-em/d8fef254-a52d-4788-95b6-1d4c757cbed6n%40googlegroups.com.

[Matthias Vorländer]

Hi Liang,

thanks a lot for the reply! We unfortunately don't have a MATLAB license in the lab, but I might get back to you if I really get stuck ;)

Best,

Matthias

[Liang Xue]

Sure. I did persuade Dimitry to write this as an extended function in Warp or M. Anyway, let me know if you might want to try the MATLAB scripts.

Liang

Matthias Vorländer <mvorl...@gmail.com> 于2021年5月27日周四 下午4:30写道：

To view this discussion on the web visit https://groups.google.com/d/msgid/warp-em/2ecee18f-06f1-42c2-99e9-0e8fda2400a2n%40googlegroups.com.

[Alister Burt]

Hi Matthias,

What happens if you use rotation_matrix rather than inverse_matrix? (It’s just the transpose, rotation_matrices are orthogonal) one of these two should be correct but I don’t have much experience with manipulating these things in chimera

Good luck, if you come up with a nice solution I’d love to see it :)

Alister

To view this discussion on the web visit https://groups.google.com/d/msgid/warp-em/d8fef254-a52d-4788-95b6-1d4c757cbed6n%40googlegroups.com.

[Matthias Vorländer]

Hi Alister,

unfortunately the roation_matrix doesn't work either. 

With respect to the dynamo method, do I understand correctly that the shifts that are missing after conversion with warp2dynamo from the table file are only the refined offsets from the original coordinates? In other words, even the table that does not contain values for the shifts (columns 3,4 and 5) should give approximately correct positioning of the map in the tomogram? Because I have followed the walkthrough with a table file that does not contain values for the shits, and the result looked pretty bad (see screenshot, with tomo density in blue and placed maps in grey) and I am wondering where things are going wrong. 

  
Thanks again for all the help,

Best,

Matthias

[Alister Burt]

Hi Matthias,

Yes you’re correct, only the particle shifts should be missing.

I’m not sure exactly where your problems are coming in, the rotation matrices are correct for the particles (I use them to generate Euler angles, reconstructions of the same set of particles in each program give near-identical reconstructions)

It looks like you have some scale discrepancy between the tomogram and your set of meshes, are you sure that everything is on the same scale? (Tomogram/reconstruction/metadata)

One reason particle rotations can become incorrect is if you used the ‘pre-rotate particles option’, see here for details https://groups.google.com/g/warp-em/c/6FdxW9RT-gM

Cheers,

Alister

On 31 May 2021, at 10:52, Matthias Vorländer <mvorl...@gmail.com> wrote:

Hi Alister,

unfortunately the roation_matrix doesn't work either. 

With respect to the dynamo method, do I understand correctly that the shifts that are missing after conversion with warp2dynamo from the table file are only the refined offsets from the original coordinates? In other words, even the table that does not contain values for the shifts (columns 3,4 and 5) should give approximately correct positioning of the map in the tomogram? Because I have followed the walkthrough with a table file that does not contain values for the shits, and the result looked pretty bad (see screenshot, with tomo density in blue and placed maps in grey) and I am wondering where things are going wrong. 

To view this discussion on the web visit https://groups.google.com/d/msgid/warp-em/400d861d-fcac-4882-992c-1c80d7292129n%40googlegroups.com.
<Screenshot 2021-05-31 at 11.50.17.png>

[Pranav Shah]

Hi Matthias, I concur with Alister, could you check the pixel size on
your tomo and your map? try playing with the map size in the coords
dialog box in chimera to get the map to "fit" in the tomo...
Best,
Pranav
--
Pranav Shah
Postdoctoral Research Fellow.

Division of Structural Biology,
Wellcome Trust Centre for Human Genetics,
University of Oxford,
Roosevelt Drive, Oxford OX3 7BN,
UK

> To view this discussion on the web visit https://groups.google.com/d/msgid/warp-em/73429215-4946-4637-AF76-DECE4ADCC03F%40gmail.com.

[Shiwei Victor Zhu]

Hi Matthias,

Here is the way I use imod to map back:

1, make an empty tomogram, use newstack function from IMOD

 newstack -multadd 0,0 aaa.rec sz.rec

2, use clonevolume function from IMOD

clonevolume -at pos.csv  -into sz.rec  average.mrc output.rec

## pos.csv file would have coordinates and Euler angles. You might use Alister script to change the euler angle from Z1YZ3 to Z1XZ3 ; Also you may play the opposite direction: Z3YZ1.

This would work for mapping back.

Shiwei

To view this discussion on the web visit https://groups.google.com/d/msgid/warp-em/73429215-4946-4637-AF76-DECE4ADCC03F%40gmail.com.

[Pranav Shah]

Thanks for sharing Shiwei, that is a nice little 2 liner

To view this discussion on the web visit https://groups.google.com/d/msgid/warp-em/CAFM4spfyZEr8CHuwLFFEm_-zJ4mp0JkhJM97ztgBoxdx1MkG5Q%40mail.gmail.com.

--

[Matthias Vorländer]

Hi everyone, 

Thanks a lot for all the helpful answers! Really appreciate it. I will definitely try Shiwei's solution! Would you be willing to share an example pos.csv file to check the formatting?

Best wishes,

Matthias

[Shiwei Victor Zhu]

Hi Matthias

Here is the format of  .csv file, imaging you are mapping 3 ribosomes back to your tomogram.

1,77,634,149,169.0838019076472,30.93724842575327,-147.5644844095804
2,193,587,149,-30.06121772007089,77.79893513392658,85.45611563866991
3,254,475,148,131.9316408814744,18.71770867616055,29.98927735963991

1st column is the number; the 2nd to 4th columns standing for the coordinate XYZ; the last 3 columns are Euler angles.

Let me know if it works. 

Best,

Shiwei

[Juha Huiskonen]

Hi all,

Seems like a good solution! This is more a question to IMOD list, but does "clonevolume" automatically normalize for overlaps? I.e. areas where more than one sub-volume contributes to?

Best wishes,

Juha

To view this discussion on the web visit https://groups.google.com/d/msgid/warp-em/CAFM4spdDArQ3sN3%3D_s_t4bdfCV7iiYQ-j6YUQrm7UWciOZpQLg%40mail.gmail.com.

[Matthias Vorländer]

Hi everyone, 

after trying Shiweis solution and failing, I wrote to the imod group and would like to share John Heumanns reply here. It looks likes the clonevolume pos.csv file uses yet a different type of rotation angles but this can be converted using scripts supplied with the program PEET. Thanks everyone for your input!

Hi Matthias,

If you install PEET, you'll find it contains an executable relion2MOTL which will take care of converting Euler angles from Relion's convention to PEET's. (There's also a corresponding program MOTL2Relion, and each program also supports conversion to / from arbitrary Euler angle conventions.  Next, you'll have to address the fact that the csv summary file which is used by clonevolume wants z-y-x slicer angles rather than Euler angles as input. For that conversion, you'd use PEET program MOTL2Slicer. So briefly, 

relion2MOTL followed by MOTL2Slicer should give you what you need. 

Regards,

-jh-