TIFF-LZW | super-resolution

[Julien Maufront]

Hello everyone,

I have a question about TIFF-LZW format and super-resolution :

First of all is it safe to process TIFF-LZW format in super-resolution mode or we should favorize EER format in this case (even if LZW-EER format generates much smaller files compared to EER) ?

Actually my situation is the following :

We generated LZW-TIFF movies in counted mode with a falcon 4i camera.

These movies are made of 8k x 8k images (super-resolution).

Meanwhile we generated a gain reference image whom dimension is of 4kx4k (which is logical since it follows the true dimension of the camera).

The problem here is I can not directly apply the gain to my images since the dimensions are not consistent.

Meanwhile I saw that within Relion or Cryosparc, if you import eer files and you want to process it in super-resolution mode the softwares are able to upsample the gain reference image (initially 4k x 4k) to proceed. Thus, I tried to find such a function usable out of Relion or Cryosparc but in vain. I could neither find any established procedure to do so.

So I decided to try with EMAN using the following command :

e2proc2d.py --clip=8192,8192 --scale 2 --outtype=mrc GainReference.gain GainReference_8k.mrc

(the input gain image is 4096 x 4096).

Then i used MotionCor2 to aligned the frames while correcting for the gain and it seems to work since the grey gradient and dead pixels lines disappeared after this procedure (See attached images : on the right side a white line disappeared after gain correction).

Here i wanted to double check with experts that this procedure is valid and if you do it differently or you don't use TIFF-LZW for supper-resolution ?

Thanks in advance for your help and advices.

Julien Maufront

[Steve Ludtke]

Haven’t tried this ourselves, but what you are describing sounds reasonable. LZW compression is lossless, so I don’t see any issues with it.

--
--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2

---
You received this message because you are subscribed to the Google Groups "EMAN2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/11783af9-a3af-4295-941e-c25e7f69b23dn%40googlegroups.com.
<Gain-correction_8k_ZOOM.png><noGain-correction_8k_ZOOM.png>

[Tapu Shaikh]

We tested under conditions which were not quite the same:
-- We weren't using super-resolution; we stayed at 4k.
-- We didn't compress the TIFFs.

We grouped the EERs frames using the same settings that we use in MotionCor.  Under those conditions, we got to the same resolution on apoferritin (1.7-something) whether we used the EERs or the TIFFs.

Without compression, the TIFF stacks were smaller than the EERs by a a factor of about 4.5, but your mileage may vary.  I didn't think about compressing the TIFFs; I'm curious how fast the compression-decompression is. 

Hope this helps.

-Tapu Shaikh

[Xiong Pi]

Hello Everyone,

I am trying to do template picking in EMAN2 tomo on our cluster but there is always an error, which said 

zero-size array to reduction operation maximum which has no identity. Does anyone know what the problem is? Thank you so much for your help and advices.

Best,

Xiong

Tapu Shaikh <taput...@gmail.com> 于2023年11月10日周五 07:44写道：

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/868fd68d-d3e6-4959-9590-fddfcf75a25d%40gmail.com.

[Ludtke, Steven J.]

Hi Xiong,

Need the full output of e2version.py from your cluster to help identify what the problem might be. 

Personally I'd suggest using the deep-learning based picker instead of the template matching. This particular implementation of template matching is (intentionally) very heavily downsampled, so it is really more of a "blob finder" than a true template based search.

---

Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine
Charles C. Bell Jr., Professor of Structural Biology        Dept. of Biochemistry 
Deputy Director, Advanced Technology Cores                  and Molecular Pharmacology
Academic Director, CryoEM Core
Co-Director CIBR Center

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/CAB453QKkYXWr8YNmgtthaZiqQ9VtdEY3QfJCrzpTR%3Dc8OBN%2BjA%40mail.gmail.com.
<Screen Shot 2023-11-10 at 11.19.08.png>

[Xiong Pi]

Hi Steven,

Thank you so much for your quick response. I also tried convnet based auto-boxing and the train procedure worked successfully, but when I selected a tilt series and click apply, The GUI always crashed. Do you have any advices? 

Best,

Xiong

Ludtke, Steven J. <slu...@bcm.edu> 于2023年11月12日周日 15:50写道：

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/E534F7A8-00AF-4C92-8028-710B211D3408%40bcm.edu.

[Xiong Pi]

Hi Steven,

Thank you so much for your quick response. I also tried convnet based auto-boxing and the train procedure worked successfully, but when I selected a tilt series and clicked apply, The GUI always crashed. Do you have any advice? 

Best,

Xiong

Xiong Pi <xing...@gmail.com> 于2023年11月13日周一 10:44写道：

[Steve Ludtke]

You selected a tomogram, not a tilt-series, right?

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/CAB453QKzjgC4ZN13Uc%2B%2BRRbBkHTMSu%2BrrPhAoMGd_g7ca-vDsQ%40mail.gmail.com.
<Screen Shot 2023-11-13 at 10.37.50.png>

[Julien Maufront]

Hi Steven and Tapu,

Thank you both for confirming to me the use of TIFF should not be a limit for the analysis and I may be able to upsample my gain reference image with eman commands.

PS : Tapu, I suspect that your TIFF files are actually LZW-TIFF files since the factor mentionned here between EER files and TIFF files correspond to what is expected with such compression (see https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6597182/, Table 1). There is no additional procedure to be used to handle this file with other software. As a test, you can try to convert it from tif to mrc with imod function tif2mrc, you may see a drastic increase of the file size.

Best regards

[Xiong Pi]

Yes, I selected a tomogram.

Steve Ludtke <slud...@gmail.com> 于2023年11月13日周一 10:56写道：

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/09DD9A10-6E11-4364-BD6F-99AAF983BF15%40gmail.com.

[Steve Ludtke]

I only asked because you said you selected a tilt-series :^). If the GUI is crashing, I’d need to see the error it produced in the crash to figure anything out. “it crashed” isn’t very useful in trying to diagnose the problem.  In general, if something is crashing, for us to investigate, we need the full output of e2version.py on the machine where the crash occurs, and the full set of error messages produced in the crash (generally on the console where it was launched).

cheers

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/CAB453QL1O3mD3eD1i9yS7RRDjvcaqW7-TSnhKvhRz%3D0zpUL3Dg%40mail.gmail.com.