Importing SerialEM tilt-series in HDF format

[Colin Gauvin]

Hi,

First - I'm not sure if this is an EMAN2 problem, or a SerialEM problem, so feel free to redirect me to them if necessary.

The problem:

I just updated to SerialEM 4 which has an option to save a tilt-series in the .hdf file format. When saving the tilt-series in .hdf, it no longer writes an autodoc file (which is theoretically quite convenient). However, when I try to import the tilt-series into EMAN2, I get the following error:

Traceback (most recent call last):
File "/opt/miniconda3/envs/eman2/bin/e2proc2d.py", line 1169, in <module>
main()
File "/opt/miniconda3/envs/eman2/bin/e2proc2d.py", line 471, in main
threed.read_image(infile)
File "/opt/miniconda3/envs/eman2/lib/python3.7/site-packages/EMAN2.py", line 2684, in db_read_im
age
return self.read_image_c(fsp, *parms, **kparms)
RuntimeError: InvalidValueException at /opt/eman2/libEM/emdata_metadata.cpp:844: error with '0': '
x size <= 0' caught

Error running:  e2proc2d.py /home/t93j956/blackmore/t93j956/Projects/SARS-CoV-2/data/2022-05-11/to
mograms/sars_20220512_tomo_02.hdf ./tiltseries/sars_20220512_tomo_02.hdf --inplace

IMOD can open them fine, so I assume it's something to do with metadata.

Some debugging information:

The file browser views the dimensions as 0x0x0, but it does see that there's 41 tilts.

Here's the header information:

e2version.py:

EMAN 2.99 ( GITHUB: 2022-01-19 13:21 - commit: ba39969d8 )
Your Python version is: 3.7.9

Exact command:

It's not showing in the notebook for some reason, but it's straight importing, 8 bit, copy, .hdf file, and no further information/setttings.

Using e2projectmanager.py

Thanks for the help.

--

Colin Gauvin

PhD Candidate, Montana State University

Lawrence Laboratory

Cryo-EM Facility Assistant Manager

[Ludtke, Steven J.]

Hi Colin,

thanks for reporting this. Indeed, we encouraged David to add HDF5 support to IMOD, and while the file format itself is standardized, specific work has to be done to support specific header metadata produced by a specific software package. Any chance you could make one of the problem files available to us so we can debug the metadata? I suspect we can fix the issue pretty trivially.

--------------------------------------------------------------------------------------

Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine

Charles C. Bell Jr., Professor of Structural Biology        Dept. of Biochemistry 

Deputy Director, Advanced Technical Cores                   and Molecular Biology

Academic Director, CryoEM Core

Co-Director CIBR Center

On May 17, 2022, at 2:49 PM, Colin Gauvin <ccgau...@gmail.com> wrote:

Hi,

First - I'm not sure if this is an EMAN2 problem, or a SerialEM problem, so feel free to redirect me to them if necessary.

The problem:

I just updated to SerialEM 4 which has an option to save a tilt-series in the .hdf file format. When saving the tilt-series in .hdf, it no longer writes an autodoc file (which is theoretically quite convenient). However, when I try to import the tilt-series into EMAN2, I get the following error:

Traceback (most recent call last):
File "/opt/miniconda3/envs/eman2/bin/e2proc2d.py", line 1169, in <module>
main()
File "/opt/miniconda3/envs/eman2/bin/e2proc2d.py", line 471, in main
threed.read_image(infile)
File "/opt/miniconda3/envs/eman2/lib/python3.7/site-packages/EMAN2.py", line 2684, in db_read_im
age
return self.read_image_c(fsp, *parms, **kparms)
RuntimeError: InvalidValueException at /opt/eman2/libEM/emdata_metadata.cpp:844: error with '0': '
x size <= 0' caught

Error running:  e2proc2d.py /home/t93j956/blackmore/t93j956/Projects/SARS-CoV-2/data/2022-05-11/to
mograms/sars_20220512_tomo_02.hdf ./tiltseries/sars_20220512_tomo_02.hdf --inplace

IMOD can open them fine, so I assume it's something to do with metadata.

Some debugging information:

The file browser views the dimensions as 0x0x0, but it does see that there's 41 tilts.

Here's the header information:

<Screenshot_20220517_134540.png>

e2version.py:

EMAN 2.99 ( GITHUB: 2022-01-19 13:21 - commit: ba39969d8 )
Your Python version is: 3.7.9

Exact command:

It's not showing in the notebook for some reason, but it's straight importing, 8 bit, copy, .hdf file, and no further information/setttings.

Using e2projectmanager.py

Thanks for the help.

--

Colin Gauvin

PhD Candidate, Montana State University

Lawrence Laboratory

Cryo-EM Facility Assistant Manager

--
--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2

---
You received this message because you are subscribed to the Google Groups "EMAN2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/089bfb99-284d-46e0-a2ef-36017eccd2a7n%40googlegroups.com.
<Screenshot_20220517_134540.png>

[Colin Gauvin]

I'd very much appreciate that. I've emailed a link to the file to your BCM email.

To unsubscribe from this group, send email to eman2+unsubscribe@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/eman2

---
You received this message because you are subscribed to the Google Groups "EMAN2" group.

To unsubscribe from this group and stop receiving emails from it, send an email to eman2+unsubscribe@googlegroups.com.

[Ludtke, Steven J.]

Sorry it took a day to get to, but it's done. You should be able to import these files without difficulty now. Not sure how universal this solution will be, but it's a start. The change is only on GitHub at the moment, so you will need to build from source to get the fixed version (it isn't difficult). We will probably trigger a new binary snapshot sometime in the next few days, but waiting for a couple of other fixes before we do that...

--------------------------------------------------------------------------------------

Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine

Charles C. Bell Jr., Professor of Structural Biology        Dept. of Biochemistry 

Deputy Director, Advanced Technical Cores                   and Molecular Biology

Academic Director, CryoEM Core

Co-Director CIBR Center

To unsubscribe from this group, send email to eman2+un...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/eman2

--- 
You received this message because you are subscribed to the Google Groups "EMAN2" group.

To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/ad155af2-fc52-49c7-96d6-627a186cebc0n%40googlegroups.com.

[Colin Gauvin]

Seems to be working so far - just successfully reconstructed a couple tomograms. Thanks for doing it so quickly!

To unsubscribe from this group, send email to eman2+un...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/eman2

--- 
You received this message because you are subscribed to the Google Groups "EMAN2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/089bfb99-284d-46e0-a2ef-36017eccd2a7n%40googlegroups.com.
<Screenshot_20220517_134540.png>

-- 
-- 
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com

[Ludtke, Steven J.]

Great, glad that worked!

--------------------------------------------------------------------------------------

Steven Ludtke, Ph.D. <slu...@bcm.edu>                      Baylor College of Medicine

Charles C. Bell Jr., Professor of Structural Biology        Dept. of Biochemistry 

Deputy Director, Advanced Technical Cores                   and Molecular Biology

Academic Director, CryoEM Core

Co-Director CIBR Center

To view this discussion on the web visit https://groups.google.com/d/msgid/eman2/7ce8032c-78a8-4a46-9b28-5a6ea64c8a9bn%40googlegroups.com.