batch conversion of FEI .ser files

[JK]

Hi all,

I found it a pain to convert the .ser images from FEI scopes to EMAN2-readable format using the available TIA, Digital Micrograph, and ImageJ plugins because all of them require you to take at least one action per file--attached is a macro for ImageJ that automates the process.  Run ImageJ, hit Plugins -> Macros -> Run and choose this ijm file to get a dialog box prompting you for source and destination directories and output format.  Also you can run it from command line.

I hope this saves someone some time.

JK

[Ludtke Steven]

I believe EMAN2 should read them (FEI format) files natively, if they are the format I'm thinking of. If this is a different format, give us what information you have, and we will add support for it. Could you provide an example image/stack in this format ?  (http://bigfile.bcm.edu can be used to transfer large files)

--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2
<BatchConvertSer.ijm>

[Robn]

Hi JK,


EMAN does not read FEI type of files. So, I ran your script on *.ser files and got an error: unrecognized command TIA reader. What does this mean?

Robn

[Robn]

Hi Steve,

1. I tried opening FEI *.ser files with e2helixboxer.py and got an error:
$ e2helixboxer.py --gui Q86Rgrid2_0000_4.ser
Traceback (most recent call last):
  File "/usr/local/EMAN2/bin/e2helixboxer.py", line 1197, in micrograph_table_selection
    micrograph = EMData(self.micrograph_filepath)
  File "/usr/local/EMAN2/lib/EMAN2db.py", line 370, in db_emd_init
    else : self.__initc(*parms)   
RuntimeError: FileAccessException at /home/eman64/EMAN2/src/eman2/libEM/emutil.cpp:373: error with 'Q86Rgrid2_0000_4.ser': 'cannot access file 'Q86Rgrid2_0000_4.ser'' caught

None

Also helixboxer, boxer and e2boxer.py fail to read *.ser files

2. Because EMAN1 & 2 both provide useful helixboxing utilities, can we expect e2ihrsy.py in EMAN 2.1?

Robn

[Ludtke Steven]

EMAN2 DOES read .ser files. Support was added about 3 weeks ago.

[Ludtke Steven]

Hi Robn,

This error indicates the file itself cannot be opened. This is an OS issue, not an EMAN2 issue. Either the file isn't there, you spelled the name wrong, or you don't have read permission...

[John Flanagan]

Hi JK

These files were generated in TIA. You can export them using TIA to TIFF files, and then read the TIFF files into EMAN2. TIA has a method from doing this in batch, but I forget how. Look it up in the TIA documentation.

To export just a few files: 

     TIA_>File->Export Data

      select 8 bit TIFF images.

John

--

[Ludtke Steven]

Hi John. Hope you're doing well  :^)

Did you see my message ?  EMAN2 has natively supported .ser files for the last month or so. There should be no need to convert first.

[JK]

Hi Robn,

The script I posted was for ImageJ, not for EMAN2.  I was converting the files as a batch with ImageJ to tiff and then running EMAN2 on the tiff.

This should no longer be necessary because of the updates to EMAN2 to read .ser files, but as Dr. Ludtke said it looks like there is some problem opening the file which is unrelated to it being in .ser format.  If the file could be opened but you wanted to change the format before manipulating it you could just enter the command

e2proc2d.py Q86Rgrid2_0000_4.ser Q86Rgrid2_0000_4.hdf

[Paul Penczek]

Hi,

I used eman2/sparx to process ser files for ~10 days now and there were absolutely no problems.

I have to admit I was not aware it was a novelty, I just tried it and it worked in a context of various

programs, including conversion to hdf and such...

Regards,

Pawel Penczek.

---

From: JK <jt...@cornell.edu>
To: em...@googlegroups.com
Sent: Fri, September 14, 2012 6:56:46 PM
Subject: [EMAN2] Re: batch conversion of FEI .ser files

--

[John Flanagan]

Sorry, I only saw the first e-mail.

BTW: I am doing well 

[julien...@gmail.com]

Thanks you saved my time! Now I will have a look to insert automatically the scale that would be nice... if you have any idea let me know.

Cheers,

J.D

[Steven Ludtke]

Insert the scale ?  Normally all of the images in one project would be collected at the same mag, and the A/pix value is a project-level setting, if you're using EMAN2...  Why do you need the mag stored in the header ?   Generally the set value on the microscope is only +-3-5% anyway, and the program can't get A/pix from mag without also knowing the CCD/DDD camera scaling factor (unless you store a scaled calibrated mag in the header).   That is, I'd regard A/pix as the definitive value, not mag.

----------------------------------------------------------------------------

Steven Ludtke, Ph.D.

Professor, Dept of Biochemistry and Mol. Biol.         (www.bcm.edu/biochem)

Co-Director National Center For Macromolecular Imaging        (ncmi.bcm.edu)

Co-Director CIBR Center                          (www.bcm.edu/research/cibr)

Baylor College of Medicine                             

slu...@bcm.edu

-- 

-- 
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2
 
--- 
You received this message because you are subscribed to the Google Groups "EMAN2" group.

To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[julien dugay]

thanks for your answer. Actually I'm a beginner and just need to analyze TEM pictures recorded by my colleague. She use to convert images with gatan digital micrograph but in my case I'm on mac so I would use imagej instead. So with your macro I can convert images from ser (I have the emi files as well) to tiff format but I would appreciate to have the scale bar automatically added and why not auto brightness contrast...

2013/10/27 Steven Ludtke <slud...@gmail.com>

You received this message because you are subscribed to a topic in the Google Groups "EMAN2" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/eman2/2TTnZ-Q88yc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to eman2+un...@googlegroups.com.

[Steven Ludtke]

Hi Julien. I'm not quite clear on what exactly you're doing.  A scale bar is a modification of the image itself. That is, it destroys some of the information in the image (not desirable when doing quantitative analysis, like single particle reconstruction). So, normally in cryo-EM it would be added when necessary (for a powerpoint presentation or somesuch) by the visualization program or even in PowerPoint, not as part of the image processing pipeline. 

EMAN2's visualization programs (like e2display.py) allow you to measure features in the image individually (by dragging a line) so we don't display a visual scale-bar. Brightness and contrast in the display is also handled automatically. In your case, I don't even know what you're using to do the visualization. 

If you explain exactly what it is you're trying to do, and with what software, we can probably make some suggestions of how to accomplish it. You seem to be assuming that everyone else is using images in the same way you are, but it sounds like you're doing something much more qualitative than most EMAN2 users. Nothing wrong with this, but you need to explain...

[Julien Dugay]

hey,

so thanks to a collaboration I got raw images (ser and emi files) from my colleague recorded with a TITAN TEM. I guess they use gatan digital micrograph but I'm not sure actually. As a final stage I would like to use imageJ to analyze the shape of my nanoparticles and build size distribution histograms from those pictures taken at different magnification. In this scope I wanted to use a macro to analyze the nanoparticle size on several micrographs (the more the better) but I need the good scale and good contrast/brightness settings. So that's what I'm trying to do with imagej with only one macro. I hope it's a bit more clear this time… thanks!

JD

2013/10/27 Steven Ludtke <slud...@gmail.com>

[Steven Ludtke]

I don't believe the .SER files contain any magnification information in the header. At least the specifications we used didn't include this information. .SER files come from the Titan software itself. Digital Micrograph is made by Gatan and uses DM4 format (at least the latest versions do) by default. The trick is that there are 3 factors involved:

1) the mag of the microscope

2) the pixel size of the sensor

3) relative mag factor based on the position of the sensor in the column

Anyway, when the file format contains A/pix information in the header we do faithfully copy it. 

[Julien Dugay]

ok it's a bit mysterious for me but at least I did a macro which start to work (for the moment just a contrast enhancement and an automatic scale) so I can manually build the size distribution of my NPs let's see if can automate the thing… thanks anyway!

2013/10/28 Steven Ludtke <slud...@gmail.com>