Problem using EMAN2 for obtain final averages in 2D refinement for AFM images

[Witchukorn Phuthong]

Hi everyone,

I'm trying to do single particle analysis - 2D refinement - of an AFM image with length 201 nm and width 201 nm with 256x256 pixels.  I used the Interactive Boxing - e2boxer to manually box the particles seen clearly in the beginning that are not circles or points (in Capture3.jpg) Since I have minimal knowledge in computer programming and totally new to image processing - parameter terms to choose, I will describe what I did in details; hopefully, it might help in figuring out how to solve my problems.

I switch the computer that I use EMAN to box particles to another; I copy the folder of boxed particle and open e2workflow.py in that folder. I tried to import the particles, but EMAN says it's already in the project.

Then I click Reference Free Class Averages and browse to add the particles (filename_filt_ptcls) and it shows in 2D Class Averages window (capture4.jpg). After that I click Generate Classes -e2refine2d but the 'Choosing Sets For e2refine2d' windows doesn't ask me to choose 'original data (3), Phase flipped (3), Wiener filtered (3), Phase filtered (3)', but has only 'Specify' to choose. I check it and click OK. (capture5.jpg)

In the Run e2refine2d window, General tab, I choose # classes to be 3 anad # alignment references to be 3 since I have small number of particles to begin with, and choose refine aligner = refine in Simmx tab that are not default value. Then I click  OK and let it run. However, it shows runtime error' saying ' the rotational footprint is too small (capture6.jpg) which I have no idea on where to go next.

In the folder r2d_01 from EMAN2 Browser from EMAN2 tasks, I can see only file 00image_counts and all2.

Do I have too small number of particles? Or the image that I use to process is not suitable for EMAN2 to begin with? What features of EMAN2 would you recommend for such image for that particular purpose (I might not have chosen the right function to suit what I need)? If I can go on with the same function, could you recommend what might be the right choice of parameters for Main comparator, Aligner, Aligner comparator and Refine align comparator?

Thank you very much in advance and I'm looking forward to your reply.

 

Best regards,

Champ

[Steven Ludtke]

Hi. EMAN was originally designed for transmission electron microscopy, so the end-user programs primarily

focus on that aim. However, fundamentally it is a versatile image processing library for grey-scale scientific

image processing of any sort. It has occasionally been used by people in the AFM community before, and I'd

be happy if it was used more frequently in that domain. Nonetheless, right now you're running up against

some of the fundamental differences between TEM/cryoEM and AFM.  The GUI workflow interface is 

targeted at one specific process, single particle reconstruction, where noisy projection images of

individual particles from TEM in random orientation are reconstructed to produce a 3-D volumetric

(not surface) model. 

That said, a number of the processes in this workflow would also be suitable for AFM analysis, if you keep

a few things in mind. I think the problem you're running into is that typical single particle reconstructions 

have box sizes of 64 - 1024 pixels. 32 is considered an extremely small box, and some routines simply 

won't work below ~32 pixels due to internal assumptions that are made.

The simple stopgap solution would be to simply scale your images so they are ~2x larger (you may also

want to look at http://blake.bcm.edu/emanwiki/EMAN2/BoxSize) as well. To accomplish this scaling, you

could do something like this

e2proc2d.py inputfile.hdf outputfile.hdf --clip=<newboxsize> --scale=2

or somesuch. The bdb: notation can take a little getting used to. 

However, I would also say that since you have an 'all2' file in your r2d_01 directory it implies that you 

specified a shrink of 2 in the GUI. This will downscale the particles to 1/2 the already small size you

started with. So, the first step might be to just try the 2D refinement again with shrink=1 in the GUI.

----------------------------------------------------------------------------
Steven Ludtke, Ph.D.
Associate Professor, Dept. of Biochemistry and Mol. Biol.      Those who do

Co-Director National Center For Macromolecular Imaging            ARE

Baylor College of Medicine                                     The converse
slu...@bcm.edu  -or-  ste...@alumni.caltech.edu               also applies
http://ncmi.bcm.edu/~stevel

--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2
<Capture5.JPG><Capture6.JPG><Capture4.JPG><Capture3.JPG>

[Witchukorn Phuthong]

Thank you very much for your suggestion. However, I have tried to

1) change the parameter shrink to '1' with the same image 256 pixel and 26 pixel boxes. It just didn't show the file 'all2', but the error message was still 'No child process'.

2) the same things ('all2' file showing up in shrink = 2, error due to 'no child process')  happened even when I increased the starting resolution of the image to 630 pixels and box size 64 pixels using imageJ program, 1024 pixels image resolution with 103 pixels, 2048/205, and 4096/410. For the last three, I generated the same images with higher resolution from the AFM image processing program.

Doing that, I maintain the same size box in nm scale. I agree that the previous box size is too small, but with the images that I have if I increased the box size in nm scale for example 40 nm, one box would contain more than one particle. Is there any other way that EMAN2 can make 2D refinement work under this restriction? I wonder though why input_fp of 64 pixels (in the r2d_... folder) show different boxed particles from those in the beginning.

Regards,

Champ

ps. I would also like to report a bug: the boxed particles saved in imagic extension can't be opened in Imagic program.

[Ludtke Steven]

Hi Champ. It's very difficult to tell from your description exactly what's going wrong. Let's take a single concrete example,

and describe exactly what fails and where, with the complete error messages you get. The 64 pixel box size data would 

be the best to test on. So, please try with that data, and send me more complete output. One additional thing you can try is

to look in the .eman2log.txt file, and find the e2refine2d.py command line, then run that command manually from the 

command line in your project directory with the '--verbose=3' option appended to the end of the command. That will give

more information about what's failing.

Looking at your images I'm a bit concerned that they may not process very well, but you can certainly try. I'm having a hard time seeing

exactly what it is you're interested in averaging ?  I see som apparent patterns of 3 black blobs, a larger one with two nearby smaller

ones, but they don't seem very homogeneous in size, and they don't seem well contained by your boxes. Is that what you're trying

to average, or something else I'm missing ?

--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2

<Capture-par-shrink1.JPG>

[pu qian]

Hi,

There is an alternative way to convert eman2 boxed particles to imagic5 although as you said that Imagic 5 can not open it if you output it from 2man2 to .img directly. The way is: first output your boxed particles to spider format, then convert spider format to imagic format by adding information of A/pixel. It is very possible that when eman2 convert boxed particle to imagic format, it does not add this information into its header file. Hope it is useful for you.

Best regards

Pu Qian

Sheffield University

--

[Ludtke Steven]

Current versions of EMAN2 (2.04+) should write IMAGIC files compatible with the most recent version of IMAGIC. If you're

having problems with this, please email Guang Tang <gt...@bcm.edu> with details on what you tried to do and how, and

he can either help sort out the problem you're having, or if necessary fix the code.  Thanks !

[Witchukorn Phuthong]

This time I will tell you in a bit more detail; hopefully it helps.

In the double quotation mark, I put the word, phrase, sentences displayed in the command prompt.

1. I save an image of 256 pixel for 200 nm to be come 640 pixel for 200 nm in tiff format using Gwyddion, AFM image processing software. This is because I want the box size to be 20 nm.

2. In command prompt, I go to the folder and open e2workflow.py

"WARNING: Using the out of date 'esddb' module instead of bsddb3. Consider installing newer berkeleydb and bsddb3."

3. I import my data using 'Raw Data'

"{}"

"EMSelectorBase.ok_button_clicked"

3. I import my data using 'Filter Raw Data'

"EMSelectorBase.ok_button_clicked"

I select the option Filter X-ray pixels, Edge norm, Inplace processing, Associate with project

4. I select 'Interactive Boxing - e2boxer'

"{'D66-640.tiff': {'Original Data': 'D66-640.tiff'}}

, change the box size to 64, select the image and OK

"<type 'exceptions.AttributerError'>

("'NonType' object has no attribute 'sort'",)

error, couldn't get image 0 0 0 0"

Then I pick particles.

4. I select 'Interactive Boxing - e2boxer'

"{'D66-640.tiff': {'Original Data': 'D66-640.tiff'}, 'D66-640_filt.mrc': {'Original Data': 'D66-640_filt.mrc'}"

, I change the box size to 64, select the image and OK

"<type 'exceptions.AttributerError'>

("'NonType' object has no attribute 'sort'",)

error, couldn't get image 0 0 0 0"

Then I pick particles.

5. I select 'Generate Output -e2boxer'

"d:/EMAN2/bin/e2boxer.py D66-t20_filt.mrc --norm=normalize.edgemean --format=bdb ==boxsize=64 --write_dbbox --write_ptcls --dbls=global.spr_ptcls_dict started process 5664

WARNING: Using the out of date 'esddb' module instead of bsddb3. Consider installing newer berkeleydb and bsddb3."

6. Since I can't type in the following line, I started a new command prompt, go to the folder and run the command e2refine2d.py --verbose=3

"WARNING: Using the out of date 'esddb' module instead of bsddb3. Consider installing newer berkeleydb and bsddb3.

Traceback <most recent call last>:

File "D:\EMAN2\bin\e2refine2d.py", line 427, in  <module> main ()

File "D:\EMAN2\bin\e2refine2d.py", line 129, in main tmp.read_image(options.input,0)

File "d:\EMAN2\lib\EMAN2db.py", line 422, in dp_read_image return self.read_image_c(fsp,*parms, **kparms)

RuntimeError: (

What I am interested in averaging looks like a pair of cylinders lying on the box, for example box 21 (in the new output file). I would like to determine the average diameter of the cylinder, distance between the two cylinders' axes and the length of the cylinder. The three blobs you mentioned (I think you meant for the particles I boxed for example in box 5 in the new output file) are parts of the cylinder: one can be a cylinder and another two can be a cylinder. 

I feel more comfortable to send more complete output file to the email of helper myself.

I have a question for Pu Qian: for the alternative of converting eman2 boxed particles to .img, Is the whole process that you mentioned done by EMAN2 or did you open spider format in spider and convert to imagic again?

I'll contact Guang Tang for the details.

Thank you again.

Champ 

[Witchukorn Phuthong]

Sorry I forgot to attach some of my boxed particles. 

Champ

[Steven Ludtke]

Is that the end ?

File "d:\EMAN2\lib\EMAN2db.py", line 422, in dp_read_image return self.read_image_c(fsp,*parms, **kparms)

RuntimeError: (

Nothing after "RuntimeError: ("

Grant, any idea ?

----------------------------------------------------------------------------
Steven Ludtke, Ph.D.
Associate Professor, Dept. of Biochemistry and Mol. Biol.      Those who do

Co-Director National Center For Macromolecular Imaging            ARE

Baylor College of Medicine                                     The converse
slu...@bcm.edu  -or-  ste...@alumni.caltech.edu               also applies
http://ncmi.bcm.edu/~stevel

[Grant Tang]

I fixed this issue. The EMAN2 release after 12/15/2011 should have the
apix value you set in the project written in the imagic files.

Grant

> > +unsub...@googlegroups.com

> > For more options, visit this group at
> > http://groups.google.com/group/eman2
> >
> >
> >
> >
> >
> > --
> > ----------------------------------------------------------------------------------------------
> > You received this message because you are subscribed to the Google
> > Groups "EMAN2" group.
> > To post to this group, send email to em...@googlegroups.com
> > To unsubscribe from this group, send email to eman2

> > +unsub...@googlegroups.com

> > For more options, visit this group at
> > http://groups.google.com/group/eman2
>
> --
> ----------------------------------------------------------------------------------------------
> You received this message because you are subscribed to the Google
> Groups "EMAN2" group.
> To post to this group, send email to em...@googlegroups.com
> To unsubscribe from this group, send email to eman2

> +unsub...@googlegroups.com

[Qian]

Hi grant,

Thank you very much for fixing this issue. It will benefit for many people who use EMAN for particle picking but IMAGIC package for further data processing. I will use it soon and will let you know results

Best regards

Pu Qian
Sheffield University

发自我的 iPod

> To unsubscribe from this group, send email to eman2+un...@googlegroups.com