Creating negative stain TEM 2D classes; something is wrong

[BENJAMIN MERRITT]

Hello,

Does anyone know what I may have done wrong to end up with 2D class averages that look like this (please see attached)?

When I looked at in/excluded particles, hardly any of my particles show up (I had selected ~500 particles), and the images are all messed up.  They should look a lot different than these unusual Fourier-space type objects.

The only thing I can think of is that my box size is 1.25x my particle size, because I had a rather high density of particles.  Does it have something to do with an error in entering Cs, Apix, etc?

Thanks

Ben

[Steve Ludtke]

Could you show a display of some typical particle images (phase-flipped particles from a micrograph with typical defocus)?

--
--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2

---
You received this message because you are subscribed to the Google Groups "EMAN2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
<Screen Shot 2017-08-30 at 8.10.08 PM.png>

[BENJAMIN MERRITT]

Yes.

Do I need to invert neg stain?  I found that it makes the particles darker, like raw cryo micrographs.

[BENJAMIN MERRITT]

[Steve Ludtke]

The contrast you're showing (if those are phase-flipped) is correct.  The box size is too small though. The algorithm will ignore the peripheral densities from other particles fairly well, but it is important to have the larger box for things to work correctly.  You can check the "highdensity" checkbox in CTF autoprocessing, and that will help reduce the impact of adjacent particles in the box. 

It might also be worth giving the "Bispectrum-based class averaging" a try. It's available in the current snapshot versions of EMAN2.2, and does classification in a different way. I suspect it will do a better job on particles like this. Even with that, though, your box needs to be significantly larger.

--
--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2

---
You received this message because you are subscribed to the Google Groups "EMAN2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

<Screen Shot 2017-08-30 at 8.18.43 PM.png>

[Benjamin Merritt (RIT Alumni)]

Okay, thank you very much.  I didnt realize it could handle the peripheral bits.  That's why I made my boxes small.  I had seen some EM in the literature with dense images like mine, and their classes were okay.  So I wondered if maybe they just used a small box size.

Is the snapshot akin to a plugin or so?

Thanks again,

You are very helpful

Ben

On Aug 30, 2017 8:31 PM, "Steve Ludtke" <slud...@gmail.com> wrote:

The contrast you're showing (if those are phase-flipped) is correct.  The box size is too small though. The algorithm will ignore the peripheral densities from other particles fairly well, but it is important to have the larger box for things to work correctly.  You can check the "highdensity" checkbox in CTF autoprocessing, and that will help reduce the impact of adjacent particles in the box. 

It might also be worth giving the "Bispectrum-based class averaging" a try. It's available in the current snapshot versions of EMAN2.2, and does classification in a different way. I suspect it will do a better job on particles like this. Even with that, though, your box needs to be significantly larger.

On Aug 30, 2017, at 7:19 PM, BENJAMIN MERRITT <bam...@g.rit.edu> wrote:

Yes.

Do I need to invert neg stain?  I found that it makes the particles darker, like raw cryo micrographs.

On Wednesday, August 30, 2017 at 8:15:28 PM UTC-4, BENJAMIN MERRITT wrote:

Hello,

Does anyone know what I may have done wrong to end up with 2D class averages that look like this (please see attached)?

When I looked at in/excluded particles, hardly any of my particles show up (I had selected ~500 particles), and the images are all messed up.  They should look a lot different than these unusual Fourier-space type objects.

The only thing I can think of is that my box size is 1.25x my particle size, because I had a rather high density of particles.  Does it have something to do with an error in entering Cs, Apix, etc?

Thanks

Ben

--
--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com

To unsubscribe from this group, send email to eman2+unsubscribe@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/eman2

---
You received this message because you are subscribed to the Google Groups "EMAN2" group.

To unsubscribe from this group and stop receiving emails from it, send an email to eman2+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
<Screen Shot 2017-08-30 at 8.18.43 PM.png>

--
--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com

To unsubscribe from this group, send email to eman2+unsubscribe@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/eman2

---

You received this message because you are subscribed to a topic in the Google Groups "EMAN2" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/eman2/2FI56EVMMec/unsubscribe.
To unsubscribe from this group and all its topics, send an email to eman2+unsubscribe@googlegroups.com.

[Steve Ludtke]

No, it's a full installation. One is generated from the development code every day. It works just like the release versions, but the code may or may not be stable. It should be pretty stable at the moment, and we're in the process of adding a number of interesting new programs.

To unsubscribe from this group, send email to eman2+un...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/eman2

---
You received this message because you are subscribed to the Google Groups "EMAN2" group.

To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.

[Benjamin Merritt (RIT Alumni)]

Cool, thanks!

Ben

[BENJAMIN MERRITT]

Dear Steve,

I tried again with 1.5x and 2x box sizes compared to my particles, but the program seems to eliminate many particles (I picked ~500, but it eliminates 400 it claims, and the particles it shows are not how any of mine looked).  

In the terminal, it shows:

"Skipped particle with bad edge" about 50x

"Particle outside of micrograph detected, marking as bad" about 60x

Then under CTF, there is only 106 particles listed, but I picked way more.  Are they too low quality picks?  It ends up creating these amorphous classes and particles after I try 2D classification.... I was able to get this to work before, but just not on my negative stain.

Thanks

Ben

On Wednesday, August 30, 2017 at 8:15:28 PM UTC-4, BENJAMIN MERRITT wrote:

[Steve Ludtke]

Hi Ben, 

typically classification is run on 1000-2000 particles, creating at least 30-50 classes. If the contrast is high enough, of course, then it's not clear what the point of doing a lot of reference free classification really is, since the particles themselves already have similar contrast to typical class-averages. It will help blur out some of the other peripheral densities, and may still be useful to see how many different particles you have in different characteristic views. Anyway, for an asymmetric object you need a minimal number of different views, regardless of contrast to have decent 3-D coverage. For an asymmetric object like this, I'd estimate you need at least something like 100 different views to do anything very useful.

Looking at the representative micrograph you posted, I think you are being way too conservative in your particle picking. While overlapping or aggregated particles are bad, particles in close proximity are generally fine. ie - you could have boxed a LOT more particles from that image you posted. You got maybe 25% of the usable particles. Do not be tempted to focus only on particles with a particular appearance. Remember you are dealing with all different orientations, and it may be very difficult to conceptualize exactly what a particle will look like in a strange orientation. Those orientations are critical for anything you do in 3-D. ie - don't let your human bias of what you expect to get to influence your particle picking. Now, if you see something which is clearly too large or clearly too small, you may be able to ignore those (or you can let classification sort it out for you later), but anything that is vaguely the same size should be selected.

With respect to the other issue about losing particles, there are a lot of things that could be going on here. The "outside micrograph" and "bad edge" messages are for particles where the box extended outside the physical micrograph. Looking at the sample micrograph you posted, I don't see any reason that, even with a pretty big box size, you should lose more than ~5% of your particles. Clearly something odd is going on.

If you run the project manager and click on "Particles" (not something under that, but particles itself), you should see a dialog showing you how many particles you have in each micrograph. This interface shows everything extracted from the micrograph including "bad" particles. You can similarly click on "Generate Output (e2boxer)" and hit the "browse" button and it will show you how many boxes you currently have marked in each micrograph. If these look fine, try build sets again.

-- 
-- 
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com

To unsubscribe from this group, send email to eman2+un...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/eman2

--- 
You received this message because you are subscribed to the Google Groups "EMAN2" group.

To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.

[BENJAMIN MERRITT]

Thank you Steve.  That gives me more confidence at least.  I am trying to get used to using this program so I can have more experience in the future with an actual data set.  No one else in my lab has any experience with this, so your input is very much appreciated!

I tried running 2D classification again but with a different data set, and I keep having my number of particles reduced from ~1000 to ~120 after building my set with an obscene number of particles being discarded in the terminal lines (see screenshots).  The "particle" tab shows all my particles picked correctly, but somewhere between CTF correction and building a set, it reduces my total particles.  Maybe it has something to do with my machine?  I will try it on another machine where this did not previously happen.  Maybe something to do with how I downloaded it?

Also, is it bad if "particle circles" over lap with each other, even if the particles in the circles don't?  I chose the max particle size for its long axis, so when some particles are in a different view closer together, their dimensions are small but in higher density areas, "particle circles" over lap, but the particles themselves don't seem to.

Thanks so much,

Ben

[Steve Ludtke]

I could be wrong, but it looks like you may just be getting particles from a single micrograph? When you build sets are you remembering to either check the 'allparticles' box or insure that all of the particle files have been selected with the browser?  If so, are the filter settings below this set so all of the micrographs will be included?  When you launch the set builder it will display some messages on the console which may give more insights into what's going on.

<Screen Shot 2017-09-01 at 9.15.42 PM.png><Screen Shot 2017-09-01 at 9.15.30 PM.png>

[BENJAMIN MERRITT]

Steve,

I didn't create a set, as the tutorial mentions it is not necessary since the CTF correction automatically builds one for me.  That is the automatic set that appears, only 126 particles.  For some reason, it decides to eliminate them all.  As a result, I don't hit "make set" since I am just trying to create a first class average to see what happens.

Here is one weird thing maybe:

TypeError: string indices must be integers, not unicode

import  170614full_length_collate_nachr10X_0001

import  170614fulllengthcollatenachr10X0003

import  170614fulllengthcollatenachr10X0004

import  170614fulllengthcollatenachr10X0005

import  170614fulllengthcollatenachr10X0006

import  170614fulllengthcollatenachr10X0007

import  170614fulllengthcollatenachr10X0008

I have also included my terminal happenings in a text file, perhaps you may be able to scan over it quick?  Here is also the class average it decided to make (attachment) with a screen shot of an example image.  Some of the "classes" are "not classes" and I get this error message.  It's like all my picked particles just go away!  I want to try on another machine; I'll copy my project files over and see, it is just a slower machine (2 cores).

Thanks so much,

Ben

[Steve Ludtke]

It does make one automatically, but something is clearly going wrong, so you might try doing this manually and see what happens. One possibility is that it is having difficulties with the CTF since this is stain data. What did you use for "ac" and "constbfactor" when doing the CTF autocorrection?

<Weird 2D Class.rtf><Screen Shot 2017-09-01 at 9.35.20 PM.png><Screen Shot 2017-09-01 at 9.45.34 PM.png>

[BENJAMIN MERRITT]

constbfactor was -1.0 and ac 10.  I noticed that when I imported micrographs into the program on this computer, no green squares showed up.  So, I tried particle picking again manually on my smaller computer, where these green squares show up like normal, but I still got sort of crappy classes, where they all looked like noise (a bright circle in Fourier space sort of thing).  So, perhaps there is something wrong with the program on this computer and some of the manual settings I am using? 

Sorry this is so confusing.

Here are some images from me trying to use this same data on the computer where the program DOES seem to pick particles correctly.  So on this computer, it doesn't discard my particles, but I still have issues.  I tried using more particles like you suggested and choosing "high density."  

Thanks

Ben

[Steve Ludtke]

For negative stain, %AC should be more like 60-90% not 10%. 

<Micrograph.png><Particles.png><20 Classes (1400 particles, high density setting).png><Included particles_note no longer look like particles.png>

[BENJAMIN MERRITT]

Okay, I will try that.

Thanks

Ben

On Wednesday, August 30, 2017 at 8:15:28 PM UTC-4, BENJAMIN MERRITT wrote:

[BENJAMIN MERRITT]

Dear Steve,

I tried this but still ended up getting particles that looked like above. I read through some other posts on 2D classification with negative stain TEM and thought maybe I should avoid using CTF auto correction?  It seems like my particles are boxed out OK until I do this, then they look like the attached image (shapeless, bright spots).

I have yet to try the other method you mentioned before, however, so I will also try that (bispectrum based class averaging).

Thanks

Ben

[BENJAMIN MERRITT]

Or could it be related to how one of my micrographs seems to have different staining? (see attached)

On Wednesday, August 30, 2017 at 8:15:28 PM UTC-4, BENJAMIN MERRITT wrote:

[Steve Ludtke]

Did you look at the CTF fitting in the GUI?  It is quite possible for the autofitting to fail in negative stain, particularly if you are imaging (as many do in stain) very close to focus, and you fail to adjust the min/max defocus search range appropriately. If there are gross defocus fitting errors, adjusting them to ~correct then rerunning autofitting should solve that problem.

--
--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2

---
You received this message because you are subscribed to the Google Groups "EMAN2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

<darker_particles(most).png><lighter_particles.png>

[BENJAMIN MERRITT]

Steve,

The actual defocus in my raw micrographs is ~31.4, but the range in the program was set 0.6-4, and the micrographs were given a defocus value of between 1-2.  Therefore, this means that I set the range incorrectly, no?  As I understand it, 31.4 is fairly far from the focus, or?  I will try using this value and see what happens.

Thanks

Ben

[Steve Ludtke]

31.4 microns, or 31.4 nm ???  EMAN defocus values are expressed in microns, as typical CryoEM images are taken 1-3 microns underfocus. In negative stain, often people target near focus conditions, but if you meant nm, then your range should be more like 0.02 to 0.5, and you need to make sure your %AC is set pretty high (maybe 80%?)

----------------------------------------------------------------------------

Steven Ludtke, Ph.D.

Professor, Dept of Biochemistry and Mol. Biol.         (www.bcm.edu/biochem)

Co-Director National Center For Macromolecular Imaging        (ncmi.bcm.edu)

Co-Director CIBR Center                          (www.bcm.edu/research/cibr)

Baylor College of Medicine                             

slu...@bcm.edu

[BENJAMIN MERRITT]

I'm fairly certain it's microns, not sure why it's so high, it's simply the value I used to get good focus.  It could be I didn't adjust the stage properly for these images...but that still didn't work when I set the defocus to 31.  Why would my particles lose all their resolution/detail, and create these bright spots?

[Steve Ludtke]

If you're willing to send me a representative image, I can take a look.

<Screen Shot 2017-09-22 at 12.53.52 PM.png>