convergence of 3D-refine

[Philip Koeck]

In a single particle refinement using either eman1 or eman2, is it possible that results can get worse after too many iterations, similar to what happens in algebraic iterative methods?

I'm thinking about very small proteins imaged as single particles. The images are very noisy and there's probably some structural heterogeneity.

Alternatively, could the reconstruction simply keep changing from iteration to iteration, meaning that there's never any convergence, not even quasi-convergence?

[Steven Ludtke]

I will confine the answer to EMAN2.1, where you actually have a mechanism for monitoring "better" and "worse" (gold-standard). If you look at the report generated while EMAN2.1 runs, you will note that it shows you two different sets of FSC curves. It shows the gold standard resolution as it changes from iteration to iteration, and it shows the 'convergence plot' independently for the even and odd maps. By looking at both of these sets of curves, you can learn quite a lot about what's going on. 

- if you observe the gold-standard resolution curve getting worse for one or two iterations, then getting better again, that is indicative of a bad initial model, and shows that both refinements have independently found their way to the correct structure. This is a very good sign.

- If you observe the gold-standard resolution starting at some point then getting worse, and eventually getting stuck at a worse point, this indicates that your initial target resolution was higher than the data could sustain, and you need to rerun the refinement with a lower (worse) target resolution.

- If you observe the gold standard curve getting stuck at some resolution, but the individual convergence plots continue to converge past this point, this is a clear indication of structural variability in the data, and means you really need to work with e2refinemulti.py to try and separate the data.

- If you are unable to achieve convergence & self-consistence in your map, there are two primary causes: 1) variability in the data or 2) a perversely bad initial model

- to exclude case 2 you need to rerun refinements with a couple of different starting models and observe what happens. If all of the starting models produce pretty similar results, then you know 2 isn't the issue.

- In case 1, running e2refinemulti, which you may need to run for a significantly larger number of iterations than e2refine_easy, is the best way to learn more. e2refine_easy should normally converge in 5-6 iterations, and should really never require more than 10 iterations for a homogeneous data set. With e2refinemulti, it can be worthwhile to run as many as 20 iterations in some unusual situations, though normally ~10 will give the results you are after.

Both Relion and EMAN can get stuck in local minima with heterogeneous particles, but there is a difference in behavior. In EMAN, if the data is very heterogeneous, and no self-consistent structure has been obtained, it will tend to 'bounce around' the local minima and continue to change at least subtly. In Relion, due to its use of ML, if it gets 'stuck' like this, it will tend to give very solid looking answers which won't change. It can be very difficult in this situation to tell whether the answers are 'real' or not. 

--
--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2

---
You received this message because you are subscribed to the Google Groups "EMAN2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

----------------------------------------------------------------------------
Steven Ludtke, Ph.D.
Professor, Dept. of Biochemistry and Mol. Biol.                Those who do

Co-Director National Center For Macromolecular Imaging            ARE

Baylor College of Medicine                                     The converse
slu...@bcm.edu  -or-  ste...@alumni.caltech.edu               also applies
http://ncmi.bcm.edu/~stevel

[Paul Penczek]

Hi,

first, please do not use term g***-standard as it makes me angry.

Second, I was referring to your description of relion, which was not quite right.

It does suppresses initial co-dependence, but in a different way.  Adding noise in high frequencies typically

would not change anything as templates are filtered by the FSC, which would set this region to zero anyway/

As for what you write below, actually relion uses very little ML, if any.

Pawel.

---

From: Steven Ludtke <slud...@gmail.com>
To: em...@googlegroups.com
Sent: Friday, January 22, 2016 7:23 AM
Subject: Re: [EMAN2] convergence of 3D-refine

[Steven Ludtke]

Ahh, now you've put it back in the mailing list again.

first, please do not use term g***-standard as it makes me angry.

I understand your statistical objections to g*** as a robust measure of consistent convergence. Certainly the work you have done to better explore the space of possible structures is on a more rigorous footing. Nonetheless, if you start at two different points (different starting models) with two different sets of particles and get them to converge to a structure which agrees at high resolution, it is a valid test of something. Whether you believe this is the correct criterion for the community to use for resolution or not, like all resolution criteria, is obviously a debatable and much debated point. The name "gold standard" is admittedly not ideal. Nonetheless, it does have a fairly precisely defined meaning, and when someone abuses it (calls it 'gold standard' but actually did something different) you can point at literature which defines what they should have actually done. Just like cryoEM -> cryo-electron microscopy, it is a bad name (the electrons aren't cold, the specimen is). Wah used to get quite upset, insisting that we had to change it to electron cryo-microscopy. I fear it may be too late to stop 'gold standard' from being used.

Second, I was referring to your description of relion, which was not quite right.

It does suppresses initial co-dependence, but in a different way.

In what way does it suppress the initial co-dependency?  Are you referring the the randomization they insert later at a resolution a bit lower than the FSC cutoff?

 Adding noise in high frequencies typically would not change anything as templates are filtered by the FSC, which would set this region to zero anyway/

This is not true! 

- In EMAN, yes we do filter by FSC, but this first happens only after one complete iteration, at which point the resolutions have typically already converged to the neighborhood of the full resolution

- If you compute an FSC between two maps at a resolution where they have been filtered to exactly zero, then of course, the FSC is undefined. If there are even small amplitudes, then, of course, any phases present will still produce an FSC. If the phases are random, the mean FSC is zero. 

- The filter we apply is a Wiener filter derived from the pseudoequivalence of FSC and SSNR. ie - the FSC is converted to an SSNR using your old FSC/(1-FSC) formula, then converted into a Wiener filter. These values can get quite small, but tend not to be exactly zero, so there are still random phase terms involved

As for what you write below, actually relion uses very little ML, if any.

There are two situations:

1) high quality highly homogenous particles which can converge to high resolution. In this case, you are correct, at the end of the refinement there is really no ML left in Relion.  In the first few iterations when it is starting with a 60 A low pass filtered map, ML is still having some impact on the convergence.

2) structurally variable or low quality particles, with fairly large angular uncertainties, not suitable for high resolution, but still used for biological interpretation. In this situation, ML continues to play a strong role even later in refinements.

If you put a starting model in which, for example, has a fairly sharp spherical mask to zero at some radius, then use this as a starting model, the mask will persist due to model bias for quite a long time in the refinement process. Even if you low-pass filter the mask to 60 A, if it is the same in both starting maps, it can have a persistent impact on the FSC curves for many many iterations (in certain situations). By randomizing the phases independently for the two maps, this sort of correlation never appears in the FSC curves. 

The point you and Nico have made repeatedly, that this process doesn't eliminate the model bias is true. What it does is eliminates the influence of any model bias on the FSC calculation. If the phases on a mask start out randomized there is really no mechanism for them to ever become un-randomized and start producing resolution exaggeration. There will still be some vague shadow of the mask present in the two maps, but it will be a different vague shadow in the resulting even and odd outputs, and due to other anti-bias algorithms in EMAN (and I assume Relion) even this shadow will rapidly fade.

[Mike Strauss]

 > Just like cryoEM -> cryo-electron microscopy, it is a bad name (the electrons aren't cold, the specimen is). Wah used to get quite upset, insisting that we had to change it to electron cryo-microscopy.

Hi.  I thought I had put this to rest already.  Obviously not.

In this case, Wah is categorically wrong.  No one in the entire universe goes to a purveyor of fine drinks and orders an Earl Grey hot tea.  And when they do say "hot Earl Grey tea", nobody fears they will obtain the water in which was steeped a feverish Earl Grey. The same is true of an electric dish washer. Dishes aren't electric, but nobody claims we should be saying dish electric washer.  That's because "Earl Grey tea", or "dish washer", are concepts, and these concepts can be modified by an adjective, "hot" or "electric".  Similarly, electron microscopy is a concept which is modified by "cryo-", since it describes a special type of electron microscopy.

[Steve Ludtke]

Ahh, people do get upset about the funniest things.   I confess I don't see a lot of point in wasting quite so much of one's life on labels. I may also have exaggerated when I said Wah got 'quite upset'.   

I spent quite a bit of time a week or so ago in earnest discussion with a gentleman who honestly felt that my statement on the 3dem list where I used MB/sec as a unit rather than MB/s was completely inexcusable, and he could not believe that a tenured professor would ever fail to use perfect SI symbology ... well I won't go on. I'm still not sure if I was being trolled or not.   Anyway, people do get upset about labels. I am not among them. So long as there is no ambiguity in what is meant, I really could care less what label is used.  

For the people who DO get upset the reasoning is clear. In one ordering there is ambiguity:  cryo(electron microscopy) or (cryo electron) microscopy. Either are possible interpretations in English, as electrons can be cold. However, electron cryo-microscopy has no such ambiguity because electron has no meaning as a modifier on 'cryo'.   Good lord, now I'm getting sucked in again.     I'm going to sleep now.

----------------------------------------------------------------------------
Steven Ludtke, Ph.D.
Professor, Dept. of Biochemistry and Mol. Biol.                Those who do

Co-Director National Center For Macromolecular Imaging            ARE

Baylor College of Medicine                                     The converse
slu...@bcm.edu  -or-  ste...@alumni.caltech.edu               also applies
http://ncmi.bcm.edu/~stevel

[Mike Strauss]

It reminds me of the debate on whether Pluto should be a planet.

> You received this message because you are subscribed to a topic in the
> Google Groups "EMAN2" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/eman2/Jcl-Vr0igNA/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to

[Paul Penczek]

Of course Pluto is a planet. I remember from school.

Regards,
Pawel