bandpass tanh filter

[Manikandan Karuppasamy]

Hi,

I would like to know how to use the bandpass tanh filter to filter a volume.

I see the following option, but i do not understand the parameters.

filter.bandpass.tanh             :  Low_fall_off(FLOAT)   apix(FLOAT)   cutoff_abs(FLOAT)   cutoff_freq(FLOAT)   cutoff_pixels(FLOAT)   fall_off(FLOAT)   high_cutoff_frequency(FLOAT)   high_fall_off(FLOAT)   low_cutoff_frequency(FLOAT)   sigma(FLOAT)  

The band pass that we want : 25 A (low pass) and 300 A (for high)

The pixel size is 3.6 A.

thanking you

with kind regards

Mani.

[Steven Ludtke]

Hi. Yes, if all you look at is that, I agree it is confusing. However, you can get more information by saying

e2help.py processors -v2

...

filter.bandpass.tanh : Bandpass tanh processor applied in Fourier space.

Low_fall_off(FLOAT) - Tanh low decay rate.

apix(FLOAT) -  Override A/pix in the image header (changes x,y and z)

cutoff_abs(FLOAT) - Processor radius in terms of Nyquist (0-.5)

cutoff_freq(FLOAT) - Resolution in 1/A (0 - 1 / size*apix)

cutoff_pixels(FLOAT) - Width in Fourier pixels (0 - size()/2)

fall_off(FLOAT) - Tanh decay rate.

high_cutoff_frequency(FLOAT) - Absolute [0,0.5] high cut-off frequency.

high_fall_off(FLOAT) - Tanh high decay rate.

low_cutoff_frequency(FLOAT) - Absolute [0,0.5] low cut-off frequency.

sigma(FLOAT) - Gaussian sigma (0-.5)

...

This still needs a little explanation. All of the filter.* processors take a standard set of 3 arguments:

cutoff_abs(FLOAT) - Processor radius in terms of Nyquist (0-.5)

cutoff_freq(FLOAT) - Resolution in 1/A (0 - 1 / size*apix)

cutoff_pixels(FLOAT) - Width in Fourier pixels (0 - size()/2)

These arguments allow you to specify the cutoff frequency for a filter in a number of different ways, without making mistakes in the interconversion. Additionally, the different parameters allow you to deal more effectively with non-square images (in EMAN1 only cutoff_pixels was used).

The trick is that with a bandpass filter, you clearly need two cutoff values, not just one, and some filters have additional parameters beyond the cutoff value (such as the steepness of the falloff).  These parameters are often specified with less flexibility than the above 3 options. In this case, you would normally use:

high_cutoff_frequency(FLOAT) - Absolute [0,0.5] high cut-off frequency.

high_fall_off(FLOAT) - Tanh high decay rate.

low_cutoff_frequency(FLOAT) - Absolute [0,0.5] low cut-off frequency.

Low_fall_off(FLOAT) - Tanh low decay rate.

The "L" in the low falloff is actually a typo. All parameters are supposed to be lower-case only  :^(

--
--
----------------------------------------------------------------------------------------------
You received this message because you are subscribed to the Google
Groups "EMAN2" group.
To post to this group, send email to em...@googlegroups.com
To unsubscribe from this group, send email to eman2+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/eman2
 
---
You received this message because you are subscribed to the Google Groups "EMAN2" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eman2+un...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Paul Penczek]

Hi,

you might be looking at the wrong place.

The description of the filter is here:

http://sparx-em.org/sparxwiki/filt_tanh

You will find there equation de.  Filter will ask for input in absolute frequency units, and the appropriate page

is linked under Input.   You will find there equations to convert resolution to abs freq.

There is a confusing comment by somebody there (Steve?).  It follows from an unfortunate convention selected

for EMAN2.

Vast majority of low-pass filters require two parameters and there are two choices how to do it:

1. (stop band, pass band) - meaning frequencies at which the filter drops below certain value (ideally one, more often

something arbitrarily chose), and frequency at which the filter seizes to be higher than selected value (ideally one, usually

arbitrarily chosen).  Example is a Butterworth filter

2.  (cut-off, fall-off) - first is the frequency at which filter value is 0.5 and the "width" of the filter around this

value, again arbitrarily chosen levels, as above.   This convention makes more sense for EM as cut-off is simply

the resolution specified by FSC at 0.5 cut-off value.  Example is tangent 

A notable exception is Gaussian filter, which is characterized by just one parameter, cut-ft value, but it is often

unclear what this cut-off is.  Often it is not specified whether it means value of 0.5 or half-width or whatever.

Gaussian filter should be avoided at all cost at it never drops reasonably close to zero.

Bandpass filter will be characterized by at least FOUR parameters (not two as stated somewhere), as one has to specify

two pairs of values.  Bandpass filters start from zero at freq 0 , then rise to one, and drop to zero again.  See 

http://sparx-em.org/sparxwiki/filt_tanb

for details.

Again, an exception is a Gaussian bandpass filter which indeed is specified by two values

http://sparx-em.org/sparxwiki/filt_tanb

As a low-pass filter should resemble the FSC curve, one has two choices:

1. simply get 0.5 freq from the FSC curve and measure from it a fall-off value (distance between 0.9 and 0.1 values, for example).

2. for "regular" fsc curve (meaning not jittery) there are functions that wild o the job for you:

fit_tanh(dres, low=0.1)

    dres - list produced by the fsc funcion

    dres[0] - absolute frequencies

    dres[1] - fsc, because it was calculated from the dataset split into halves, convert it to full using rn = 2r/(1+r)

    dres[2] - number of point use to calculate fsc coeff

    low cutoff of the fsc curve

    return parameters of the tanh filter: freq - cutoff frequency at which filter value is 0.5, and fall_off, the 'width' of the filter

or

fit_tanh1(dres, low=0.1)

    dres - list produced by the fsc funcion

    dres[0] - absolute frequencies

    dres[1] - fsc, to be conservative, do not use factor of 2.

    dres[2] - number of point use to calculate fsc coeff

    low cutoff of the fsc curve

    return parameters of the tanh filter: freq - cutoff frequency at which filter value is 0.5, and fall_off, the 'width' of the filter

Second more "conservative".

The easiest approach is to use the selected filter using sparx command line.

Otherwise, lowpass tangent filter tilt_tanl is available through:

e2proc3d.py tmp.hdf ../model_structure.hdf --process=filter.lowpass.tanh:cutoff_abs=0.45:fall_off=0.1 

There is much more on various filters on SPARX wiki pages.

Pawel.

---

From: Manikandan Karuppasamy <manikandan....@gmail.com>
To: em...@googlegroups.com
Sent: Monday, February 24, 2014 4:39 AM
Subject: [EMAN2] bandpass tanh filter

[Steven Ludtke]

The one standardized parameter in the EMAN2 convention (the "cutoff") is designed such that if you try different lowpass filters with the same cutoff frequency you should achieve somewhat comparable effects. Indeed, the majority of filters do require specifying another parameter, but it is much more difficult to standardize these in any way, as the mathematical definitions of the filters vary so widely.  

I still disagree with you on the point of whether Gaussian filters are useful or not. The fact that they can be combined sequentially, and are somewhat invertable (due to the fact that they don't get all the way to zero), makes them extremely useful in many contexts, and they are used widely in EMAN2.   That is not to say they are the only choice, and for low-pass filtering when you have an FSC, they are not used.

When you wish to apply a low-pass filter based on an FSC plot, there is a specific filter for this in EMAN2. I'm assuming it's also available in some way from SPARX. 

filter.wiener.byfsc

will convert an FSC curve to a corresponding Wiener filter (making a few assumptions). It is very important to note that filtering final 3-D maps is a tricky issue. Applying an additional filter at the resolution determined by the FSC curve may often result in overfiltering your map. The first issue is CTF amplitude correction. If you don't perform any amplitude correction during reconstruction, then your map is already implicitly filtered by the B-factor of the data, and applying an additional filter must at least take this into account.  The second issue is how filters can be combined sequentially.   If you use a "sharp" filter (goes immediately from 1 to 0), then, of course, there is no historical "memory". You can take a map and first filter to 5 Å, then later filter to 8 Å without worrying if the 5 Å filter happened first or not. With Gaussian filters, they are additive: e^(-B1 s^2) * e^(-B2 s^2) = e^(-(B1+B2) s^2). This means you know a B factor of B1 has already been applied, and your goal is to achieve a final B-factor of B, you know you should filter with a B factor of B-B1.  If you use some other sort of filter, like a tanh or butterworth, then you are someplace between these two extremes, and it may be unclear the correct way to proceed.

The current EMAN2 approach during refinement is to do full amplitude correction during reconstruction to virtually produce an "uncorrected map" which is then filtered using the FSC based low-pass filter using a "gold standard"  (unbiased) FSC curve.  Pawel and I could probably argue back and forth for an hour about various aspects of this, but in any case, this is EMAN2's canonical procedure.

[Paul Penczek]

---

From: Steven Ludtke <slud...@gmail.com>
To: em...@googlegroups.com
Sent: Monday, February 24, 2014 10:39 AM
Subject: Re: [EMAN2] bandpass tanh filter

The one standardized parameter in the EMAN2 convention (the "cutoff") is designed such that if you try different lowpass filters with the same cutoff frequency you should achieve somewhat comparable effects. Indeed, the majority of filters do require specifying another parameter, but it is much more difficult to standardize these in any way, as the mathematical definitions of the filters vary so widely.  

>>>>  This is a math impossibility.  Clearly a given filter can have the same cutoff value but different fall-offs.  Just please look at the math.

I still disagree with you on the point of whether Gaussian filters are useful or not. The fact that they can be combined sequentially, and are somewhat invertable (due to the fact that they don't get all the way to zero), makes them extremely useful in many contexts, and they are used widely in EMAN2.   That is not to say they are the only choice, and for low-pass filtering when you have an FSC, they are not used.

>>>>  The reason they should not be used is that unlike Gaussian filter a resolution curves normally drops to zero (or oscillates around it)

at relatively low frequencies.  I have 3D refinement in mind here.  If a Gaussian filter if used the strange (mis)information in high frequencies

is passed.

When you wish to apply a low-pass filter based on an FSC plot, there is a specific filter for this in EMAN2. I'm assuming it's also available in some way from SPARX. 

filter.wiener.byfsc

will convert an FSC curve to a corresponding Wiener filter (making a few assumptions). It is very important to note that filtering final 3-D maps is a tricky issue. Applying an additional filter at the resolution determined by the FSC curve may often result in overfiltering your map. The first issue is CTF amplitude correction. If you don't perform any amplitude correction during reconstruction, then your map is already implicitly filtered by the B-factor of the data, and applying an additional filter must at least take this into account.  The second issue is how filters can be combined sequentially.   If you use a "sharp" filter (goes immediately from 1 to 0), then, of course, there is no historical "memory". You can take a map and first filter to 5 Å, then later filter to 8 Å without worrying if the 5 Å filter happened first or not. With Gaussian filters, they are additive: e^(-B1 s^2) * e^(-B2 s^2) = e^(-(B1+B2) s^2). This means you know a B factor of B1 has already been applied, and your goal is to achieve a final B-factor of B, you know you should filter with a B factor of B-B1.  If you use some other sort of filter, like a tanh or butterworth, then you are someplace between these two extremes, and it may be unclear the correct way to proceed.

The current EMAN2 approach during refinement is to do full amplitude correction during reconstruction to virtually produce an "uncorrected map" which is then filtered using the FSC based low-pass filter using a "gold standard"  (unbiased) FSC curve.  Pawel and I could probably argue back and forth for an hour about various aspects of this, but in any case, this is EMAN2's canonical procedure.

>>>>>  Outside of "bronze standard" this is indeed how this is done even if details vary, even though I am not sure what "uncorrected map" means.  Sounds to me like a map reconstructed without any adjustments and corrections for anything.

Pawel.

[Paul Penczek]