History of flattening algorithm?
W. Trevor King
was wondering if anyone could shed some light on the choice of an Nth
order polynomial rather than a sinusoid for removing the background
oscillation. I was under the impression that the oscillation was due
to interference between the laser reflecting off the surface and that
reflecting off the tip, as pointed out by Jaschke and Butt in 1994
( http://dx.doi.org/10.1063/1.1146018 ) and the figure 4 caption of
Weisenhorn et al.'s 1992 paper ( http://dx.doi.org/10.1103/PhysRevB.45.11226
). So it seems like a sinusoid would be more appropriate than a
polynomial. The flattening code dates back the beginning of the
versioned history, so there aren't any enlightening commit messages.
Does anyone remember who added the code and what their reasons for
selection a polynomial fit were?
ms
The oscillation is sinusoidal in fact, and for this fact sinusoid was
the first choice for me too, but in the end it didn't work as well -e.g.
drag sometimes adds a linear slope to the underlying sinusoidal, and
other optical disturbances can be present. I also played with filters,
but they messed out the phase.
The polynomial does a better job: it doesn't care if it deviates from a
purely sinusoidal pattern, it naturally goes through noise, it is easier
to fit correctly and it is immediate to fit with polyfit. I would
recommend you to use a polynomial as well, unless you have good reasons
to avoid it.
cheers,
m.
W. Trevor King
> The oscillation is sinusoidal in fact, and for this fact sinusoid was
> the first choice for me too, but in the end it didn't work as well -e.g.
> drag sometimes adds a linear slope to the underlying sinusoidal,
Do you mean "laser/detector drift" instead of "drag"? For constant
piezo-speed experiments, I'd expect the drag to be pretty constant
over the length of the pull. See Figure 2b of Janovjak, Struckmeier,
and Müller, 2004 ( http://dx.doi.org/10.1007/s00249-004-0430-3 ).
> and other optical disturbances can be present.
This does complicate things.
> The polynomial does a better job: it doesn't care if it deviates from a
> purely sinusoidal pattern, it naturally goes through noise,
I'm not sure what you mean by "goes through the noise".
Thanks for explaining, these features don't seem to get the airtime
I'd like ;).
--
This email may be signed or encrypted with GPG (http://www.gnupg.org).
The GPG signature (if present) will be attached as 'signature.asc'.
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy
My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt
ms
> On Sun, Aug 08, 2010 at 11:11:11PM +0100, ms wrote:
>> The oscillation is sinusoidal in fact, and for this fact sinusoid was
>> the first choice for me too, but in the end it didn't work as well -e.g.
>> drag sometimes adds a linear slope to the underlying sinusoidal,
>
> Do you mean "laser/detector drift" instead of "drag"? For constant
> piezo-speed experiments, I'd expect the drag to be pretty constant
> over the length of the pull. See Figure 2b of Janovjak, Struckmeier,
> and Müller, 2004 ( http://dx.doi.org/10.1007/s00249-004-0430-3 ).
Uh, well, yes, I meant drift. Sorry.
>> and other optical disturbances can be present.
>
> This does complicate things.
Well, not so much if you use a reasonable-degree polynomial
>> The polynomial does a better job: it doesn't care if it deviates from a
>> purely sinusoidal pattern, it naturally goes through noise,
>
> I'm not sure what you mean by "goes through the noise".
Sure, sorry. If you don't constrain the frequency of the sinusoidal, I
could think you can end up with some kind of sinusoidal which fits the
noise - definitely not what you want. Just a random guess when I wrote
the email, not necessarily probable.
Also, if the line to flatten is good (i.e. horizontal), how does the
sinusoidal behave? It would require an infinite (very high) period
sinusoid to fit correctly, which perhaps can make the fit unstable?
If you want to play with trigonometric functions, you are welcome, but
I've never needed to complain about the polynomial fit :) -is there
perhaps some issue you want to address?
cheers,
m.
W. Trevor King
> On 10/08/10 23:09, W. Trevor King wrote:
> > I'm not sure what you mean by "goes through the noise".
>
> Sure, sorry. If you don't constrain the frequency of the sinusoidal,
Ah, I see. It would certainly be a good idea to constrain the
sinusoid frequency to something on the order of half your laser
wavelength.
> Also, if the line to flatten is good (i.e. horizontal), how does the
> sinusoidal behave?
With a constrained freqyency, it would just revert to zero amplitude.
> I've never needed to complain about the polynomial fit :) -is there
> perhaps some issue you want to address?
No particular reason, I'm just leery about subtracting a random
polynomial from the data without some sort of theoretical support.
"Because it makes the bad stuff go away" doesn't sound very convincing
on its own ;).