How Does Linearized LSQ Work?

[W. eWatson]

See Subject. Is there some general approach on applying it? I believe
it's an iterative process. How does one linearize a function in this
case? Suppose one does linearize it, how does one achieve the next step?
Can someone provide a simple example of it, say, for something like y
= a+b*sin(x)+c*x*x or even just y=a+b*x*x*x?

[Ray Vickson]

If by LSQ you mean "least squares", then your examples are not what
you want: they are *linear* (as functions of the "unknowns" a, b and
c). These are nonlinear in x, but that does not matter. A truly
nonlinear least squares problem might, for example, be of the form y =
a + b*exp(c*x) or y = a + b*sin(x+c) or y = a*x^b. Here, there is a
nonlinear relation between the unknown parameters a, b, c and the
observable y. See, eg., http://en.wikipedia.org/wiki/Least_squares or
http://www.statsoft.com/TEXTBOOK/stnonlin.html . One of the standard
method used in nonlinear least-squares problems is the Levenberg-
Marquardt algorithm. Do a Google search for more details.

R.G. Vickson

[W. eWatson]

Ah, thanks. It's been awhile. Good wiki reference on the iterative
method, which is really what I think I'm after.