Initial parameters to fminsearch function
Arash
I've been using fminsearch to optimize 5 parameters for an
error function. Three of the input parameters are used as
coefficients so their initial value is set to 1. The other
2 parameters are more like an offset to other variables so I
use the initial value of 0 so my function call looks like this:
OptArg=optimset('MaxFunEvals',10000,'MaxIter',10000,'TolX',1e-6,'TolFun',1e-6);
x0 = [1, 1, 1, 0, 0];
[x,MSE]=fminsearch(@pmodel_err,x0,OptArg);
where pmodel_err is my error function.
If I use a non zero inisial step such as:
x0 = [1, 1, 1, 10, 5];
I get good results, but if using zero initial values, the
final optimized x(4) and x(5) are close to zero. I
monitored the values that are passed to my error function as
the trial for x and with x0(4:5)=0 those values stay close
to zero.
It seems to me that fminsearch is sensitive to its initial
input parameters which is not a good thing for an
optimization tool. Does anybody have similar problem and
how do you go about fixing it?
Thanks,
Arash
John D'Errico
<fl0b0u$cto$1...@fred.mathworks.com>...
Virtually all optimizers are dependent on their
starting values. Some are more robust than other
others against convergence problems. But if an
optimizer is placed in the basins of attraction of
two distinct local minimizers, then you will get
two different results.
Nothing that you have said has yet convinced me
that your objective function is well posed. For
example, from your meager description, consider
the model
y = (a0 + a1)*x
From data, estimate both a0 and a1 as coefficients
in the model. Here a1 might be described as an
"offset" to a0. Can you uniquely estimate both
a0 and a1?
If you really do have a validly posed optimization
problem (hey, it does happen, 8-) then you might
consider the application of constraints on the
problem. Surely you know something about this
system, since you are unhappy with the results
in at least some of your efforts. So why not apply
constraints that embody your knowledge of this
system?
Fminsearch is a rather basic tool, so you might
also consider using a different code, perhaps
one from the optimization toolbox.
John
Arash
message <fl0dis$4sl$1...@fred.mathworks.com>...
John,
Thanks as always for your reply.
My function a model to generate blood pressure waveform and
then compare it with the actual measured blood pressure.
The model is well defined and the fit for the model vs. the
measured waveforms are good. My function returns the MSE of
these 2 waveforms. I ended up suing non zero initial values
but I have to justify why I expect the offset in first
place; it would be much nicer if I could start offsets from
zero. If I pick a non zero offset (1 or 10 or 100 -- units
don't matter), fminsearch converges to the same results, but
using zero or close to zero initial value takes it somewhere
else.
I don't have the optimization toolbox and it's not that easy
to put constraints on fminsearch. Is there any other
alternatives outside of optimization toolbox for that?
Thanks,
Arash
John D'Errico
> My function a model to generate blood pressure waveform and
> then compare it with the actual measured blood pressure.
> The model is well defined and the fit for the model vs. the
> measured waveforms are good. My function returns the MSE of
> these 2 waveforms. I ended up suing non zero initial values
> but I have to justify why I expect the offset in first
> place; it would be much nicer if I could start offsets from
> zero. If I pick a non zero offset (1 or 10 or 100 -- units
> don't matter), fminsearch converges to the same results, but
> using zero or close to zero initial value takes it somewhere
> else.
>
> I don't have the optimization toolbox and it's not that easy
> to put constraints on fminsearch. Is there any other
> alternatives outside of optimization toolbox for that?
But if you start an optimizer in the wrong basin
of attraction, it will converge to the wrong solution.
Deliberately giving it poor starting values will not
produce an unbiased solution, just the wrong one.
An optimizer is just a tool that tries to walk downhill
from your starting point. It cares not that the downhill
direction leads it into the wrong valley.
There are a few tools on the FEX that will allow you
to impose constraints. My own fminsearchbnd and
fminsearchcon are two simple ones, built on top of
fminsearch. There is also now an fminsearchbnd_new
that improves on a few things with the newer releases
of matlab that have come out since I wrote those tools.
http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?
objectId=8277&objectType=file
http://www.mathworks.com/matlabcentral/fileexchange/loadCategory.do?
objectId=5&objectType=Category
John
Serge
It seems the way fminsearch seeds the problem is undocumented. When you give fminsearch a stating point, it needs to generates nD + 1 points (see Wikipedia - Nelder Mead as to why). It does this by increasing each parameter by 5% from the initial value given, if however a parameter value is exactly zero it sets it to 2.5e-4. eg for a 3D problem if starting point given by the user is x0 = [2 1 0] then the 4 points generated are [2 1 0],[2.1 1 0],[2 1.05 0],[2 1 2.5e-4].
How quickly Nelder Mead finds the solution depends on all these points, depending on your problem these default step sizes may be too small or too large to get to the minimum efficiently. I have not figure out how to specify all the points manually, but i haven't really tried. I know that setting TolX in optimset does not appear to have an effect. I used MatLab 2010b.
Why this is not documented anywhere is beyond me.
Alan Weiss
> "Ari" wrote in message <fl0b0u$cto$1...@fred.mathworks.com>...
>> Hi,
>>
>> I've been using fminsearch to optimize 5 parameters for an
>> error function. Three of the input parameters are used as
>> coefficients so their initial value is set to 1. The other
>> 2 parameters are more like an offset to other variables so I
>> use the initial value of 0 so my function call looks like this:
>> Thanks,
>> Arash
>
> I know I am 10 years late but...
>
> It seems the way fminsearch seeds the problem is undocumented. When you
> give fminsearch a stating point, it needs to generates nD + 1 points
> (see Wikipedia - Nelder Mead as to why). It does this by increasing each
> parameter by 5% from the initial value given, if however a parameter
> value is exactly zero it sets it to 2.5e-4. eg for a 3D problem if
> starting point given by the user is x0 = [2 1 0] then the 4 points
> generated are [2 1 0],[2.1 1 0],[2 1.05 0],[2 1 2.5e-4].
>
> How quickly Nelder Mead finds the solution depends on all these points,
> depending on your problem these default step sizes may be too small or
> too large to get to the minimum efficiently. I have not figure out how
> to specify all the points manually, but i haven't really tried. I know
> that setting TolX in optimset does not appear to have an effect. I used
> MatLab 2010b.
>
> Why this is not documented anywhere is beyond me.
http://www.mathworks.com/help/matlab/math/optimizing-nonlinear-functions.html#bsgpq6p-11
Alan Weiss
MATLAB mathematical toolbox documentation