How to reduce the initial step size for fmincon
Felipe
I'm using the 'interior-point' algorithm and would like to know how to reduce the initial step size taken by the algorithm.
My practical problem is the following: resize a system to reduce its power consumption (cost function) while maintaining the same performance (non-linear constraint).
The initial solution (X0) satisfies the constraints and the optimal solution is expected to lay in the close vicinity of X0. As you can see below, Fmincon takes a large step at the first iteration and, then, slowly comes back towards X0.
################################################
# Starting sub-problem 1: Power
First-order Norm of
Iter F-count f(x) Feasibility optimality step
0 5 3.428240e+00 0.000e+00 2.021e+00
1 10 4.530210e-01 7.767e-01 5.080e+00 6.369e-01
2 16 6.209910e-01 5.751e-01 3.664e+00 2.578e-01
3 21 1.158880e+00 2.375e-01 7.423e-01 2.724e-01
4 27 1.307470e+00 1.775e-01 1.079e+00 6.966e-02
5 33 1.574470e+00 1.057e-01 7.093e-01 8.303e-02
6 38 2.238530e+00 4.978e-02 8.452e-01 1.438e-01
7 44 2.527530e+00 2.811e-02 1.916e+00 8.172e-02
8 50 2.752310e+00 1.654e-02 8.862e-01 6.678e-02
9 55 3.108490e+00 2.796e-03 2.198e+00 7.223e-02
10 61 3.134380e+00 1.728e-03 9.909e-01 2.268e-02
11 66 3.246230e+00 0.000e+00 1.294e+00 3.086e-02
#################################################
How can I modify the options as to reduce the initial step?
Thanks,
Felipe
Alan Weiss
well with "warm start," meaning they don't pay as close attention to the
initial point as they might.
I can think of two things to try:
1. Use the sqp algorithm for warm start
2. Set the InitBarrierParam option to a small value, say 1e-5, instead
of the default 1e-1.
Good luck,
Alan Weiss
MATLAB mathematical toolbox documentation
Felipe
Thanks for the reply. Indeed, the SQP algorithm ('active-set' in Matlab2009, right?) performs better, but changing the InitBarrierParam had no effect in reducing the step size. I'm running the tests using 2009b, but I could upgrade it to 2011a if there is any change in the implementation of SQP.
Also, in my problem, I'm chaining several optimizations (Gain, Bandwidth, Power). The solution of one is the starting point of the other. It might be possible for me to reuse the gradient and hessian information. But I'll have to figure out how to write a function that can return me the grad and hessian for all these objectives even if they might not appear en every problem.
If you have any tips, let me know.
Felipe FRANTZ
Alan Weiss <awe...@mathworks.com> wrote in message <iuani9$2tn$1...@newscl01ah.mathworks.com>...
Paul Kerr-Delworth
There is a new implementation of SQP in the Optimization Toolbox as of R2010a. For more information on this algorithm, see the release notes
<<http://www.mathworks.co.uk/help/toolbox/optim/rn/bsba32w-1.html#bsba34z>>
It may be worth upgrading so you can try out this algorithm for your problem.
Also, fmincon can also return an estimate of the gradient and hessian when it returns the solution. See the help for fmincon to see how to return these estimates
<<http://www.mathworks.co.uk/help/toolbox/optim/ug/fmincon.html>>
If your problem has nonlinear constraints, then you will have to use the interior point algorithm if you want to specify a Hessian. There is an example in the documentation on how to specify gradients and Hessians when using the interior point algorithm in fmincon
<<http://www.mathworks.co.uk/help/toolbox/optim/ug/brn4nh7.html#brv_i_1>>
Hope this helps!
Best regards,
Paul
"Felipe" wrote in message <iuc8do$54q$1...@newscl01ah.mathworks.com>...
Alan Weiss
The InitBarrierParam option applies only to the interior-point
algorithm. I am pretty sure that lowering its value to 1e-5 or so will
make your problem run faster when using the interior-point algorithm.
As Paul said, the sqp algorithm is not exactly the same as the
active-set algorithm. The two have similar behavior, for the most part,
but sqp is more robust and can be more accurate.
If you have Symbolic Math Toolbox, the easiest way to include gradient
and Hessian information is to use the procedure in this example:
http://www.mathworks.com/help/toolbox/optim/ug/brn4nh7.html#brv_i_1
Good luck,
Alan Weiss
MATLAB mathematical toolbox documentation
Felipe
For the 'interior-point' method, it turned out that I had to increase (instead of decreasing) the 'InitBarrierParam' value to get it to use smaller steps. I'm testing values between 1 to 10. Is this behavior counter-intuitive for you?
In any case, it did reduce the step size, but did not help it to converge faster (almost the same number of evaluations). For information, all my variables and objectives vary in the range 0 to 1.
For the writing of the Hessian function, it would be a little bit trickier. The performances I optimize come from an external circuit simulator. Moreover I'm chaining optimizations: what is the objective in the first become a constraint in the later. I'm thinking about something like copying what is done by fmincon for finite-differences, but adding a storage layer so that I can reuse the information across my problems. Pretty specific, isn't it? I guess I'm on my own on this one!
Best regards,
Alan Weiss <awe...@mathworks.com> wrote in message <iuclm7$bj9$1...@newscl01ah.mathworks.com>...
> I might have been misleading before.
> The InitBarrierParam option applies only to the interior-point
> algorithm. I am pretty sure that lowering its value to 1e-5 or so will
> make your problem run faster when using the interior-point algorithm.
>
> As Paul said, the sqp algorithm is not exactly the same as the
> active-set algorithm. The two have similar behavior, for the most part,
> but sqp is more robust and can be more accurate.
>
> If you have Symbolic Math Toolbox, the easiest way to include gradient
> and Hessian information is to use the procedure in this example:
> http://www.mathworks.com/help/toolbox/optim/ug/brn4nh7.html#brv_i_1
>
> Good luck,
>
> Alan Weiss
> MATLAB mathematical toolbox documentation
>
> >> > > How can I modify the options as to reduce the initial step?
> >> > > Thanks,
> >> > > Felipe