FIMINCON, TOLX and TOLFUN: I don't understand how they work 8-(
Gabriele
I'm trying to minimize a function of 15 variables, each of them
bounded between 0 and 1. For this I use fmincon, starting from a guess
point given me by a genetical algorithm.
My problem is that many times fmincon hangs, and I've seen that this
is a quite common problem.
In order to avoid hanging, I reduced the numebr of iterations, but
this is not the right choice for me, so I tried to change the TOLX and
TOLFUN parameter.
My objective function to be minimized has values near 200, so I tried
to change TOLFUN from the default value to 0.1:
options=optimset('Display','iter','Tolfun',0.1)
then I run fmincon as follows:
X=fmincon('fp_gatefn',bestx_norm,[],[],[],[],zeros(15,1),ones(15,1),[],options)
Setting Tolfun to 0.1, I thought that a differences of the order of
0.1 in successive function evaluation would have caused the algorithm
to stop, but this is what I obtain:
Iter F-count f(x) constraint Step-size
derivative Procedure
1 16 -191.959 -0.06567 1 -566
2 37 -195.554 -0.06156 0.0625 -530
3 63 -196.587 -0.06144 0.00195 -375
4 93 -196.633 -0.06156 0.000122 -495
5 114 -197.206 -0.1134 0.0625 -463
6 139 -197.365 -0.113 0.00391 -234
7 159 -202.683 -0.09885 0.125 -324
8 182 -203.699 -0.09731 0.0156 -306
9 203 -209.17 -0.09123 0.0625 -322
10 227 -213.417 -0.09052 0.00781 -413
11 257 -213.467 -0.0905 0.000122 -295
12 289 -213.476 -0.0905 3.05e-005 -403
13 325 -213.477 -0.0905 1.91e-006 -344
14 361 -213.478 -0.0905 1.91e-006 -384
15 401 -213.478 -0.0905 1.19e-007 -232
16 443 -213.478 -0.0905 2.98e-008 -239
17 485 -213.478 -0.0905 2.98e-008 -204
18 513 -213.574 -0.09046 0.000488 -162
Error in ==> C:\MATLABR11\toolbox\optim\private\qpsub.m
On line 345 ==> if strcmp(dirType, NewtonStep)
Error in ==> C:\MATLABR11\toolbox\optim\private\nlconst.m
On line 380 ==> [SD,lambda,exitflagqp,outputqp,howqp] ...
Error in ==> C:\MATLABR11\toolbox\optim\fmincon.m
On line 413 ==> [X,FVAL,lambda,EXITFLAG,OUTPUT,GRAD,HESSIAN]=...
As you can see I need to stop the function through CTRL-C. My question
now is: why fmincon has not stop after the 10th iteration? In
successive steps the difference in function evaluation is less than
the tolerance I set.
In a successive test I tried to change the value of TOLX, from the
default valuie (1e-6) to a value of 1e-2:
options=optimset('Display','iter','TolX',1e-2)
then I run fmincon, and that is what I obtained:
Iter F-count f(x) constraint Step-size
derivative Procedure
1 16 -191.959 -0.06567 1 -566
2 37 -195.554 -0.06156 0.0625 -530
3 63 -196.587 -0.06144 0.00195 -375
4 93 -196.633 -0.06156 0.000122 -495
5 114 -197.206 -0.1134 0.0625 -463
6 139 -197.365 -0.113 0.00391 -234
7 159 -202.683 -0.09885 0.125 -324
8 182 -203.699 -0.09731 0.0156 -306
9 203 -209.17 -0.09123 0.0625 -322
10 227 -213.417 -0.09052 0.00781 -413
11 257 -213.467 -0.0905 0.000122 -295
12 289 -213.476 -0.0905 3.05e-005 -403
13 325 -213.477 -0.0905 1.91e-006 -344
14 361 -213.478 -0.0905 1.91e-006 -384
15 401 -213.478 -0.0905 1.19e-007 -232
16 443 -213.478 -0.0905 2.98e-008 -239
17 485 -213.478 -0.0905 2.98e-008 -204
18 513 -213.574 -0.09046 0.000488 -162
Error in ==> C:\MATLABR11\toolbox\optim\private\qpsub.m
On line 520 ==> ind = 0;
Error in ==> C:\MATLABR11\toolbox\optim\private\nlconst.m
On line 380 ==> [SD,lambda,exitflagqp,outputqp,howqp] ...
Error in ==> C:\MATLABR11\toolbox\optim\fmincon.m
On line 413 ==> [X,FVAL,lambda,EXITFLAG,OUTPUT,GRAD,HESSIAN]=...
As you can see I need to use CTRL-C in order to stop the hanged
functions, and I don't understand why fmincon hasn't stop at least
near the 11th iteration, where the step-size is less than the TOLX
value I set (0.01).
Maybe I haven't understood how the TOLFUN and TOLX parameters work,
but I can't find any exaplanation neither in the mathworks site nor in
the old articles of the newsgroup, so I decided to ask directly here.
Many thanks in advance...
Sorry for my poor english.
Ciao e 73-51 de Tartaruga .
.oO-=> TARTARUGA (* Gabriele *) <=-Oo.
E-Mail: ru...@libero.it
Packet CB: http://www.geocities.com/ita490/
Astronomy: http://www.geocities.com/rugabri/
"Chi dorme non piglia pesci, ma chi non dorme, alla fin fine...muore..."
(C) Tartaruga 1999 ;-)
Hiu Chung Law
My constraint is
sum(x.*x) - 1 = 0
So I create a small function with it and get the result I want.
However, when I try to make the optimization faster by providing the
gradient information of the constraint (2*x), fmincon turns
out to be slower and is stuck in a very poor local minimum.
This looks weird to me, because providing extra information turns out
hurt the actual optimization.
However, this is only the first time I use the optimization toolbox.
So maybe there is some subtlety that I am not aware of.
Nabeel
I haven't had a chance to look at this thread thoroughly, but:
> However, when I try to make the optimization faster by providing the
> gradient information of the constraint (2*x), fmincon turns
> out to be slower and is stuck in a very poor local minimum.
This usually happens when the gradient is incorrectly specified. Turn
on the gradient checking, and see if the optimizer throws a warning
about your gradient being incorrect.
Otherwise, providing the gradient should significantly speed up the
optimization. If the gradient is correctly specified, post the problem
so others can take a look and see what's happening.
-- Nabeel
Peter Spellucci
constraint violation decreases slowly. devil knows what they are doing
there (first set 'largescale' to 'off') since it is really strange to
see a code that is not able to maintain boxconstraints.
hth
peter
Mary Ann Branch Freeman
Try changing the TolX and TolFun at the same time, not in separate
runs.
Is your function continuous? There are some continuity/smoothness
assumptions. It's usually when these are violated that the behaviour
you describe happens.
Mary Ann
In article <3cb0acec...@powernews.libero.it>, ru...@libero.it
says...
Mary Ann Branch Freeman
spel...@mathematik.tu-darmstadt.de says...
Peter,
actually from the output it shows the constraints are satisfied
through out the run: negative values mean no constraints are active,
that it's in the interior.
Mary Ann
Randy Poe
> then I run fmincon as follows:
>
> X=fmincon('fp_gatefn',bestx_norm,[],[],[],[],zeros(15,1),ones(15,1),[],options)
>
> Setting Tolfun to 0.1, I thought that a differences of the order of
> 0.1 in successive function evaluation
I'm not sure that this is what Tolfun means, but perhaps
somebody at Mathworks can address this question.
Can you describe your objective function or provide code if
it's not too long?
As I use fmincon heavily, I'm very curious about the next
chapter in this story.
I too haven't figured out the termination criteria. My
runs always go to MaxIter or MaxFunEvals even when I
can clearly see by the output values that a minimum has
been reached.
- Randy
Gabriele
Branch Freeman) wrote:
>Hi
>
>Try changing the TolX and TolFun at the same time, not in separate
>runs.
>
>Is your function continuous? There are some continuity/smoothness
>assumptions. It's usually when these are violated that the behaviour
>you describe happens.
My function is surely continuous...not very smooth, but surely not
hill...
>Mary Ann
Gabriele
wrote:
>I'm not sure that this is what Tolfun means, but perhaps
>somebody at Mathworks can address this question.
>
>Can you describe your objective function or provide code if
>it's not too long?
The code is long, so I can't post it there. But I could say that the
function evaluate, in some manner, the distance covered during a man's
walk. I use the fmincon in order to maximize this distance changing
some parameters that control the way the man walk.
> - Randy
Randy Poe
>
> On Mon, 08 Apr 2002 17:16:37 -0400, Randy Poe <rp...@atl.lmco.com>
> wrote:
> >Can you describe your objective function or provide code if
> >it's not too long?
>
> The code is long, so I can't post it there. But I could say that the
> function evaluate, in some manner, the distance covered during a man's
> walk. I use the fmincon in order to maximize this distance changing
> some parameters that control the way the man walk.
What are those parameters, how is the walk represented,
and how is the distance evaluated?
I'm trying to get enough information to assess the
smoothness/differentiability issue.
- Randy
Gabriele
wrote:
>Gabriele wrote:
>>
>> On Mon, 08 Apr 2002 17:16:37 -0400, Randy Poe <rp...@atl.lmco.com>
>> wrote:
>> >Can you describe your objective function or provide code if
>> >it's not too long?
>>
>> The code is long, so I can't post it there. But I could say that the
>> function evaluate, in some manner, the distance covered during a man's
>> walk. I use the fmincon in order to maximize this distance changing
>> some parameters that control the way the man walk.
>
>What are those parameters, how is the walk represented,
>and how is the distance evaluated?
The walk is represented by 5 mean angles, 5 angle amplitudes and 5
phase angles. The man is represented by his bust and his legs (divided
in the upper and lower part): in total 5 parts. Each part oscillates
around its mean angle with its amplitude and phase. The oscillation
frequency is only a time scale parameter.
The time goes from 0 to a multiple of the oscillation period. In the
walk period an objective function is evaluated: this function
calculates the global distance covered by the barycenter of the man
and evaluate the distance convered in "quasi static equilibrium",
i.e., the distance covered with the barycenter projection between
feet. The objective function is obtain by means of the product of this
two distances, and, at the end, the results is divided by the total
period.
>I'm trying to get enough information to assess the
>smoothness/differentiability issue.
> - Randy
Gabriele
>Hi everybody, I've the following problem.
[...]
>Maybe I haven't understood how the TOLFUN and TOLX parameters work,
>but I can't find any exaplanation neither in the mathworks site nor in
>the old articles of the newsgroup, so I decided to ask directly here.
>Many thanks in advance...
In these days I often had the same problem, so I tried to take a look
at the code, and I have seen something that looks me strange: FMINCON
calls NLCONST. In this call the OPTIONS variable is passed from
FMINCON to NLCONST. Then NLCONST calls QPSUB, but it does not pass the
OPTIONS structure to QPSUB, so the variable 'maxiter' in QPSUB is
always set at the default value 'maxSQPiter' that, unfortunately, is
set to 'inf', because in QSPSUB the 'options' variable is always an
empty structure.
Changing the value of maxSQPiter from 'inf' to a resonable low number,
that is, 50, I've seen that I could avoid the classical hangs of
fmincon in QPSUB, but I think that setting a quasi-random ;-) value,
like 50, for 'maxSQPiter' is not the right way, because the control of
the tolerance should be passed from FMINCON to QPSUB through NLCONST
using the OPTIONS variable.
Setting 50 as value for 'maxSQPiter', instead of waiting fo an hanging
code, I receive the following information from fmincon:
Iter F-count f(x) constraint Step-size
derivative Procedure
1 16 -5.69684 -0.1127 1 -32.2
2 50 -5.69708 -0.1127 7.63e-006 -27.6
3 88 -5.6971 -0.1127 4.77e-007 -26.1
4 126 -5.69711 -0.1127 4.77e-007 -25.8
5 164 -5.69712 -0.1127 4.77e-007 5.28
ill-conditioned
6 182 0.544624 -0.06878 0.5 -11.6
ill-conditioned
7 199 0.00134663 5.551e-017 1 0.215
ill-conditioned
8 216 1.53459e-013 0 1 -1.59e-011
Hessian modified twice; ill-conditioned
Optimization terminated: No feasible solution found.
Magnitude of directional derivative in search direction
less than 2*options.TolFun but constraints are not satisfied.
Setting 'maxSQPiter' to 'inf' the code hangs at the 5th step.
Why in the last two step the function is wrongly evaluated ? (it can't
returns values like the three in the 6th, 7th and 8th step inside the
boundary fixed in FMINCON)
I don't know how te code works, so I shouldn't speak ;-), but I would
like to know why the OPTIONS structure is not passed from NLCONST to
QPSUB...is this right?
Many thanks...
Randy Poe
>
> On Sun, 07 Apr 2002 20:43:42 GMT, ru...@libero.it (Gabriele) wrote:
>
> >Hi everybody, I've the following problem.
> [...]
> >Maybe I haven't understood how the TOLFUN and TOLX parameters work,
> >but I can't find any exaplanation neither in the mathworks site nor in
> >the old articles of the newsgroup, so I decided to ask directly here.
> >Many thanks in advance...
>
> In these days I often had the same problem, so I tried to take a look
> at the code, and I have seen something that looks me strange: FMINCON
> calls NLCONST. In this call the OPTIONS variable is passed from
> FMINCON to NLCONST. Then NLCONST calls QPSUB, but it does not pass the
> OPTIONS structure to QPSUB, so the variable 'maxiter' in QPSUB is
> always set at the default value 'maxSQPiter' that, unfortunately, is
> set to 'inf', because in QSPSUB the 'options' variable is always an
> empty structure.
Mathworks folks, are you listening? Is there a technical
article addressing Gabriele's concerns? Or mine? My optimization
problems usually converge pretty well, but I too am wondering
about the OPTIONS for termination. I always end at MaxIter
or MaxFunEvals, never by tolerance criteria.
Can you at least tell us exactly what the test for TolFun
and TolX termination is, so we can define proper settings?
- Randy
Hiu Chung Law
on TolCon, TolFun and TolX (from "help optimset") pretty obscure.
It will be very helpful if mathwork can explain more on how
these tolerances work. Thank you.
TolCon - Termination tolerance on the constraint violation [ positive scalar ]
TolFun - Termination tolerance on the function value [ positive scalar ]
TolX - Termination tolerance on X [ positive scalar ]
Randy Poe
> Randy Poe <rp...@atl.lmco.com> wrote:
> > Mathworks folks, are you listening? Is there a technical
> > article addressing Gabriele's concerns? Or mine? My optimization
> > problems usually converge pretty well, but I too am wondering
> > about the OPTIONS for termination. I always end at MaxIter
> > or MaxFunEvals, never by tolerance criteria.
>
> > Can you at least tell us exactly what the test for TolFun
> > and TolX termination is, so we can define proper settings?
>
> I second Randy's request. I also find the following documentation
> on TolCon, TolFun and TolX (from "help optimset") pretty obscure.
> It will be very helpful if mathwork can explain more on how
> these tolerances work. Thank you.
>
> TolCon - Termination tolerance on the constraint violation [ positive scalar ]
> TolFun - Termination tolerance on the function value [ positive scalar ]
> TolX - Termination tolerance on X [ positive scalar ]
>
OK, my most recent run made a liar out of me.
On previous runs, I've gotten to what looks like
termination (small step sizes ~1E-6, function value not
changing for 1000s of iterations, etc). It continues in
what looks like a constant state till MaxFunEvals or
MaxIter expires.
Half an hour ago I got a termination with "optimization
terminated successfully". I guess that means my previous
optimizations were not actually optimal.
Here's the message:
"Search direction less than 2*options.TolX and
maximum constraint violation is less than options.TolCon"
My guess as the meaning of the first test is that
it is related to the norm of the gradient vector
or B*grad(f(x)) where B=Hessian inverse. So the test
would be more on the gradient rather than whether
the function itself is not changing.
If we want to force termination when the search appears
to be stuck, I'm not sure how to do that with the
available criteria.
- Randy
Gabriele
wrote:
>If we want to force termination when the search appears
>to be stuck, I'm not sure how to do that with the
>available criteria.
With different minimization test I obtained your same results, and I
agree with the fact that this type o termination rule is difficulty
controllable, IMHO...