scale=0 improves performance: What is wrong with my model?

[Herbert Meier]

Hello,

I encountered the (to me quite weird) situation that setting "scale=0" improves the performance of the solver significantly.

When letting scaling be its default value, I get:

Final objective value               =   1.10316742590580e+08
Final feasibility error (abs / rel) =   0.00e+00 / 0.00e+00
Final optimality error  (abs / rel) =   5.89e-01 / 5.89e-01
# of iterations                     =       1000
# of CG iterations                  =       2285
# of function evaluations           =       1696
# of gradient evaluations           =       1001
Total program time (secs)           =      43.74758 (    49.445 CPU time)
Time spent in evaluations (secs)    =      42.56104

Whereas scaling=0 delivers:

Final objective value               =   1.93055126320965e+02
Final feasibility error (abs / rel) =   0.00e+00 / 0.00e+00
Final optimality error  (abs / rel) =   3.33e+03 / 1.67e-01
# of iterations                     =       1000
# of CG iterations                  =       4844
# of function evaluations           =       1543
# of gradient evaluations           =       1001
Total program time (secs)           =      47.44957 (    53.134 CPU time)
Time spent in evaluations (secs)    =      46.15006

Could that be an indication that something is wrong with my model ?

I provide an analytic gradient to the objective and an analytic Jacobian to the constraints. When I run small instances of the model, the derivative check passes without troubles, however, for large instances of the same model, the derivative check is giving errors sometimes (error being in the region of 1e-2 instead of 1e-6). I assumed that is may be due to numerical issues / error accumulation due to the large instance of the model. Does that make sense? Could these issues be related to be above mentioned scaling observation?

What other causes could there be that turning off scaling delivers better results? Is it maybe just because he is going for more CG iterations if scaling is off and this helps to solve my non-convex problem?

Thanks so much for your help!

Regards,

Herbert

[Richard Waltz]

Dear Herbert,

Although it is not the usual case, on some small percentage of problems Knitro may converge faster with scaling turned off.  There is no obvious answer to why this may be the case.  It may be that the automatic scaling used by Knitro is not a good scaling in this case and has an adverse affect or it or it may just be that the problem is quite sensitive and it just gets lucky with scaling off.  Note that Knitro allows you to provide your own custom scaling which may often work better than the automatic scaling if you have knowledge about the expected scale of the objective and constraint functions in your model.

Regarding the derivative check, values on the order of 1e-4, 1e-5 and 1e-6 may often just be a result of numerical errors in the finite-difference check and not indicative of real errors.  Errors on the order of 1e-2 are getting quite big and could represent real errors.  However, if your model is badly scaled and/or highly nonlinear it's possible that the finite-difference approximation could have an error this large.  I recommend performing the gradient check using central differences via the "derivcheck_type" option.

https://www.artelys.com/tools/knitro_doc/3_referenceManual/userOptions.html#c.derivcheck_type

and see if the error values decrease.  If the errors decrease, then the errors are likely just due to the finite-difference approximation and likely do not represent real errors.  However you still may want to look more closely at a few of the elements that generate larger errors and just double-check the derivatives for these.

Best,

Richard

Richard WALTZ

Senior Scientist

 

Artelys USA

 

---

From: 'Herbert Meier' via Artelys Knitro forum [kni...@googlegroups.com]
Sent: Tuesday, September 04, 2018 2:58 AM
To: Artelys Knitro forum
Subject: [Knitro] scale=0 improves performance: What is wrong with my model?

--
You received this message because you are subscribed to the Artelys "Knitro Nonlinear Optimization Solver" google group.
To post to this group, send email to kni...@googlegroups.com
To unsubscribe from this group, send email to knitro-un...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/knitro?hl=en
Thank You,
Artelys
http://www.artelys.com/en/optimization-tools/knitro
---
You received this message because you are subscribed to the Google Groups "Artelys Knitro forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to knitro+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Herbert Meier]

Dear Richard,

Thanks for your great answer!

I will try to play with manual scaling then.

Out of curiosity, is there a way to figure out which internal scaling factors have been used by knitro in the case of automatic scaling?

Thanks & Regards,

Herbert

To unsubscribe from this group and stop receiving emails from it, send an email to knitro+u...@googlegroups.com.

[Herbert Meier]

A follow-up question: If I manually specify the scaling parameters e.g. for the variables only, will Knitro still adopt automatic scaling for the objective and constraints?

[Richard Waltz]

Hi Herbert,

Yes, if you choose the scale option 1 (user_internal),

https://www.artelys.com/tools/knitro_doc/3_referenceManual/userOptions.html?highlight=scale#c.scale

then it should use your manual scaling for variables, while using the Knitro internal scaling for the objective and constraints (if you do not specify a scaling for these).  If you use scale option 2 (user_none), then it would use your scaling for the variables, but no scaling for the objective and constraint (if you do not specify a scaling for these).

Unfortunately, there is no way currently to see the internal scaling factors used by Knitro.  We will make a note to consider exposing this information in future releases.

Best,

-RIchard

Richard WALTZ

Senior Scientist

 

Artelys USA

 

From: 'Herbert Meier' via Artelys Knitro forum [kni...@googlegroups.com]

Sent: Wednesday, September 05, 2018 9:46 AM
To: Artelys Knitro forum
Subject: Re: [Knitro] scale=0 improves performance: What is wrong with my model?

To unsubscribe from this group and stop receiving emails from it, send an email to knitro+un...@googlegroups.com.

[Mark L. Stone]

Seeing 1009 iterations for booth scaling settings seems to suggest that 1000 is the iteration limit, and no claim of convergence to an optimum occurred in either case.