Re: [Knitro] Understanding why hessian sparsity is not used w/o exact hessian

20 views
Skip to first unread message

Richard Waltz

unread,
May 19, 2023, 1:43:31 PM5/19/23
to Artelys Knitro forum
Hi,

The hessian sparsity pattern would be useful for 2 cases:

  1. The first case is when the user is providing the exact hessian.  It is useful in this case because the Knitro API expects the hessian to be provided as a sparse matrix and so we need to know the sparsity pattern (rows and column indices) to match them to the hessian values returned in the user hessian callback.


  2. The second case is when the hessian is approximated via finite-differences.  Knitro does not provide an option to approximate the full hessian via finite-differencing because it is generally too expensive for large-scale problems. In the MATLAB example you reference, they are approximating the hessian via finite-differences, which is why they use the sparsity pattern here.
Knitro offers 4 methods for approximating the hessian, see:



The "bfgs" and "sr1" options are dense by construction so they cannot make use of the sparsity pattern (even when it might be available).

The "lbfgs" and "product_findiff" options do not explicitly store a hessian (approximation) so they cannot make use of the sparsity pattern.  The first is able to build an approximation from a couple arrays of data, and the second only approximates directly hessian-vector products (without storage of a hessian).

I hope this helps.

Regards,
-Richard Waltz



From: kni...@googlegroups.com <kni...@googlegroups.com> on behalf of Christian Cox <christi...@gmail.com>
Sent: Friday, May 19, 2023 9:18 AM
To: Artelys Knitro forum <kni...@googlegroups.com>
Subject: [Knitro] Understanding why hessian sparsity is not used w/o exact hessian
 
In the guide: https://www.artelys.com/docs/knitro/2_userGuide/derivatives.html it says that the hessian sparsity matrix is not used when the exact hessian is not provided. I don't quite understand why: if you have the exact hessian, then isn't the sparsity redundant? I thought the main usage of the sparsity matrix was to provide some information when the exact hessian is unknown.

This is how (it appears to me) Matlab fminunc uses it: https://www.mathworks.com/help/optim/ug/minimization-with-gradient-and-hessian-sparsity-pattern.html - here you provide the exact gradient and hessian sparsity.

Apologies if I am misunderstanding something simple here.

--
You received this message because you are subscribed to the Artelys "Knitro Nonlinear Optimization Solver" google group.
To post to this group, send email to kni...@googlegroups.com
To unsubscribe from this group, send email to knitro-un...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/knitro?hl=en
Thank You,
Artelys
http://www.artelys.com/en/optimization-tools/knitro
---
You received this message because you are subscribed to the Google Groups "Artelys Knitro forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to knitro+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/knitro/2277f6e3-c580-4955-aa46-911a12ac07d7n%40googlegroups.com.

Christian

unread,
May 19, 2023, 3:18:29 PM5/19/23
to Artelys Knitro forum
Thanks for the explanation!
Reply all
Reply to author
Forward
0 new messages