Re: [Knitro] Understanding why hessian sparsity is not used w/o exact hessian

[Richard Waltz]

Hi,

The hessian sparsity pattern would be useful for 2 cases:

1. The first case is when the user is providing the exact hessian.  It is useful in this case because the Knitro API expects the hessian to be provided as a sparse matrix and so we need to know the sparsity pattern (rows and column indices) to match them to the hessian values returned in the user hessian callback.
   
   
   
   
   
2. The second case is when the hessian is approximated via finite-differences.  Knitro does not provide an option to approximate the full hessian via finite-differencing because it is generally too expensive for large-scale problems. In the MATLAB example you reference, they are approximating the hessian via finite-differences, which is why they use the sparsity pattern here.
   

Knitro offers 4 methods for approximating the hessian, see:

https://www.artelys.com/docs/knitro//2_userGuide/derivatives.html#second-derivatives

https://www.artelys.com/docs/knitro/3_referenceManual/userOptions.html?highlight=hessopt#hessopt

The "bfgs" and "sr1" options are dense by construction so they cannot make use of the sparsity pattern (even when it might be available).

The "lbfgs" and "product_findiff" options do not explicitly store a hessian (approximation) so they cannot make use of the sparsity pattern.  The first is able to build an approximation from a couple arrays of data, and the second only approximates directly hessian-vector products (without storage of a hessian).

I hope this helps.

Regards,

-Richard Waltz

---

From: kni...@googlegroups.com <kni...@googlegroups.com> on behalf of Christian Cox <christi...@gmail.com>
Sent: Friday, May 19, 2023 9:18 AM
To: Artelys Knitro forum <kni...@googlegroups.com>
Subject: [Knitro] Understanding why hessian sparsity is not used w/o exact hessian

 

In the guide: https://www.artelys.com/docs/knitro/2_userGuide/derivatives.html it says that the hessian sparsity matrix is not used when the exact hessian is not provided. I don't quite understand why: if you have the exact hessian, then isn't the sparsity redundant? I thought the main usage of the sparsity matrix was to provide some information when the exact hessian is unknown.

This is how (it appears to me) Matlab fminunc uses it: https://www.mathworks.com/help/optim/ug/minimization-with-gradient-and-hessian-sparsity-pattern.html - here you provide the exact gradient and hessian sparsity.

Apologies if I am misunderstanding something simple here.

--
You received this message because you are subscribed to the Artelys "Knitro Nonlinear Optimization Solver" google group.
To post to this group, send email to kni...@googlegroups.com
To unsubscribe from this group, send email to knitro-un...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/knitro?hl=en
Thank You,
Artelys
http://www.artelys.com/en/optimization-tools/knitro
---
You received this message because you are subscribed to the Google Groups "Artelys Knitro forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to knitro+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/knitro/2277f6e3-c580-4955-aa46-911a12ac07d7n%40googlegroups.com.

[Christian]

Thanks for the explanation!