Positive SEMIdefinite metrics

[Jesus Briales]

Hi everyone!

I have a relatively simple question, so I'll try to keep it short and we can go into details if necessary.

Is it possible (does it make sense), in the context of the manifolds and solvers in Manopt, to define a metric that is positive semidefinite, with actual zero eigenvalues, rather than positive definite?

This question arose when I was wondering if I can define a metric so that the trust regions inner step does not bound the step length in certain (linear) components of my manifold.

Best,

- Jesus

[Nicolas Boumal]

Hello Jesus,

That's an interesting question -- and I understand your motivation regarding trust regions.

Endowing the manifold with a singular "metric" might break too many things in the current theory (it might be interesting to investigate separately.) Another perspective could be the following: you keep your proper metric, but you ask: can I use a singular preconditioner when I solve the trust-region subproblem (TRS)? This is essentially equivalent for your purpose, except "trouble" is confined to that question of the TRS.

Framed as such, it becomes a question of rather general interest, since the TRS lives on a linear space (the tangent space at the current iterate), so it's possible that this was already answered in the classical theory of trust regions. I would start by having a look at the TR bible: the book by Conn, Gould and Toint aptly named Trust Region Methods. Hopefully this will give something.

(There is an obvious difficulty here: if the Hessian or its approximation has a negative eigenvalue along the singular directions of your preconditioner, the TRS no longer has a solution...)

(Note that Manopt allows you to specify a preconditioner, and the trustregions solver will use it inside its tCG function, which is an implementation of the truncated Steighaug-Toint CG method, originally from the GenRTR implementation of Chris Baker et al.)

Best,

Nicolas

[Jesus Briales]

Thanks for your comments Nicolas!

Endowing the manifold with a singular "metric" might break too many things in the current theory (it might be interesting to investigate separately.) Another perspective could be the following: you keep your proper metric, but you ask: can I use a singular preconditioner when I solve the trust-region subproblem (TRS)? This is essentially equivalent for your purpose, except "trouble" is confined to that question of the TRS.

Framed as such, it becomes a question of rather general interest, since the TRS lives on a linear space (the tangent space at the current iterate), so it's possible that this was already answered in the classical theory of trust regions. I would start by having a look at the TR bible: the book by Conn, Gould and Toint aptly named Trust Region Methods. Hopefully this will give something.

I will have a look a it the moment I have a while, and try it in Manopt if possible.

(There is an obvious difficulty here: if the Hessian or its approximation has a negative eigenvalue along the singular directions of your preconditioner, the TRS no longer has a solution...)

Yes, I also thought about this, although in the particular case of application I had in mind I think there are some guarantees that those directions have always positive curvature.

I'll let you know when I check this :)

Best,

- Jesus