ehess2rhess – Literature & Documentation?

[Ronny Bergmann]

Hi,

I am currently trying to understand how the ehess2rhess functions work.

I think I understood formula (5.41) in the recent book by you, Nicolas, as well as all its ingredients.

But when I implement that, I usually need at least two projections and a Weingarten map, but most ehess2rhess functions in Manopt are just 3 lines – I have no clue which simplifications are done to get to those.

For example for the Stiefel manifold. That is 3 matrix summands in the implementation while mine would need – 9?

Is that somewhere documented how the 2 projections and the Weingarten map simplify that much?

Second, for manifolds like Hyperbolic or SymmetricPositiveDefinite, I think Section 5.11 can not be used, since they do not inherit the metric from their embedding? (At least for gradients I had to do extra Steps compared to Proposition 3.61). Can someone maybe provide references for a Hessian conversion when the manifold is embedded but does not inherit the metric from the embedding? I would be very interested in that.

Best

Ronny

[Ronny Bergmann]

To answer that partly myself – quite some theory and several manifolds are covered in the paper https://arxiv.org/abs/2009.10159 – where for now I mainly miss to understand the alpha-range of embedded Stiefel manifolds and how their conversion works (which includes the Canonical Metic; I do understand the Euclidean case, for sure).

Best

Ronny

[Nicolas Boumal]

Hello Ronny,

For Riemannian submanifolds, Eq. (5.41) (which gives the Riemannian Hessian in terms of the Euclidean Hessian + a correction term involving the Weingarten map) is informative theoretically but (as you saw in the Manopt code) we seldom use it "as is" to convert Euclidean derivatives into Riemannian derivatives. Rather, we work out the Riemannian Hessian "directly" and obtain a simplified formula "automatically". For example, see (7.29) for Stiefel. Of course, you would obtain the same formula via (5.41), but perhaps it would not be immediately obvious how to simplify it until you get (7.29) (or maybe it is: I did not try).

Long story short: for Riemannian submanifolds, both a direct computation as in (7.29) or using the general formula as in (5.41) work. The former requires you to know something about the connection; the latter requires you to know something about the Weingarten map. Of course, mathematically those are interchangeable (for submanifolds), but one may be more convenient than the other computationally.

About your second question, regarding submanifolds which are not Riemannian submanifolds: in those cases, we cannot use the simple general formulas as above, and we need to work from first principles: see the definitions of gradient and Hessian in Chs 3 and 5. Section 7.6 details how to do that for hyperbolic space (in the hyperboloid model, where the manifold in embedded in R^(n+1), and it does inherit a metric from R^(n+1) but it's the Minkowski metric, not a Euclidean one). It turns out that you can still pretty easily figure out the gradient, connection and Hessian in that case. For positive definite matrices, there are some results in Section 11.7, but few details for how to figure out the connection; there is a pointer to a paper by Sra & Hosseini, SIAM Opt. 2015, Section 3.

I hope this helps -- happy to answer more questions.

Best,

Nicolas

[Ronny Bergmann]

Hello Nicolas,

Yes, (5.41) was quite helpful for the sub manifolds, we even started that by introducing a `Weingarten` function, cf

https://juliamanifolds.github.io/ManifoldsBase.jl/dev/functions/#ManifoldsBase.Weingarten-Tuple{AbstractManifold,%20Any,%20Any,%20Any}

Oh, I did miss (7.29) but Du Nguyen (see linked paper above) was kind enough to. explain a few things so I could use his approach, that allows a change of metric. I think by now I even have the Canonical and \alpha (or Submersion metric, mainly developed by Hüper & colleagues I think) metric. The paper by Nguyen also states the SPD Hessian, but in the end that is probably similar to Sra&Hosseini as well.
Similarly for Hyperbolic, Du Nguyen helped a bit (to convert from Euclidean) but you are right, using the Lorenzian space/Minkowski metric in the embedding is of course an equivalent approach.

Overall, I am slowly getting where I want. Here is a sneak preview (still on a pull-request branch) of how it should look in the end

https://juliamanifolds.github.io/Manifolds.jl/previews/PR644/manifolds/hyperbolic.html#ManifoldDiff.riemannian_Hessian-Tuple{Hyperbolic,%20Vararg{Any,%204}}but some typos are still present and some details and especially tests are still missing.

Thanks for the comments :)

Ronny

[Pierre Absil]

To partially address Ronny's question: the ehess2rhess function in https://www.manopt.org/reference/manopt/manifolds/stiefel/stiefelfactory.html (i.e., the Stiefel manifold as a Riemannian submanifold of R^nxp equipped with the usual Frobenius inner product) can be obtained from the concise paper https://sites.uclouvain.be/absil/2013-01/Weingarten_07PA_techrep.pdf by combining (7) with the first line of the long displayed equation on page 5.

Be mindful that the published version has a typo: see https://sites.uclouvain.be/absil/2013.01 for details.

Best,

PA.

[Ronny Bergmann]

Hi Pierre-Antoine,

that is exactly what I started with and by now I also already understood why that simplifies again, but at first I ended up with a lot of (6 or so) summands.

Besides that – your paper and Section 5.11 from Nicolas where the reason I went for an implementation of the Weingarten map, which for the sub manifold case really yields nice code in the end, which allowed for this nice default

https://juliamanifolds.github.io/ManifoldDiff.jl/stable/library/#ManifoldDiff.riemannian_Hessian-Tuple{AbstractManifold,%20Any,%20Any,%20Any,%20Any}

or in code

https://github.com/JuliaManifolds/ManifoldDiff.jl/blob/d30c34a3a5e756e083e0164a67a9c00094910aa7/src/riemannian_diff.jl#L365-L369

so your paper was a really great start for that :)

Best,

Ronny

[Ronny Bergmann]

Good evening

To shortly follow up in this: We not have the `riemannian_Hessian` in Manifolds.jl, that turns a Euclidean Hessian evaluation into a Riemannian one – for a few manifolds at least.

The reason I started that is, that in Manopt.jl one can now state the keyword `objective_type=:Euclidean` and all parts of the objective (cost, grad, Hess) are treated as being defined in the embedding.

For the cost this means before calling f(p) the p is embedded, for grad and Hess additionally said conversion.

You can see an example in the tutorial https://manoptjl.org/stable/tutorials/EmbeddingObjectives/; the interesting code block is the one starting with r2 = .

Thanks for the help here :)

Kind regards,

Ronny