Geodesic regression

[Anita Sinha]

Does Manopt have the functionality to perform geodesic regression. I'm using covariance matrices computed from timeseries data and am looking for an efficient manifold regression scheme.

-Anita

[Nicolas Boumal]

Dear Anita,

Not as a built-in tool, though Manopt can certainly be used to help in the development of code that would do that.

I did some work on this 10 years ago, but that was pre-manopt and the code isn't user friendly unfortunately. Since then, lots of people have worked on regression on manifolds though, so there must exist some good tools out there but I'm not up to date. Some places to look:

 - The work of Pierre-Yves Gousenbourger: https://scholar.google.be/citations?user=M7URiy4AAAAJ&hl=fr

 - You may also be interested in Geomstats: a Python toolbox dedicated to statistics on manifolds, https://github.com/geomstats/geomstats.

Best,

Nicolas

[Ronny Bergmann]

Dear Anita,

there is also this nice paper by Tom https://hal.inria.fr/inria-00623920/document.

I have a code from a bachelor thesis doing something like that quite a while back, and I was planning to get this into Manopt.jl (so the Julia package) then it would not be that much to get it to Manopt/Matlab, but I did not yet find the time. The original code of the bachelors thesis is – like Nicolas code from before a time where I used Manopt (but its Matlab code), so I first have to refactor that. But it's on my list.

Best

Ronny

[Ronny Bergmann]

As a follow up, I started typing my notes into text and code, though only in Julia (and the second part is missing – but the basic algorithm is stated)

https://manoptjl.org/previews/PR79/tutorials/GeodesicRegression.html

The one thing one has to find out in Matlab/Manopt is how to compute this adjoint of the differential of the exponential map (compare also Formulae 7 and 8 in Fletcher), or maybe whether something like that can be computed using a retraction instead in the model and take the adjoint of the differential there.

In Geomstats (though that's then in Python) I am not yet aware of a finished regression.

[Nicolas Boumal]

That's an interesting point about the need for the differential of retractions and exponentials and for the adjoints of these differentials: they are currently unavailable in Manopt, but they could be useful here evidently, and in other contexts as well (this has come up before).

They tend to be a bit painful to figure out, which is even more motivation to try and integrate this in factories.

[Ronny Bergmann]

This is a slight side track, but: For me it felt easiest (also did that in MVIRT already) to implement Jacobi fields (and for the adjoint differential the adjoint Jacobi field) and then the specific (adjoint) differentials are “just” the right tangent space basis (to be precise a transported frame, usually an ONB where the first tangent vector points along the geodesic of interest for the Jacobi field) and the coefficients with respect to that basis, see https://manoptjl.org/stable/functions/Jacobi_fields.html#Manopt.jacobi_field (and the next one for a first coefficients set). Looking up this link, I might have to improve the documentation for the Jacobi field to cover more mathematical details in there. I like this technique to compute adjoint differentials (since they are very helpful to compute gradients).

Do you have literature for (adjoint) differentials of retractions? Until now I mainly cover exp/log (w.r.t. base point and argument) and geodesics (w.r.t. start and end point)

I think this could even be done outside of a factory, but the factory should provide the frame (ONB and curvature tensor eigenvalues), compare https://juliamanifolds.github.io/Manifolds.jl/stable/interface.html#ManifoldsBase.DiagonalizingOrthonormalBasis

[Nicolas Boumal]

For the exponential map I can see this working out with the Jacobi fields, that's very nice.

For retractions, since they are much more loosely defined, I think it'll have to be case by case (and as such, inside of factories).

Of course, we can always approximate these differentials with finite differences: sometimes that's enough to run some quick computations for a theoretical investigation (but it's completely impractical when the dimension of the manifold is somewhat large). Some tools in Manopt that can assist are: tangentorthobasis and operator2matrix. The first one can be used to generate a random orthonormal basis of a tangent space (say, one basis for T_x M and one basis for T_y M where y = R_x(v)). The second one can be used to compute the matrix that represents a linear operator from T_x M to T_y M with respect to the chosen bases. Since the bases are orthonormal, the adjoint is represented by the transposed matrix, so that's convenient. The tool operator2matrix requires the user to provide code for the operator in mat-vec form: that can be approximated with finite differences for example. Then, to switch back and forth between tangent vector representation and coefficients-in-the-bases representation, one case use lincomb and tangent2vec.

To be clear: the above is inefficient; but it's convenient from time to time.

[Anita Sinha]

Thank you for the thorough follow-up, I appreciate it. I primarily use Matlab and Python, so I think geomstats will be sufficient for my purposes. Thank you!

[Ronny Bergmann]

A concrete geodesic regression is prepared (but not yet finished) see https://github.com/geomstats/geomstats/issues/845 for details.