10 years of Manopt!

[BM]

Dear all, 

Wished to bring to your attention that Manopt has completed 10 years of its existence today. Manopt 1.0 was released on 03 Jan 2013. 

Personally, wanted to thank everyone for the support and encouragement and for maintaining an active forum.

@Nicolas, your thoughts? 

Regards,

Bamdev

[Ronny Bergmann]

Dear Bamdev,

interesting and Happy Birthday, Manopt!

Also thanks to you and Nicolas for starting that project.

It helped me a lot to get started with numerical differential geometry and implementing my first own algorithms back in Matlab (though I have moved on to a next language since then).

Best,

Ronny

[Nicolas Boumal]

Hello Bamdev,

Nice! I hadn't realized that. July 2022 also marked the 50th anniversary of optimization on manifolds, originating with Luenberger's seminal paper.

Back when we started this project, we made a number of conscious design choices. Here are four that served us well:

• We kept the code technically simple (no fancy OOP etc.) so as to facilitate tinkering (that is, research).
• We aimed for a low barrier to entry (both code and math), so that prospective users could easily get a "first feel" for what they can expect (and hopefully be intrigued enough to dig deeper).
• We isolated solvers and other tools from how manifolds represent points and tangent vectors, and from how users describe their cost function, gradients, etc. Over the years, this made it possible to seamlessly add features we had not foreseen, such as some GPU support, new manifolds, new algorithms, automatic differentiation, etc.
• We put effort in documentation and support (this forum), and we provided a number of diagnostics tools to help our users check their own code (checkgradient, checkhessian, checkmanifold, ...).

For the future, there are of course always many features we can add and things we can improve. We will, but I believe we should keep striving to "do few things really well". For Manopt, that is: provide robust computational tools for unconstrained optimization on smooth manifolds. Some bigger extensions (e.g., constrained optimization on manifolds, or optimization on nonsmooth sets through smooth parameterizations) might be better built as separate toolboxes that use Manopt as a building block.

To reinforce what we already do, I'm hopeful that we will find the resources to:

• Rethink how we describe manifolds.
  This is a frequent pain point for new users, in part because of the subtleties in distinguishing representations of points, tangent vectors, vectors in the embedding space, and horizontal vectors. This has many ramifications. There is room to clarify, but it's not trivial.
  On the Julia side, people build a separate toolbox entirely dedicated to manifolds (Manifolds.jl). That could serve as inspiration.
  
• Create more documentation by examples.
  Ideally, these would be duplicated for Manopt, PyManopt and Manopt.jl. Ronny already started an effort in that direction, but we need to allocate more resources to this.

Many, many thanks to all who contributed with code, bug reports, questions & answers on the forum, publicity and parallel projects (shout out to Ronny for Manopt.jl, and to Jamie, Niklas and Sebastian for PyManopt!) -- it's a lot better and a lot more fun with all of you.

We keep going :).

Best wishes,

Nicolas

[Ronny Bergmann]

Hi Nicolas,

thanks for the super-nice summary!

While in Julia there is no thing like OOP – for the rest of the points I actually completely follow your design choice motivations.

Maybe one thing we did differently is to even modularise the solver, such that stopping criteria and debug could be implemented easier/more general in Julia. But that was also due to the experience from before with other solver frameworks that followed the same path.

For the description of a manifold, Matlab might miss the concrete distingtion between these elements (that is “everything” is a matrix/vector in Matlab), so maybe such a clarification is mainly a way of thinking about how to best document / communicate these differences to the user(s)?

For the note in Manifolds.jl – thanks! – we even started to do a separate package ManifoldsDiff.jl – that focuses on computation of gradients/differentials/Hessians... – But I would say for both packages, that this only makes sense if one has in mind that multiple different tool(boxes) might use that.

For example in Python (with all its Riemannian packages) a common basis PyManifolds might be nice. In Matlab, a separate package might be useful, but for now might only be a split of the toolbox? Though the split might yield that – I don't know maybe – geometric integration might be based on a package like that.

For the examples – yes I would love to continue in that, just did not have the time recently (its https://github.com/kellertuer/ManifoldTutorials this page by the way), but I hope to continue on this one some time.

Concerning the thanks – well I would like to give that back as well – it is really a lot of fun working on these code bases (in all 3 languages).

Best wishes,

Ronny

[BM]

Thank you, Nicolas and Ronny, for your thoughts. 

I agree with the points discussed. 

IMHO, an impact of Manopt has been in providing easy access for researchers to try out constrained algorithms quickly for different problems, including unconstrained ones. All of these while maintaining a great collection of algorithms and tools. All credits go to great contributors and users. 

A recent interest of mine has been in differential private algorithms on manifolds [1, 2]. It seems it should be doable in Manopt, which I intend to work on in the coming days.

Regards,

Bamdev

[1] Differential Privacy Over Riemannian Manifolds (neurips.cc).

[2] [2205.09494] Differentially private Riemannian optimization (arxiv.org).

[Mark L. Stone]

How many Riemannian years is that?