Background knowledge needed for using Manopt

[taylor.c...@gmail.com]

I recently got introduced to Manopt when I encountered a problem requiring optimization over manifolds. My background is in physics and applied math (undergrad/grad) but I don't have any training in optimization or differential geometry. This package seems really useful but I can tell I'm missing some very important background knowledge. So now I'm wondering just how much background research I will need to do. I only plan to learn enough to solve my particular problem, as I don't have time to take 4 online classes and read a book or two (so hopefully that won't be needed......)

Without going into the problem I am trying to solve here I would like to know:
1. What would you consider the absolute minimum prerequisite knowledge to be able to formulate your own problems in Manopt.
2. What are the top 3 (5?) most important concepts in optimization over manifolds?
3. Any other need-to-know subjects or tips in general?

(I am asking this in a general way b/c I'm hoping that there is a general answer so that other folks new to the field can see this and know where to get started)

Any guidance would be much appreciated.

[Nicolas Boumal]

Hello Taylor,

Thanks for your interest. This is a fairly open question, which could take us far. Trying to keep it short:

1. I would recommend having a look at the first example in the tutorial. If you have the prerequisites, this should hopefully make sense: http://manopt.org/tutorial.html#firstexample

2. You want to understand what the notation "min f(x) subject to x \in M" means; what a manifold M is at a high level (think: a sphere; the set of orthonormal matrices of a certain size..); you want to understand the concept of "steepest descent" (or gradient descent) even in the classical, unconstrained case; you need to know what an orthogonal projection to a linear space is (the linear spaces will be the tangent spaces to the manifold); then you want to understand what a retraction is (it's a means to move away from a point x on M along a tangent direction u, so that R_x(u) is another point on the manifold M.) See this blog post: https://afonsobandeira.wordpress.com/2015/03/16/optimizing-in-smooth-waters-optimization-on-manifolds/

3. It's good to understand the notion of "necessary optimality conditions" (aka, KKT conditions). They are particuarly simple on manifolds: gradient must be zero, Hessian must be positive semidefinite. I often point younger students to appendix A in this paper: https://arxiv.org/pdf/1605.08101.pdf 

Best,

Nicolas

[taylor.c...@gmail.com]

Great! Thanks Nicolas!