Question Regarding the Product Manifold Hessian

[gran...@gmail.com]

I'm a bit confused by the way Product manifolds are handled. Suppose I have a product manifold RxR such that R is normal euclidean space, and I have a simple function f(x,y) = (x*y,x*y) then the euclidean hessian is just {(0,1),(1,0)}. However, if I am reading the code right the Riemannian Hessian would be zero.

Is it the case that the Riemannian Hessian is 'block diagonal' for product manifolds?

Regards,

Grant

[Nicolas Boumal]

Hello Grant,

In Manopt, the user provides Hessians as operators, not as matrices. By this, I mean that you should provide code to compute H*v, where H is the Hessian and v is a vector (a direction) that the solver would specify. You do not need to provide code to compute H itself. In all cases I know of, this way is more convenient and more efficient.

For your example where R = euclideanfactory(1) and M = R x R, points and tangent vectors are represented as a structure. For example,

R = euclideanfactory(1);

elements.x = R;

elements.y = R;

M = productmanifold(elements);

Points on the manifold M are structures with two fields, called "x" and "y". Each field contains a point on its respective manifold (in this case, both manifolds are simply R.) So, if P is a point, then P.x and P.y are the two corresponding real numbers.

Likewise, tangent vectors to a point of M are represented as structures, also with two fields called "x" and "y": each field corresponds to a tangent vector to the corresponding point of the corresponding manifold. So, if V is a tangent vector at P, then V.x is tangent to the first manifold at P.x, and V.y is tangent to the second manifold at P.y.

Let's illustrate this with a simple cost function (I'm not using f in your message because I need a function whose output is a single real number; otherwise, it doesn't have a gradient and Hessian.)

Here, f(x, y) = x²y.

Its gradient is grad f(x, y) = (2xy, x²).

Its Hessian, as an operator at (x, y) applied to the tangent direction (u, v), is: Hess f(x, y)[u, v] = (2uy + 2xv, 2xu).

problem.M = M;

problem.cost = @(P) P.x^2 * P.y;

problem.grad = @(P) struct('x',  2*P.x*P.y,  'y', P.x^2);

problem.hess = @(P, V) struct('x', 2*V.x*P.y + 2*P.x*V.y, 'y', 2*P.x*V.x);

You should find that checkgradient(problem) and checkhessian(problem) pass all tests.

I hope this helps.

Best,

Nicolas

[Nicolas Boumal]

(Just to clarify: for Hess f(x, y)[u, v], the code is hess(P, V), where the correspondence that (x, y) = (P.x, P.y) and (u, v) = (V.x, V.y).)

[Grant G]

I managed to resolve my misunderstanding of Quasi-Netwon methods on Manifolds.

Thanks for your previous help,
Grant