# Hessian matrix calculation

73 views

### 邱知道

Apr 7, 2021, 1:19:54 PMApr 7
to Manopt
Hello, Nicolas
I want to minimize a cost function: f(s) = s'*C*s*s'*C'*s.
Its first-order derivative is： df(s) = 2*(s'*C*s*C'*s + s'*C'*s*C*s) and it could be used as
Problem.egrad = @(s) 2*(s'*C*s*C'*s + s'*C'*s*C*s);
Could you please tell me how to calculate and use its Hessian matrix? I realized there is a '‘H’ in the given examples, like Problem.ehess = @(s, H)...... and I don't know why there is a ’H‘ in the variables.

Apr 7, 2021, 2:33:56 PMApr 7
to Manopt
Hello,
To my understanding ehess function is expected to compute the directional derivative of egrad at a point 's' along a vector 'v'.
I have written a small code using euclideanfactory manifold. I have assumed that 's' is an n-dimensional real vector. The gradient and hessian checks confirm that my calculations are correct.

Please correct me if I am wrong.

function test()

n = 100;
C = magic(n);
A = C + C';

manifold = euclideanfactory(n,1);
problem.M = manifold;

% Note that for a vector s, s'*C*s = s'*C'*s
problem.cost  = @(s)     1/2*(s'*C*s)^2;
problem.ehess = @(s,v) (s'*C*s)*(A*v) + (v'*A*s)*A*s;

checkhessian(problem); pause;

end

Best Regards,
Message has been deleted

### 邱知道

Apr 7, 2021, 8:09:57 PMApr 7
to Manopt

Hello,
Yes, your code runs correctly. But when I change   'manifold = euclideanfactory(n,1) ' to 'euclideancomplexfactory(n,1); ', the 'checkhessian(problem)' output a wrong result:
The slope should be 3. It appears to be: 1.
If it is far from 3, then directional derivatives,....
Best,
Xiangfeng

Apr 7, 2021, 11:38:47 PMApr 7
to Manopt
Hello Xiangfeng,
I have corrected the code to work with euclideancomplexfactory. I am attaching the code below. The changes I made are that to treat s'*C*s and s'*C'*s as different entities and carefully evaluate the gradient and the directional derivative of gradient. For numerical reasons, there is a small residue of imaginary part left in computing (s'*C*s)*(s'*C'*s), so I had to return the real part of it. Now the gradient and hessians check pass!

function test()

n = 100;
C = rand(n)+rand(1)*1i*rand(n);

manifold = euclideancomplexfactory(n);
problem.M = manifold;

% Note that s'*C*s is the complex conjugate of s'*C'*s.
% Yet (s'*C*s)*(s'*C'*s) might have some residual imaginary
% part, so we have return real part of the cost function.
problem.cost  = @(s)   real(1/2*(s'*C*s)*(s'*C'*s));
problem.egrad = @(s)   (s'*C'*s)*(C*s) + (s'*C*s)*(C'*s);
problem.ehess = @(s,v) (s'*C'*s)*(C*v) + (s'*C'*v)*(C*s) + ...
(v'*C'*s)*(C*s) + (s'*C*s)*(C'*v)+ ...
(s'*C*v)*(C'*s)+ (v'*C*s)*(C'*s);

checkhessian(problem); pause;

end

Regards,

### Nicolas Boumal

Apr 8, 2021, 4:42:35 AMApr 8
to Manopt
Hello everyone,

Jayadev, thanks for weighing in. You can also write the cost function as 1/2*abs(s'*C*s)^2, in which case there is no need to take the real part.

Best,
Nicolas

### 邱知道

Apr 8, 2021, 10:21:12 AMApr 8
to Manopt
Thank you very much for your help.
Best,
Xiangfeng
Message has been deleted

### Robert Qiu

Jul 2, 2021, 7:32:26 AMJul 2
to Manopt
Thank you very much for your previous help. But I am still not very clear about the calculation process of the Riemann Hessian matrix. I can only calculate the Euclidean gradient and Euclidean Hessian of the objective function, but I don't know how to calculate the directional derivative of the Euclidean gradient along a certain direction, nor can I calculate the Riemann Hessian matrix. Can you give me a detailed calculation process. thank you very much.
Best,
Xiangfeng

### Robert Qiu

Jul 2, 2021, 7:43:01 AMJul 2
to Manopt

Can you clarify the similarities and differences between Euclidean Hessian and  the direction derivatives of  Euclidean gradient ?

### Nicolas Boumal

Jul 2, 2021, 8:43:09 AMJul 2
to Manopt
Hello,

This question is really about linear algebra and multivariate calculus: the Hessian is a "matrix", sure, but more importantly it is a linear map. That linear map is precisely the directional derivative of the gradient vector field (in the Euclidean case). For the Riemannian case, we use a special notion of derivative for vector fields, but it's the same idea.

My recommendation would be first to make sure you are comfortable with the 'linear map' interpretation of Hessians in the Euclidean case, then that you read up on Riemannian Hessians in Chapter 5 of my book (freely available on my webpage).

Best wishes,
Nicolas