Definition of cost and egrad in rayleigh quotient min. problem for Grassman

[Hiroyuki Kasai]

Dear Nicolas and Bamdev,

Regarding the rayleigh quotient min. problem in case of Grassman,

are the following definitions correct? 

problem.cost  = @(x) -trace(x'*(A*x));

problem.egrad = @(x) -2*(A*x-x*x'*A*x);

This follows Section 4.9 in the book written by Prof. Absil. 

Note that x'x=I is assumed here.

Regards,

Hiro

[Florian]

Dear Hiroyuki,

Manopt is used for minimization problem, so I guess, if you want to minimize the rayleight quotient, the cost function should be :
problem.cost  = @(x) trace(x'*(A*x));

With the minus sign, it would solve the maximization problem.

Regarding the gradient, I think the Euclidean gradient should be :
problem.egrad = @(x) 2*A*x;

In case of doubt (in the cost or the gradient), I found this tool (provided in the toolbox) very useful :

checkgradient(problem);

Best regards
Florian

[BM]

[Edited]

Dear all, 

I second Florian's post. For a minimization problem, you should remove the minus sign. (A minor addition. The matrix A should be symmetric.)

 problem.egrad should be the Euclidean gradient of trace(x'*(A*x)) and we need not be concerned with the projected gradient. An observation, 2*(A*x-x*x'*A*x) (Hiroyuki's formula) is in fact the projected gradient. Putting in problem.egrad is going to be harmless, anyway. In short, both are correct. The first one is, of course, simpler and should be used.

 

problem.cost  = @(x) trace(x'*(A*x));

problem.egrad = @(x) 2*A*x;  % Euclidean gradient

and 

problem.cost  = @(x) trace(x'*(A*x));

problem.egrad = @(x) 2*(A*x - x*x'*A*x); % We do an additional projection here which is harmless, but not required

Regards,

Bamdev

[Hiroyuki Kasai]

Dear Florian and Bamdev, 

I thank you for your prompt reply.

I understand that "2*A*x" should be used in this context. 

Regards,

Hiro

[Nicolas Boumal]

Note that it is harmless only if you do not use problem.ehess (the Euclidean Hessian).

egrad is really supposed to define the Euclidean gradient, because that is what is needed internally to convert the Euclidean Hessian into the Riemannian Hessian.

If you want to specify the Riemannian (or, in this case, "projected") gradient, it is best to use problem.grad rather than problem.egrad.

Cheers,

Nicolas

[Hiroyuki Kasai]

Hi Nicolas,

Yes, this is my fault. I should use problem.grad if my first definition is used.

Thank you!

Hiro