how to compute euclidean hessian

[Yi Hong]

How to compute euclidean hessian? Are there any references?

Thanks a lot.

[Yi Hong]

It's for matrix, like f(Q) = ||Q'AQ -B||f. Thanks!

[BM]

Hello Yi,

Let us consider the function, f(Q) = ||Q'AQ -B||f ^2 (squared frobenius norm).

The Euclidean gradient is = A' Q S + A Q S' where S = 2(Q'AQ -B). (Checked)

Now, the Euclidean Hessian operator along a direction Z (we require the directional derivative of the Euclidean gradient along the direction Z) is given systematically using the Chain rule. Below is the formula

Euclidean Hess [Z] = A'ZS + A'Q S_star + AZS' + AQS_star'  (Checked)

where  S =  2(Q'AQ -B) and S_star = 2(Z'AQ + Q'AZ).

Do check this out and let us know if this sounds okay. Once formalized, you can use tools like 'checkgradient' and 'checkhessian' to see if this works out.

Regards,

BM

[Qiuwei Li]

HI BM,

I obtain that S_star=2(Z'A'Q+Q'A'Z). Maybe I am wrong. Can you check that?

Thanks,

QW

[BM]

Hello Qiuwei,

I have checked again. You derivation seems to be incorrect. The derivations that I had shown earlier in the thread are correct. Below is the Manopt code for the confirmation. 

Regards,

BM

[Qiuwei Li]

Thanks, BM,

Thanks for helping me understand the technique of computing the Hessian. 

Regards,

Qiuwei

[lineya...@gmail.com]

在 2013年6月7日星期五 UTC+8上午5:22:51，BM写道：

Hi BM,
Thanks for you explanation and I have understood how it works with this objective function. Now I have a function y = f(X_ij - X_ik), and I know how to compute the objective function, gradient function and hessian function of the scalar case. They are all a given matlab function, which is almost like a black box, and I know f(x), f'(x) and f''(x)， but I don't know how to get the directional derivative of Euclidean gradient. Do you have any idea? Thanks a lot.

[BM]

Hello, 

I am glad that it helped you. 

Euclidean derivative along a direction, say h, is simply = f''(x) h, i.e., it is the multiplication of the Hessian to the direction h. 

problem.ehess = @ehess;

function dir_derivative = ehess(x, h)

dir_derivative = f''(x)*h; % f''(x) is the black box computation of the Hessian.

end

Does this help?

Regards,

BM

[lineya...@gmail.com]

Sorry that I didn't make myself clear. What I have f(x), f'(x) and f''(x) is elementwise function. However, the problem I meet is y = \sum_{(i,j,k) in Omega} f(X_{ij}-X{ik})。 According to the rule, grad(X) = [f'(X_{ij}-X_{ik})]_{ij}+[-f'(X_{ij}-X_{ik})]_{ik}. The first term is added at the (i,j)-th element of grad(X), while the second term is added at (i,k)-th. But I don't know how to get the directional derivative of gradient. Would you please give me a little more tips? Thanks.

Kai

[BM]

Hello Kai, 

Could you share a Matlab file where you have written down the cost and gradient computations explicitly. This will be helpful in driving the directional derivative.

Regards,

BM