Computing the gradient of a function of a matrix

[Nicolas Boumal]

Hello,

Computing the gradient of a function whose arguments are matrices is a recurring question regarding the usage of Manopt.

This nice blog post should be useful to learn the basic trick:

http://thousandfold.net/cz/2013/11/12/a-useful-trick-for-computing-gradients-w-r-t-matrix-arguments-with-some-examples/

In a nutshell: the idea is to first compute the Fréchet derivative (or directional derivative) of the function; then to manipulate that expression in order to read off the gradient from it. This will give you the Euclidean gradient of your function, which you can input in Manopt as problem.egrad (notice the "e" in front of egrad, for Euclidean). Manopt will convert that to the Riemannian gradient automatically (for most geometries available; contact us if it's missing in a geometry you are using).

This is also discussed at length in my book, announced here:

https://groups.google.com/forum/#!topic/manopttoolbox/kL7G_KsM6aQ

Cheers,

Nicolas

[Nicolas Boumal]

This online tool can also be useful in automatically computing expressions for gradients of matrix functions:

http://www.matrixcalculus.org/

Thanks to Justin Solomon for pointing this out!

Nicolas

On Tuesday, September 23, 2014 at 8:14:11 AM UTC-4, Nicolas Boumal wrote:

Hello,

Computing the gradient of a function whose arguments are matrices is a recurring question regarding the usage of Manopt.

This nice blog post should be useful to learn the basic trick:

http://thousandfold.net/cz/2013/11/12/a-useful-trick-for-computing-gradients-w-r-t-matrix-arguments-with-some-examples/

In a nutshell: the idea is to first compute the Fréchet derivative (or directional derivative) of the function; then to manipulate that expression in order to read off the gradient from it. This will give you the Euclidean gradient of your function, which you can input in Manopt as problem.egrad (notice the "e" in front of egrad, for Euclidean). Manopt will convert that to the Riemannian gradient automatically (for most geometries available; contact us if it's missing in a geometry you are using).

Cheers,

Nicolas