Gradient check with product manifold

[Evelyn LI]

Hi,

I have a problem with the Euclidean gradient of a product manifold. The product manifold consist of a complex Euclidean manifold and the complex circles. 

I computed the partial gradient of the cost function with respect to two variables separately. But the I checked the gradient, it was failed. Here is the link of my code:  

https://gist.github.com/mzjie/ead175140a69703c7058ba9134238b80

 

I tried several times and I can't get the correct gradient. I considered the definition of complex inner production and I tried to change all the inner product to get the real part of two vectors. But it still can fit the slope of 2. 

Thank you so much for your time and kindly help. 

Best,

Evelyn

[Nicolas Boumal]

Hello Evelyn,

Sorry for the delay -- were you able to solve your problem in the meantime?

Looking at your code, I see one small issue (that doesn't resolve the bigger issue at hand): x0 (your initialization) is not a point on the manifold, because x0.Y is not a point on the complex circles manifold. You can resolve that by calling:

x0 = Manifold.rand()

I ran your code with options.verbosity = 2 to see what the solver is saying as it iterates, and indeed the gradient direction seems uninformative (sometimes, when the gradient is wrong just by a constant factor, the algorithm still performs fine even if the slope test is unsuccessful.)

I am wondering: why are you reshaping x.X and x.Y into 4x4 matrices, instead of picking your manifold to work directly on 4x4 matrices (as opposed to vectors of length 16x1)?

Unfortunately, I lack the time to work out this gradient right now, but one thing to ponder: when I remove the code for the gradient and run an optimization algorithm, then Manopt will automatically approximate the gradient with finite differences. This is expensive, but it's convenient for prototyping. Doing this, I see that the cost function seems to decrease to infinity (remember that Manopt always minimizes). Is this expect behavior? Or should you expect your cost function to be bounded below? This may indicate a problem with the cost function (even before any problems with the gradient).

Best,

Nicolas