gradnorm is not dropping

[Yuang Chen]

Hello!

 

When I was checking the gradient with checkgradient, I made sure it was all right. But why doesn't the info.gradnorm calculated by the solver tend to decrease in a big way? I tried conjugategradient() and trustregions() and found them basically similar. It is shown in the figure below.

Thank you.

[Yuang Chen]

And, I find that everything is fine except that info.gradnorm is not getting smaller, including info.cost monotonically decreasing. But I doubt the solution is of the best quality.

[Nicolas Boumal]

Could you show us the output of checkgradient (both the text output and the plot) and specify which factory you are using and whether your cost function is perhaps nonsmooth in some ways?

--
http://www.manopt.org
---
You received this message because you are subscribed to the Google Groups "Manopt" group.
To unsubscribe from this group and stop receiving emails from it, send an email to manopttoolbo...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/manopttoolbox/5d558929-4c4f-45f0-a0af-a3d62556bd6bn%40googlegroups.com.

[Yuang Chen]

Here is my output of checkgradient.

And I  use obliquecomplexfactory(N,6). I optimize for N column vectors. In addition, I wrote some inequality constraints into the objective function using smoothing methods as follows

[Nicolas Boumal]

Two more things to check:

 - How small is the smoothing parameter? ("u" in the snapshot you provided)  Is the behavior different if you set u = 1 for example? (To rule out numerical issues coming from that part of the cost function.)

 - If you supply an initial point x0 yourself, is it indeed on the manifold? If you replace x0 with [] (empty array) in your call to the optimization algorithm (e.g., trustregions), does the behavior improve? If so, that could indicate the x0 is not on the manifold. That would explain why checkgradient can succeed even if the optimization fails, because checkgradient (by default) calls problem.M.rand() to generate a valid random point on the manifold.

By the way, the manifold "obliquecomplexfactory(N, 6)" corresponds to matrices with 6 columns in C^N, not N columns in C^6. See 'help obliquecomplexfactory'.