Line search

[bahu....@gmail.com]

Hi all,
I want to NOT use linesearch but a 'quasi-fixed' step length of size alpha=alpha0/i in steepest descent where i is the iteration number. How is that possible in Manopt?

[BM]

Hello Bahu, 

You could do add the following lines of code to the main file, where you define problem.cost, problem.grad and so on. numberofcalls counts the iteration number and alpha0 is the initial step-size.

alpha0 = 1;
numberofcalls = 0;  % Verify that this is a good initialization, i.e., it counts the number of iterations.
problem.linesearch = @linesearch_helper;

function alpha = linesearch_helper(X, eta)
    numberofcalls = numberofcalls + 1; % Update the number of calls.
    alpha = alpha0/numberofcalls;
end

Let us know whether it works for your case. 

Regards,

BM

[BM]

One more thing: you should call the steepestdescent algorithm with options.ls_suff_decr = -inf or a large negative value. This should make the Armijo condition check redundant.

Let us know your experience.  

Regards,
BM

[Nicolas Boumal]

Hello,

I would follow Bamdev's suggestion. An alternative is to write a line-search function, similar to the ones you can find in /solvers/linesearch, and to pass it through options.linesearch (as opposed to problem.linesearch).

But even so, those calls actually do not get access to the iteration number k (I suppose we did not see the practical use for it, but that is an oversight). So even that way will not be entirely satisfactory. You could count iterations using storedb.internal of course, which would be a little bit cleaner than Bamdev's hack*, but still.

I'm adding to the TODO list that it would be good for storedb to contain some metadata, such as the proper iteration count. But I do not expect this will be added into a manopt release very soon.

Let us know how you fare,

Best,

Nicolas

* i'm claling it a hack, because everytime you call steepestdescent or conjugategradient on your problem structure, you need to remember to reset to counter to 0, which is not very nice. That would not be the case with the alternative I'm hinting at.

[bahu....@gmail.com]

Thanks, BM, that helped!