One dimensional minimization

[bahu....@gmail.com]

Your powerful Manopt is certainly not designed for one-dimensional unconstrained optimization ... nevertheless, it should work also in this case.

Please have a look at the following test function:

function X=hesstest3()

% Definition of the manifold

Man = euclideanfactory(1,1);
problem.M = Man;

% Define the problem cost function
problem.cost = @cost;

function f = cost(t)
f=(t-2)^2*exp(t);
end

% Define the problem gradient and Hessian

problem.egrad = @egrad;
function g = egrad(t)
g=exp(t)*(-2+t)*t;
end

problem.ehess = @ehess;
function h=ehess(t,s)
h=exp(t)*(-2+t^2)*s^2;
end

% Check numerically whether gradient is correct
% checkgradient(problem);
checkhessian(problem);

% Solve

X = steepestdescent(problem,3.01);

end

Running this function, it SOMETIMES produces an error like

<d1, H[d2]> - <H[d1], d2> should be zero, or very close.
Value: -0.880626 - 0.880626 = -1.76125.

If I did everything correctly, this should not happen ...

[Nicolas Boumal]

Hello,

Thank you for your message.

it's an easy fix: the Hessian should look like this:

    problem.ehess = @ehess;
    function h=ehess(t,s)       

        h=exp(t)*(-2+t^2)*s;   % there is no power 2 on s
    end

Remember: the Hessian is a linear operator in the second argument (here called s), so it cannot appear squared.

This gives the following output, calling trustregions (so we see the effect of second order information):

The slope should be 3. It appears to be: 2.99958.

If it is far from 3, then directional derivatives or the Hessian might be erroneous.

Note: if the exponential map is only approximate, and it is not a second-order approximation,

then it is normal for the slope test to fail. Check the factory for this.

The residual should be zero, or very close. Residual: 0.

If it is far from 0, then the Hessian is not in the tangent plane.

<d1, H[d2]> - <H[d1], d2> should be zero, or very close.

Value: -0.561157 - -0.561157 = 0.

If it is far from 0, then the Hessian is not symmetric.

                                            f: +2.069518e+01   |grad|: 6.167572e+01

acc TR+   k:     1     num_inner:     1     f: +1.402252e+01   |grad|: 4.571184e+01   exceeded trust region

acc TR+   k:     2     num_inner:     1     f: +5.622292e+00   |grad|: 2.333030e+01   exceeded trust region

acc       k:     3     num_inner:     1     f: +8.738668e-01   |grad|: 6.768187e+00   reached target residual-kappa (linear)

acc       k:     4     num_inner:     1     f: +6.326475e-02   |grad|: 1.492581e+00   reached target residual-kappa (linear)

acc       k:     5     num_inner:     1     f: +7.838985e-04   |grad|: 1.537797e-01   reached target residual-kappa (linear)

acc       k:     6     num_inner:     1     f: +1.772456e-07   |grad|: 2.289178e-03   reached target residual-kappa (linear)

acc       k:     7     num_inner:     1     f: +9.558416e-15   |grad|: 5.315174e-07   reached target residual-theta (superlinear)

Gradient norm tolerance reached; options.tolgradnorm = 1e-06.

Total time is 0.012150 [s] (excludes statsfun)

ans =

    2.0000

Best,

Nicolas

[bahu....@gmail.com]

Yes, of course, thanks!