Trying to change delta of initial numerical differencing

[Isaac Gerg]

I am trying to optimize a function that makes a call to a library I have made.  I am trying to get Ceres to evaluate the cost function as little as possible.  I have my cost function print out the parameter value its trying for "c" however, no matter what i set the numeric_derivative_relative_step_size to, it takes the same small step each time ~ 1e-3 which is way too small.

How do I fix this?

Isaac

ceres::NumericDiffCostFunction<SpeedOfSoundObjectiveFunction, ceres::CENTRAL, 1, 1>* cost_function 

= new ceres::NumericDiffCostFunction<SpeedOfSoundObjectiveFunction, ceres::CENTRAL, 1, 1>(new SpeedOfSoundObjectiveFunction(this, pGpuData, pGpuTemp, &motion));

Float64 c = motion.speedOfSound;

ceres::Problem problem;

problem.AddResidualBlock(cost_function, NULL, &c);

const Float64 maxSpeedOfSoundError = 0.05;

//problem.SetParameterLowerBound(&c, 0, motion.speedOfSound - motion.speedOfSound*maxSpeedOfSoundError);

//problem.SetParameterUpperBound(&c, 0, motion.speedOfSound + motion.speedOfSound*maxSpeedOfSoundError);

ceres::Solver::Options options;

ceres::Solver::Summary summary;

options.minimizer_type = ceres::TRUST_REGION;

options.numeric_derivative_relative_step_size = 1e10;

options.linear_solver_type = ceres::DENSE_QR;

options.initial_trust_region_radius = 1e1;

options.num_linear_solver_threads = 1;

options.num_threads = 1;

options.max_num_iterations = 30;

options.minimizer_progress_to_stdout = true;

//options.logging_type = ceres::SILENT;

ceres::Solve(options, &problem, &summary);

[Sameer Agarwal]

Isaac,

This is a documentation fail. Sorry about that.

http://ceres-solver.org/nnls_solving.html#Solver::Options::numeric_derivative_relative_step_size__double only refers to the numeric differentiation used for doing the gradient checking. What you want instead is to pass an argument to the constructor of NumericDiffCostFunction. If you are using the version from git, there is an options struct, 

https://github.com/ceres-solver/ceres-solver/blob/master/include/ceres/numeric_diff_options.h

that you want to use. If you are on an earlier version, then the constructor has an explicit scalar argument that controls the step size.

Hope this helps, I will work on fixing the documentation shortly.

Sameer

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/8a6e4f48-c8fe-4ab8-bfe3-747267e28c60%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Isaac Gerg]

Thanks Sameer, it works now.  Another questions, I am doing a line search in which the surface is a bit rough but overall parabolic on a large scale (roughness is an order of magnitude less than the order of the parabola).  I am trying to use a line search method to solve this.  I have made it as simple as I can and am using steepest descent.  However, when I examine the solutions Ceres is trying, it is not trying things far from the initial guess.  I have read through the math and I can't figure out what is going on?  How do I increase my step size?

[Sameer Agarwal]

On Thu, Oct 1, 2015 at 3:46 PM, Isaac Gerg <isaac...@gergltd.com> wrote:

Thanks Sameer, it works now.  Another questions, I am doing a line search in which the surface is a bit rough but overall parabolic on a large scale (roughness is an order of magnitude less than the order of the parabola).  I am trying to use a line search method to solve this.  I have made it as simple as I can and am using steepest descent.  However, when I examine the solutions Ceres is trying, it is not trying things far from the initial guess.  I have read through the math and I can't figure out what is going on?  How do I increase my step size?

hmm, I need to go looking into the line search code again, I think we maybe starting with a default of size 1 and then expanding/contracting it.

Which line search and what line search direction are you using?

Sameer

 

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/5cbed2f4-1317-4315-84f8-5db6794563ef%40googlegroups.com.

[Sameer Agarwal]

Line 291 in line_search_minimizer.cc has the initial step size logic. But it is lower bounded by 1. After that Armijo for sure is only searching between 0 and the initial step size. Alex can say more about the Wolfe line search as he wrote that code.

Sameer

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CABqdRUBhR4US73iN%3DAYT8pxUq5iiZeC-5HGnDEH7-6omqG4GQQ%40mail.gmail.com.

[Sameer Agarwal]

Alex reminded me that I read the code wrong, and that the initial step size is bounded above not below by 1. Armijo line search will only contract it further, but the Wolfe line search can increase the step size too.

Sameer

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CAK0oyEojPn0_n-HfSfo80iKhqDWZ5hCbjm03as6q46N9EtJPRA%40mail.gmail.com.

[Isaac Gerg]

Thanks.  Will check it out.