Is Ceres deterministic?

[Tobias Nöll]

Hello there,
first: Thanks for the fantastic work of Ceres!

I have a question that I don't find answered in the documentation.
I'm using Ceres to optimize a bundle adjustment problem.
I have the "problem" that during multiple runs the parameters are slightly differently optimized.
Shouldn't Ceres always optimize the parameters in the same way when called multiple times with the same problem?

In my program this is the case for automatic differentiation and numeric as well. My configuration is:
ceres::Solver::Options options;
options.linear_solver_type = ceres::SPARSE_SCHUR;
options.num_threads = 8;
options.num_linear_solver_threads = 8;

I'm using MKL for Lapack

Is this normal behavior of Ceres (maybe because of the threads) or does this indicate that there could be a problem (e.g. reading uninitialized memory) in my error functional?
Thanks for your feedback!

Regards Tobias

[Sameer Agarwal]

With threading turned in determinacy cannot be guaranteed.
Sameer

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/7cc40d9a-9dc9-4d8e-8385-193aff064b31%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[William Rucklidge]

I've found that Ceres is deterministic if you set "options.num_threads" to a value >1, but keep "options.linear_solver_threads" set to 1. This only gets you part of the available speedup: your residuals and Jacobians are evaluated in parallel, but all the matrix operations run serially.

-wjr

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CAK0oyEr5j96zptN5844s_GNCPtFd3dGp_Mq3hDpDP4r49UVEcw%40mail.gmail.com.

[Sameer Agarwal]

Almost deterministic.. since the gradient computation also happens inside the evaluator and that involves threaded accumulation.. its not as bad the linear solver, but theoretically it can create non-determinism too.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CABsh3u6Vu-s1szeY5zqtToVYGsC-iC7oALqCLpdZKxz-hmvCvA%40mail.gmail.com.

[Jon Zubizarreta Gorostidi]

Hi, I am solving an iterative non-linear problem using ceres. My configuration is: Levenberg Marquardt, Sparse Normal Cholesky, num_threads = 1 and I use my own parameter block ordering. With this configuration I expect a deterministic behaviour. However, I get diffferent results each run. Do I miss something?

Thanks,

Jon

 

[Sameer Agarwal]

For a well conditioned problem it should be deterministic.  But if your problem is poorly conditioned, then rounding will likely produce non-deterministic results.

--

You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/f3ee17f4-18df-40b7-a712-bc762d5c6137%40googlegroups.com.

[Jon Zubizarreta Gorostidi]

Thanks Sameer!

How could I improve the conditioning of my problem? Where can I read about it?

Thanks again for all the support!

Jon

You received this message because you are subscribed to a topic in the Google Groups "Ceres Solver" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ceres-solver/F_bbG_5afnY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CABqdRUADwES8cOF-SyKMpMuXJMYgFo9qyTtima%3DMLbGqf3hfYA%40mail.gmail.com.

[Sameer Agarwal]

Jon,

Here is some basic advice.

http://www.alglib.net/optimization/preconditioning.php

http://www.alglib.net/optimization/scaling.php

Sameer

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CAMi868V78G2N-CgQR%2BbOc7ha9K_weSxu4QfDbkP2zpH_hKiFTg%40mail.gmail.com.

[Jon Zubizarreta Gorostidi]

Really good links! Thanks Sameer.

How could I implement variable scaling with ceres?

I cannot find any function to establish the scale for each variable.

Thanks,

Jon

[Sameer Agarwal]

Jon,

You will have to implement it yourself. There is nothing in the ceres API that enables this right now.

Sameer

--

You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/755f23aa-0a97-4ac7-8420-7f4ca90ad8e8%40googlegroups.com.

[sba...@gmail.com]

As Tobias ask, I want to know how I can put CERES in deterministic option, my version is V1.12 ? 

best regards

Ali