Ceres Solver Version 2.0.0 Release Candidate 1

[Sameer Agarwal]

Hello Everyone,
We are pleased to announce the first release candidate for version 2.0.0. The tarball can be downloaded from

http://ceres-solver.org/ceres-solver-2.0.0rc1.tar.gz

This is the culmination of more than two years of work covering almost 300 commits, so the list of changes is rather large. Here are some highlights:

1. Ceres Solver now requires a C++14 compatible compiler, Eigen version > 3.3 & CMake version >= 3.5, XCode version >= 11.2
2. C++ threading based multi-threading support.
3. Problem::AddResidualBlock, SizedFunction, AutoDiffCostFunction, NumericDiffCostFunction support an arbitrary number of parameter blocks using variadic templates
4. On Apple platforms, support for Apple’s Accelerate framework as a sparse linear algebra library.
5. Significantly faster AutoDiff
6. Mixed precision solves when using SPARSE_NORMAL_CHOLESKY.
7. LocalParameterization objects can have a zero sized tangent size, which effectively makes the parameter block constant. In particular, this allows for a SubsetParameterization that holds all the coordinates of a parameter block constant.
8. A new (SUBSET) preconditioner for problems with general sparsity.
9. Faster Schur elimination using faster custom BLAS routines for small matrices.
10. Automatic differentiation for FirstOrderFunction in the form of AutoDiffFirstOrderFunction.
11. Backward Incompatible API Changes

And a few API breaking changes (besides the move to C++14):

1. EvaluationCallback has been moved from Solver::Options to Problem::Options for a more correct API.
2. No more Android.mk based builds.
3. Solver::Options::num_linear_solver_threads is no more.

Besides this there is the usual bevy of bug fixes & performance improvements. We would appreciate you testing on as many platforms as possible. Please report back with your experiences, good and bad. And thank you for all your code contributions & bug reports.

Happy optimizing,
Ceres Solver Team

[Sameer Agarwal]

Folks,

I forgot to mention that our plan is to make 2.0.0 final at the end of August.  That should be enough time to test and fix any bugs that you may find.

We would especially appreciate testing on windows. None of the maintainers use Windows and could use your help there.

Sameer

[Ky Waegel]

I use Ceres on Windows (among several other platforms). Where would you prefer bugs to be reported? Google group, GitHub, etc?

-Ky Waegel

[Sameer Agarwal]

Ky,

Github would be great!

Thank you,

Sameer

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/e0258913-5dd1-4054-ba8d-f874776c7410o%40googlegroups.com.

[Nikolaus Demmel]

Hi,

Thanks for the release candidate! I tested it a bit on various projects on Linux and Mac. Overall it works great. I needed no code changes, only some minor cmake adaptation (there are minor open issues related to using custom Eigen from a source tree and Intel's TBB binary packages).

I can share 2 timing tests on Linux (Ubuntu 18.04). Keep in mind that these are just exemplar runs on desktop computers, and not 100% controlled timing experiments.

The first is for Lie group splines (experiments from this paper). In the results overview you can see around 20% speedup with ceres 2.0 (these correspond essentially to the timing tables from the paper). All output in the ci jobs is linked from the gist, but I also included directly the ceres output of one of the calibration experiments, namely "ceres_se3_old", which is quite heavy on autodiff. It seems half the runtime improvement is in jacobian evaluation, and half in the linear solver (no Schur complement).

The second one is a bundle adjustment type optimization with photometric residuals using Schur complement and auto diff. Residuals are not simple, but also not hugely complex (relative pose, point transformation, non-linear projection function, bi-cubic image interpolation). Overall I see a quite consistent runtime improvement of around 10% across different problem sizes (100s to 1000s of cameras). Looks like it's mostly the linear solver, maybe the improved Schur elimination you mentioned. Here is the ceres output for one example. (The residual and jacobian evaluation is actually a bit slower in the ceres 2.0 run, but I'm not sure if the difference is even significant or just accidental due not 100% controlled environment.)

To summarize, I think it's a nice runtime improvement "for free"! I'm curious on your thoughts on the linear solver runtime improvements, in particular in the experiments without Schur complement.

Best,

Nikolaus

PS: I'm just having a thought about the linear solver timings of the BA experiment. On that machine the installed blas / lapack is openblas, and I recently noticed some issues where openblas can be much slower if it uses openmp multithreading for small operations. Not sure if ceres configures this somehow differently, since it also switched away from openmp for it's multi-threading. For the lie-spline experiments, that ci image where this is run has the standard libblas from Ubuntu, not openblas, so is probably not a factor.

[Sameer Agarwal]

Hi Nikolaus,

Thank you very much for sharing your detailed experience. Comments inline.

On Sat, Aug 29, 2020 at 6:38 AM Nikolaus Demmel <nde...@gmail.com> wrote:

Hi,

Thanks for the release candidate! I tested it a bit on various projects on Linux and Mac. Overall it works great. I needed no code changes, only some minor cmake adaptation (there are minor open issues related to using custom Eigen from a source tree and Intel's TBB binary packages).

Yes, we will fix these two before release.

 

I can share 2 timing tests on Linux (Ubuntu 18.04). Keep in mind that these are just exemplar runs on desktop computers, and not 100% controlled timing experiments.

The first is for Lie group splines (experiments from this paper). In the results overview you can see around 20% speedup with ceres 2.0 (these correspond essentially to the timing tables from the paper). All output in the ci jobs is linked from the gist, but I also included directly the ceres output of one of the calibration experiments, namely "ceres_se3_old", which is quite heavy on autodiff. It seems half the runtime improvement is in jacobian evaluation, and half in the linear solver (no Schur complement).

Yes. I am a bit surprised by the linear solver time improvements, because I do not think the SPARSE_NORMAL_CHOLESKY solver has changed this release.

 

The second one is a bundle adjustment type optimization with photometric residuals using Schur complement and auto diff. Residuals are not simple, but also not hugely complex (relative pose, point transformation, non-linear projection function, bi-cubic image interpolation). Overall I see a quite consistent runtime improvement of around 10% across different problem sizes (100s to 1000s of cameras). Looks like it's mostly the linear solver, maybe the improved Schur elimination you mentioned. Here is the ceres output for one example. (The residual and jacobian evaluation is actually a bit slower in the ceres 2.0 run, but I'm not sure if the difference is even significant or just accidental due not 100% controlled environment.)

This one is definitely affected by the new small blas routines I think. I think this may also have something to do with improved cholesky factorization for small blocks, which was optimized for another application. 

To summarize, I think it's a nice runtime improvement "for free"! I'm curious on your thoughts on the linear solver runtime improvements, in particular in the experiments without Schur complement.

I am delighted to hear this! To be honest, this release has been in the works for so long, I have lost talk of all the performance improvements!

Best,

Nikolaus

PS: I'm just having a thought about the linear solver timings of the BA experiment. On that machine the installed blas / lapack is openblas, and I recently noticed some issues where openblas can be much slower if it uses openmp multithreading for small operations. Not sure if ceres configures this somehow differently, since it also switched away from openmp for it's multi-threading. For the lie-spline experiments, that ci image where this is run has the standard libblas from Ubuntu, not openblas, so is probably not a factor.

Ceres does not configure the underlying blas libraries. And yes you should basically turn threading off in openblas.

Sameer

 

On Wednesday, August 5, 2020 at 5:52:20 AM UTC+2 Sameer Agarwal wrote:

Hello Everyone,
We are pleased to announce the first release candidate for version 2.0.0. The tarball can be downloaded from

http://ceres-solver.org/ceres-solver-2.0.0rc1.tar.gz

This is the culmination of more than two years of work covering almost 300 commits, so the list of changes is rather large. Here are some highlights:

1. Ceres Solver now requires a C++14 compatible compiler, Eigen version > 3.3 & CMake version >= 3.5, XCode version >= 11.2
2. C++ threading based multi-threading support.
3. Problem::AddResidualBlock, SizedFunction, AutoDiffCostFunction, NumericDiffCostFunction support an arbitrary number of parameter blocks using variadic templates
4. On Apple platforms, support for Apple’s Accelerate framework as a sparse linear algebra library.
5. Significantly faster AutoDiff
6. Mixed precision solves when using SPARSE_NORMAL_CHOLESKY.
7. LocalParameterization objects can have a zero sized tangent size, which effectively makes the parameter block constant. In particular, this allows for a SubsetParameterization that holds all the coordinates of a parameter block constant.
8. A new (SUBSET) preconditioner for problems with general sparsity.
9. Faster Schur elimination using faster custom BLAS routines for small matrices.
10. Automatic differentiation for FirstOrderFunction in the form of AutoDiffFirstOrderFunction.
11. Backward Incompatible API Changes

And a few API breaking changes (besides the move to C++14):

1. EvaluationCallback has been moved from Solver::Options to Problem::Options for a more correct API.
2. No more Android.mk based builds.
3. Solver::Options::num_linear_solver_threads is no more.

Besides this there is the usual bevy of bug fixes & performance improvements. We would appreciate you testing on as many platforms as possible. Please report back with your experiences, good and bad. And thank you for all your code contributions & bug reports.

Happy optimizing,
Ceres Solver Team

--

You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/a6f54e4e-0cec-4d44-a1ae-c8919327d157n%40googlegroups.com.

[Nikolaus Demmel]

Hi Sameer,

On Monday, August 31, 2020 at 12:47:31 AM UTC+2, Sameer Agarwal wrote:

Hi Nikolaus,

Thank you very much for sharing your detailed experience. Comments inline.

On Sat, Aug 29, 2020 at 6:38 AM Nikolaus Demmel <nde...@gmail.com> wrote:

Hi,

Thanks for the release candidate! I tested it a bit on various projects on Linux and Mac. Overall it works great. I needed no code changes, only some minor cmake adaptation (there are minor open issues related to using custom Eigen from a source tree and Intel's TBB binary packages).

Yes, we will fix these two before release.

One more thing: IMHO some small hints for migration from 1.14 to 2.0 would be nice (maybe in the release notes), since ceres' cmake config files changed such that CERES_INCLUDE_DIRS (some others) is not set any more. I understand you should use the exported target instead. Plus maybe at least a quick note that the way some third-party libraries like Eigen (and glog?) are found has changed.

 

 

I can share 2 timing tests on Linux (Ubuntu 18.04). Keep in mind that these are just exemplar runs on desktop computers, and not 100% controlled timing experiments.

The first is for Lie group splines (experiments from this paper). In the results overview you can see around 20% speedup with ceres 2.0 (these correspond essentially to the timing tables from the paper). All output in the ci jobs is linked from the gist, but I also included directly the ceres output of one of the calibration experiments, namely "ceres_se3_old", which is quite heavy on autodiff. It seems half the runtime improvement is in jacobian evaluation, and half in the linear solver (no Schur complement).

Yes. I am a bit surprised by the linear solver time improvements, because I do not think the SPARSE_NORMAL_CHOLESKY solver has changed this release.

 

The second one is a bundle adjustment type optimization with photometric residuals using Schur complement and auto diff. Residuals are not simple, but also not hugely complex (relative pose, point transformation, non-linear projection function, bi-cubic image interpolation). Overall I see a quite consistent runtime improvement of around 10% across different problem sizes (100s to 1000s of cameras). Looks like it's mostly the linear solver, maybe the improved Schur elimination you mentioned. Here is the ceres output for one example. (The residual and jacobian evaluation is actually a bit slower in the ceres 2.0 run, but I'm not sure if the difference is even significant or just accidental due not 100% controlled environment.)

This one is definitely affected by the new small blas routines I think. I think this may also have something to do with improved cholesky factorization for small blocks, which was optimized for another application. 

To summarize, I think it's a nice runtime improvement "for free"! I'm curious on your thoughts on the linear solver runtime improvements, in particular in the experiments without Schur complement.

I am delighted to hear this! To be honest, this release has been in the works for so long, I have lost talk of all the performance improvements!

Best,

Nikolaus

PS: I'm just having a thought about the linear solver timings of the BA experiment. On that machine the installed blas / lapack is openblas, and I recently noticed some issues where openblas can be much slower if it uses openmp multithreading for small operations. Not sure if ceres configures this somehow differently, since it also switched away from openmp for it's multi-threading. For the lie-spline experiments, that ci image where this is run has the standard libblas from Ubuntu, not openblas, so is probably not a factor.

Ceres does not configure the underlying blas libraries. And yes you should basically turn threading off in openblas.

Just FYI: I reran with OPENBLAS_NUM_THREADS=1 to double check. Runtime wasn't much different with that.

Best,

Nikolaus

 

 

To unsubscribe from this group and stop receiving emails from it, send an email to ceres-...@googlegroups.com.

[Sameer Agarwal]

On Mon, Aug 31, 2020 at 2:10 PM Nikolaus Demmel <nde...@gmail.com> wrote:

Hi Sameer,

On Monday, August 31, 2020 at 12:47:31 AM UTC+2, Sameer Agarwal wrote:

Hi Nikolaus,

Thank you very much for sharing your detailed experience. Comments inline.

On Sat, Aug 29, 2020 at 6:38 AM Nikolaus Demmel <nde...@gmail.com> wrote:

Hi,

Thanks for the release candidate! I tested it a bit on various projects on Linux and Mac. Overall it works great. I needed no code changes, only some minor cmake adaptation (there are minor open issues related to using custom Eigen from a source tree and Intel's TBB binary packages).

Yes, we will fix these two before release.

One more thing: IMHO some small hints for migration from 1.14 to 2.0 would be nice (maybe in the release notes), since ceres' cmake config files changed such that CERES_INCLUDE_DIRS (some others) is not set any more. I understand you should use the exported target instead. Plus maybe at least a quick note that the way some third-party libraries like Eigen (and glog?) are found has changed.

I think this belongs best in the chapter on installation AND the release notes. Since you are the one who first experienced this pain, want to send me a pull request?

Sameer

 

To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/29bd8d48-2964-47f8-bdd3-f8aac8a29672o%40googlegroups.com.

[Bogdan Burlacu]

Hi,

It appears that 2.0.0rc1 introduced a compile error with Eigen isFinite():

TEST_CASE("ceres eigen isfinite()")
{
    using J = ceres::Jet<double, 4>;

    Eigen::Array<double, 5, 1, Eigen::ColMajor> a;
    a << 1, 2, 3, 4, 5;
    std::cout << (a.isFinite()).select(a, 0.0).transpose() << "\n";

    Eigen::Array<J, 5, 1, Eigen::ColMajor> b = a.cast<J>();
    std::cout << (b.isFinite()).select(b, J(0.0)).transpose() << "\n"; // error
}

This used to work, but now fails with:

/tmp/test/implementation/ceres.cpp:14:61:   required from here /nix/store/nh6mmb3zhfxs4h8iapx8gx6fbcrl4i01-eigen-3.3.7/include/eigen3/Eigen/src/Core/MathFunctions.h:879:39: error: no matching function for call to ‘isfinite(const ceres::Jet<double, 4>&)’  879 |    

return isfinite EIGEN_NOT_A_MACRO (x);             

I am using a trunk version of Eigen (rev 46f8a185).

Best,

Bogdan

[Sameer Agarwal]

Thanks for the report Bogdan. I am unable to install Eigen at head using homebrew today. So something seems to be broken there too. 

How about the current stable release of Eigen, are you seeing problems with that too?

Sameer

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/1ca16f28-dc8e-4a3b-bbbd-acc175dce15fn%40googlegroups.com.