Detecting/handling 32 bit overflows

[Frank Neuhaus]

Hi,

I am optimizing quite large sparse optimization problems, which is generally working very well for me. Unfortunately, sometimes, the problems are so large, that the number of nonzeros in block_sparse_matrix.cc overflows "int" (32 bit) and I am receiving CHECK failures like this:

Check failed: num_nonzeros >=0 (-something > 0)

Now, it is my understanding, that there is an ongoing effort to move to 64 bit, as a part of this merge request https://ceres-solver-review.googlesource.com/c/ceres-solver/+/23960 so at some point in the future, this issue that I am observing should be fixed, correct?

Now, for the time being, the CHECK macro is an issue for me, because it simply aborts the program - and I do not really have a way to predict at what point an optimization problem is "too large". Can you think of an elegant way to either handle this overflow more gracefully without aborting and without patching ceres/glog or a way to detect this case before actually triggering the check failure?

I have already tried to use glog's google::InstallFailureFunction to install a custom failure handler that does not abort but throws instead... But throwing does not work there because it is called from inside a destructor (and to be honest, I am not sure whether it would actually really be a good idea to throw instead of abort in general...)

Thank you

   Frank

[Sameer Agarwal]

Frank this should be doable.

Two things need to happen. The way we create the Jacobian matrix should be allowed to fail without triggering a check in the constructor of the matrix class and then 

this line 

https://github.com/ceres-solver/ceres-solver/blob/310a252fb63b259c8a5730c1dfe4d387997a53e1/internal/ceres/trust_region_preprocessor.cc#L345

where we create the Jacobian matrix should handle the case that CreateJacobian returns a nullptr and pipe that information back up to indicate a failure that is then handled by the Solve function.

Sameer

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/fb29f2c7-2e00-4432-b48d-0d7834a29058n%40googlegroups.com.

[Sameer Agarwal]

Frank,

Can you try the following out and see if it solves the problem?

https://ceres-solver-review.git.corp.google.com/c/ceres-solver/+/24480

Sameer

[Frank Neuhaus]

Hello Sameer,

(If anyone else is wondering, the link for non-googlers is: https://ceres-solver-review.googlesource.com/c/ceres-solver/+/24480 .)

Thank you very much for your help and solution! Perfect, I have tried it and it works well for me, so no need for anything else. Thank you!

Frank

[Sameer Agarwal]

Thanks Frank, I will clean it up and check it in.

Sameer 

To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/9892f80d-46bd-47b1-a37d-17d3d84f1401n%40googlegroups.com.