How to include precomputed parts of JTJ, JTr

57 views
Skip to first unread message

raf...@dotproduct3d.com

unread,
Oct 21, 2017, 10:09:47 AM10/21/17
to Ceres Solver
We'd like to combine externally computed parts of JTJ and JTr with the estimator system computed in Ceres.

Concretely we'd like to perform bundle-adjustment in combination with many (millions, densely computed) pixel-wise terms. Each pixel-wise term is a pose-pose constraint that we'd like to be optimized. Adding each individual pixel-wise term as an observation in Ceres would be a drag on performance as we have a way to compute the pixel-wise terms (and their resulting blocks in JTJ, JTr) very efficiently in our code.

Is there a way to set up a classical optimization / bundle-adjustment problem in Ceres and "manually" add the pixelwise terms as externally computed JTJ, JTr blocks?

Sameer Agarwal

unread,
Oct 22, 2017, 10:45:04 PM10/22/17
to ceres-...@googlegroups.com
Rafael,
Unfortunately there is no way of doing what you want in the Ceres API.
Sameer



--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/788622d9-3846-4392-9eb3-a7137e4e6d0e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Keir Mierle

unread,
Oct 23, 2017, 12:54:16 AM10/23/17
to ceres-...@googlegroups.com
Hi Rafael,

To add more color to this discussion: We've debated issues similar to this one in the past, where users have wanted to write their own evaluator system. The problem is that it's an enormous can of worms on the API side, since currently Ceres has managed to keep a (relatively) tiny API compared to the amount of internal complexity. The small public API has allowed us to make substantial changes to all parts of the Ceres internals, including changes in the fundamental sparse matrix data structures, without breaking user code.

We're not likely to accept patches to add this functionality, since it's not clear how to do this cleanly and maintain a small API.

On the other hand, the Ceres code is reasonably cleanly written and separable. You may find it possible to make a fork of Ceres that reaches into the heart of the LM loop to do what you need done. In our experience it's worthwhile using a proven LM solver loop instead of rolling your own, so consider this route before starting your own from scratch. If you do, and are willing to share, we'd take a look and see what you came up with.

Cheers and happy optimizing,
Keir

On Sun, Oct 22, 2017 at 7:44 PM, 'Sameer Agarwal' via Ceres Solver <ceres-...@googlegroups.com> wrote:
Rafael,
Unfortunately there is no way of doing what you want in the Ceres API.
Sameer


On Sat, Oct 21, 2017 at 7:09 AM <raf...@dotproduct3d.com> wrote:
We'd like to combine externally computed parts of JTJ and JTr with the estimator system computed in Ceres.

Concretely we'd like to perform bundle-adjustment in combination with many (millions, densely computed) pixel-wise terms. Each pixel-wise term is a pose-pose constraint that we'd like to be optimized. Adding each individual pixel-wise term as an observation in Ceres would be a drag on performance as we have a way to compute the pixel-wise terms (and their resulting blocks in JTJ, JTr) very efficiently in our code.

Is there a way to set up a classical optimization / bundle-adjustment problem in Ceres and "manually" add the pixelwise terms as externally computed JTJ, JTr blocks?

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CABqdRUDJmJZVmxSVyHpm97esEYA6DDuvUkvO1FyM28Z%3D_o0%2Bcw%40mail.gmail.com.

Rafael Spring

unread,
Oct 23, 2017, 12:24:46 PM10/23/17
to ceres-...@googlegroups.com
Keir, Sameer,

thanks for the information. I'll have a look into the Ceres code then and see if I can wrap my head around it :)

Best,
Rafael

Rafael Spring
CTO, Co-Founder
DotProduct LLC / DotProduct GmbH
@DotProduct3D

--
You received this message because you are subscribed to a topic in the Google Groups "Ceres Solver" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ceres-solver/Q4rdFhkEcbY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ceres-solver+unsubscribe@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CADpYijFgatJsyOek5qFjhNPOTfPaE40tFsAH_n3v5XtziErUqQ%40mail.gmail.com.

Rafael Spring

unread,
Oct 23, 2017, 4:31:31 PM10/23/17
to ceres-...@googlegroups.com
Actually, just thinking about this a little more: Is there a way to obtain the updated state vector during the optimization, specifically every time there is a new linearization point (but before any of the CostFunctions are evaluated)?

If yes, that would allow us to catch the event and precompute at least J and r for the dense terms and then hand those over to Ceres as part of the evaluation of one or several large CostFunctions. Then Ceres could take it from there.

Would that be possible?

Sameer Agarwal

unread,
Oct 23, 2017, 5:55:30 PM10/23/17
to ceres-...@googlegroups.com
Rafael,
This is something I have thought about doing, but I have not gotten around to designing the feature. 
Would you like to put a proposal/design for this? I can do it, but it will likely take me some time to get to it.
Sameer





To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "Ceres Solver" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ceres-solver/Q4rdFhkEcbY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ceres-solver...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CAOJdUvRd_9E2V%3D8%2BgEq3_OGdfQKaNDLNveAkacMaH-q9V1C38A%40mail.gmail.com.

Rafael Spring

unread,
Oct 24, 2017, 1:34:56 PM10/24/17
to ceres-...@googlegroups.com
Hi Sameer,
unfortunately I'm also a bit short on time so it could take a while before I find time to work on this, but I'd certainly let you know before I begin.

Rafael

Rafael Spring
CTO, Co-Founder
DotProduct LLC / DotProduct GmbH
@DotProduct3D

To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to a topic in the Google Groups "Ceres Solver" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ceres-solver/Q4rdFhkEcbY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ceres-solver+unsubscribe@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CAOJdUvRd_9E2V%3D8%2BgEq3_OGdfQKaNDLNveAkacMaH-q9V1C38A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Ceres Solver" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ceres-solver/Q4rdFhkEcbY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ceres-solver+unsubscribe@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/CABqdRUAJbWSGAv9c-zF3SFSpcYXvrZi2HSgNDsjos9v9KMQmfw%40mail.gmail.com.
Reply all
Reply to author
Forward
0 new messages