How to use DynamicCostFunctionToFunctor

[Ginger C.]

Hi，Everyone

       I've been using DynamicCostFunctionToFunctor recently to solve the BA problem in SLAM：There is a keyfram, there are n feature points pi and corresponding n spatial points Pj on the keyframe. It can construct an error function：error = ||pi – π(RPj+t)||^2, π(·) is the projection function of the camera.

       Here R and t is unknown, so I try to optimize P and use a set of P to express R、t by an ThirdPartyFunction f：R= f(P),t=f(P), R and t are functions of P, respectively. Because the function parsed form cannot be written， So I thought of DynamicCostFunctionToFunctor to let Ceres automatically derive the derivation

Here I only need to optimize P, but through experiments, it is found that the optimization speed is very slow, and sometimes the optimization fails. Am I misunderstanding it, or there is a problem with the code.

[Dmitriy Korchemkin]

Hi, Ginger

This way ThirdPartyFunction will be evaluated (and numerically differentiated!) for each residual, which should take quite a while; you can use EvaluationCallback to pre-compute ThirdPartyFunction and its derivatives once for a single frame. Supposedly this will increase speed substantially.
By the way, Is it possible to use the inverse function theorem for obtaining dR/dP and dT/dP?

Another speed-wise limitation might be usage of SPARSE_NORMAL_CHOLESKY linear solver. As far as I understand, your problem still has a Schur-compatible structure (but with keyframe-specific parameter blocks having dimension of 21*3 instead of 6..7).
Probably you'll get better results with DENSE_SCHUR / SPARSE_SCHUR linear solver (might require specifying elimination order by hand).

Keep in mind that your approach results in denser Jacobian/Hessian than conventional BA: each residual now depends on (21+1)*3 parameters (compared to 9..10 in conventional BA).

Another approach might be to make a basic BA on points & poses with "regularization" with deviations of pose estimates from ThirdPartyFunction predictions. This might require some tuning of relative weighting of reprojection errors vs pose deviations, but this way you'll get a simpler jacobian/hessian structure for each observation.

[Dmitriy Korchemkin]

Hi, Ginger

This way ThirdPartyFunction will be evaluated (and numerically differentiated!) for each residual, which should take quite a while; you can use EvaluationCallback to pre-compute ThirdPartyFunction and its derivatives once for a single frame. Supposedly this will increase speed substantially.

By the way, Is it possible to use the inverse function theorem for obtaining dR/dP and dT/dP?

Another speed-wise limitation might be usage of SPARSE_NORMAL_CHOLESKY linear solver. As far as I understand, your problem still has a Schur-compatible structure (but with keyframe-specific parameter blocks having dimension of 21*3 instead of 6..7).
Probably you'll get better results with DENSE_SCHUR / SPARSE_SCHUR linear solver (might require specifying elimination order by hand).

Keep in mind that your approach results in denser Jacobian/Hessian than conventional BA: each residual now depends on (21+1)*3 parameters (compared to 9..10 in conventional BA).

Another approach might be to make a basic BA on points & poses with "regularization" with deviations of pose estimates from ThirdPartyFunction predictions. This might require some tuning of relative weighting of reprojection errors vs pose deviations, but this way you'll get a simpler jacobian/hessian structure for each observation.

On Fri, May 13, 2022 at 1:51 PM Ginger C. <ncu...@gmail.com> wrote:

Hi，Everyone

       I've been using DynamicCostFunctionToFunctor recently to solve the BA problem in SLAM：There is a keyfram, there are n feature points pi and corresponding n spatial points Pj on the keyframe. It can construct an error function：error = ||pi – π(RPj+t)||^2, π(·) is the projection function of the camera.

       Here R and t is unknown, so I try to optimize P and use a set of P to express R、t by an ThirdPartyFunction f：R= f(P),t=f(P), R and t are functions of P, respectively. Because the function parsed form cannot be written， So I thought of DynamicCostFunctionToFunctor to let Ceres automatically derive the derivation

Here I only need to optimize P, but through experiments, it is found that the optimization speed is very slow, and sometimes the optimization fails. Am I misunderstanding it, or there is a problem with the code.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ceres-solver/1310851a-9086-4291-a5b4-6d882f091439n%40googlegroups.com.

--

Dmitriy A Korchemkin

[Sameer Agarwal]

Hi Ginger,

A few comments.

1. DynamicCeresBA as I read it is a DynamicAutoDiffFunction which all it does is call a DynamicNumericDiffCostFunction by wrapping it in a DynamicCostFunctiontoFunctor. There is no need for this. The autodiff is not doing any work here, why not just use the DynamicNumericDiffCostFunction object you are constructing.

2. I am also surprised by the structure of your DynamicGetRtByFunction object. Most of this function is differentiable, it is only the ThirdPartyFunction which is the one call to an external library which cannot be differentiated. So you are paying a high cost for differentiating the whole function. 

My suggestion is that you first construct a DynamicNumericDiffCostFunction using ThirdPartyFunction, and then wrap this using DynamicCostFunctionToFunctor and call it in DynamicGetRTByFunction, which you then use as input to a DynamicAutoDiffFunction.

All that said, ThirdPartyFunction is still taking a lot of inputs and its going to be expensive to numerically differentiate it. It is worth thinking about the optimization problem that is being solved and if you can compute the derivative of the solution manually.

Hope this helps,

Sameer

--