Automatic differentiation and whitening Jacobian

[Stefan Gächter]

Dear all, I am little bit afraid to ask this question, but I am wondering where automatic  differentiation is used in GTSAM. I studied "Factor Graphs for Robot Perception", where automatic differentiation is mentioned in section 6.1.3.  In section 2.3, non-linear least-squares is explained and how the measurement covariance is used to whiten the Jacobian, equation (2.10). When applying automatic differentiation, is the whitening done likewise? Or do I misunderstand something?

[Stefan Gächter]

Jing Dong's GTSAM tutorial gave some further insight, but did not yet fully answer my question. If somebody could point me to further literature on this topic, I would appreciate it.

[Dellaert, Frank]

AD is implemented by expressions in GTSAM (see Expression​ class defined in expression.h​). Seems that the only place where I write this up in a meaningful way is on the old Bitbucket wiki, which did not get migrated to GitHub 🙁

https://bitbucket.org/gtborg/gtsam/wiki/GTSAM_Expressions

It points to an SFM example that is in github.

The AD is not done all the way to code level, as in Ceres: it does AD only on composing basic functions that need to have Jacobians defined. Things that automatically have derivatives are between​, compose​, and add​. Other "basic functions" such as Pose3::transform_from​ define their own Jacobians.

This "multivariate" approach allows us in many cases to simplify the basic function Jacobians and is more efficient than going all the way to the scalar level, where all notions of manifolds, tangent spaces, etc. are lost.

Some handy expressions, types, and sugar is provided in the expressions.h​ files in nonlinear, slam, and navigation subfolders. And, of course, testExpression​ unit tests also provide examples on how to create your own.

Frank 

gtborg / gtsam / wiki / GTSAM Expressions — Bitbucket Expressions. In many cases one can use GTSAM 4 Expressions to implement factors. Expressions are objects of type Expression<T>, and there are three main expression flavors:. constants, e.g., Expression<Point2> kExpr(Point2(3,4)); unknowns, e.g., Expression<Point3> pExpr(123) where 123 is a key.; functions, e.g., Expression<double> sumExpr(h, kexpr, pExpr); The latter case is an example of ... bitbucket.org

---

From: gtsam...@googlegroups.com <gtsam...@googlegroups.com> on behalf of Stefan Gächter <ma...@gachter.name>
Sent: Tuesday, August 10, 2021 10:28
To: gtsam users <gtsam...@googlegroups.com>
Subject: [GTSAM] Re: Automatic differentiation and whitening Jacobian

 

Jing Dong's GTSAM tutorial gave some further insight, but did not yet fully answer my question. If somebody could point me to further literature on this topic, I would appreciate it.

On Tuesday, 10 August 2021 at 16:11:52 UTC+2 Stefan Gächter wrote:

Dear all, I am little bit afraid to ask this question, but I am wondering where automatic  differentiation is used in GTSAM. I studied "Factor Graphs for Robot Perception", where automatic differentiation is mentioned in section 6.1.3.  In section 2.3, non-linear least-squares is explained and how the measurement covariance is used to whiten the Jacobian, equation (2.10). When applying automatic differentiation, is the whitening done likewise? Or do I misunderstand something?

--
You received this message because you are subscribed to the Google Groups "gtsam users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gtsam-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gtsam-users/7b463f37-d545-4298-b770-b28db66c6f2dn%40googlegroups.com.

[Stefan Gächter]

Thanks, Frank, for the quick reply. I took a look a the expression.

I am comparing GTSAM's and Cere's automatic differentiation. As you point out, the automatic differentiation in Ceres is different. When taking a look at Ceres, I was wondering if and how the whitening of the measurement Jacobian is done in case of automatic differentiation.

In case of GTSAM, if I understand it correctly, the measurement Jacobians are whitened in case of automatic differentiation.

From   "Factor Graphs for Robot Perception", the linearized least-square problem is

with the measurement Jacobians

which can be normalized or whitened

The normal equation or whitened system is then

My understanding is that in GTSAM the measurement Jacobians H can be provided explicitly or computed by automatic differentiation and in both cases, the Jacobian are whitened with the square root information matrix. That is, GTSAM always solves the normal equations.

Is my understanding correct?

In Ceres, the case seems to be different, when using automatic differentiation. Though, I have no confirmation yet.

In Ceres, the problem can only be normalized, if the Jacobians are provided explicitly. If automatic differentiation is used, then the best one can do is to normalize residuals. The system is then

where H are the unwhitened Jacobians from the automatic differentiation.

Would this result the same solution as for the whitened system? I would not expect so. I am little bit confused with Ceres here.

[Dellaert, Frank]

All whitening happens in the noise models, see noiseModel.h. You pair up an expression with a noise model by instantiating an ExpressionFactor.

Frank

---

From: gtsam...@googlegroups.com <gtsam...@googlegroups.com> on behalf of Stefan Gächter <ma...@gachter.name>

Sent: Wednesday, August 11, 2021 6:41:58 AM
To: gtsam users <gtsam...@googlegroups.com>
Subject: Re: [GTSAM] Re: Automatic differentiation and whitening Jacobian

 

To view this discussion on the web visit https://groups.google.com/d/msgid/gtsam-users/c738087d-4f62-4c03-8e5c-a73bdd232fcan%40googlegroups.com.

[Stefan Gächter]

Thanks, Frank. Yes, I saw that. If I understand it correctly, then `WhitenSystem` does exactly what I described above. Ceres, then, has to be applied with care, though, still looking for somebody who confirms my finding: Ceres cannot solve a whitened system when auto-diff is used.

[Stefan Gächter]

Ooops, I was wrong on part of Ceres. For sure, Ceres can solve a whitened system with auto-diff. I was stuck in the linearized world... Anyway, learned something and have a much better understanding of GTSAM now. Thanks for your patience.