ceres::AutoDiffCostFunction template instantiation chokes gcc

[Kip Warner]

Hello Group,

I've recently explored changing the dimensionality of my embeddings
from a few dozen to several thousand. But I ran into a compile time
issue with ceres::AutoDiffCostFunction.

What I had previously done was already discussed on the list, for each
training example I do:

   ceres::CostFunction *CostFunctor = new ceres::AutoDiffCostFunction
   <
       TripletCostFunctor,
       1, /* One residual per cost functor */
       Dimensionality /* Size of parameter block */
   >(new TripletCostFunctor(*this, single_training_example));
  
   CeresProblem.AddResidualBlock(
       CostFunctor,
       nullptr,
       StartingWeights);

The only thing I have changed is an increase in the Dimensionality
template parameter to now be in the several thousand. But this is
causing GCC to choke:

   /usr/include/ceres/internal/autodiff.h:192:63: fatal error: template instantiation depth exceeds maximum of 900 (use '-ftemplate-depth=' to increase the maximum)
     192 |     Make1stOrderPerturbation<j + 1, N, Offset, T, JetT>::Apply(src, dst);

If I bump the -ftemplate-depth to 3000, it compiles fine. But that's
well beyond what ANSI/ISO C++ requires of 1024. Would you recommend I
increase -ftemplate-depth or something else?

I feel like this is an inelegant solution. But then I suspect many ML
applications deal with vectors of large dimensionality, so I figured I
would ask an expert.

Yours truly,

--
Kip Warner
OpenPGP signed/encrypted mail preferred
https://www.thevertigo.com

[Sameer Agarwal]

yes such high template parameter count will likely cause compiler problems.

I do not know what the c++ standard requires, but this is just a lot more work for the compiler. 

so yes increase the template-depth.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/ceres-solver/6119f0fe9ae02ef174829564b704a0898e9e0f66.camel%40thevertigo.com.

[Kip Warner]

On Mon, 2024-12-02 at 10:54 -0800, 'Sameer Agarwal' via Ceres Solver
wrote:

> yes such high template parameter count will likely cause compiler
> problems.
> I do not know what the c++ standard requires, but this is just a lot
> more work for the compiler. 

It's apparently 1024 in C++11:

https://gcc.gnu.org/onlinedocs/gcc/C_002b_002b-Dialect-Options.html#index-ftemplate-depth

> so yes increase the template-depth.

I can increase it no problem. But my question is whether that's
appropriate? I imagine high dimensionality is not unusual in ML
problems, but I've also not ever seen a situation where the the
template instantiation depth was ever exhausted.

[Sameer Agarwal]

It's fine. This is about template explosion and auto diff.

--
You received this message because you are subscribed to the Google Groups "Ceres Solver" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ceres-solver...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/ceres-solver/b1e97098ef26f049cbe3c6172ed2a7532d67674d.camel%40thevertigo.com.

[Kip Warner]

On Mon, 2024-12-02 at 11:11 -0800, 'Sameer Agarwal' via Ceres Solver
wrote:

> It's fine. This is about template explosion and auto diff.

Got it. Thanks Sameer. Helpful as always.