dealing with a gazillion C++ template instantiations

[Martin R. Albrecht]

Hi there,

we’re using Cython extensively in fpylll [fn:1] which wraps fplll
[fn:2]. The latter uses templates for integer and floating point
types. In this e-mail I’m basically asking if anybody got any
clever tricks up their sleeves to make writing Cython interfaces
for all combinations of template instantiation easier. One way
could be to use/emulate some sort of Cython macros, perhaps using
the in-place string replacements (?)

Here is our current, somewhat awkward approach:

There’s a C++ class:

https://github.com/fplll/fpylll/blob/zt-long/src/fpylll/fplll/fplll.pxd#L381

cdef cppclass MatGSO[ZT, FT]:
MatGSO(Matrix[ZT] B, Matrix[ZT] U, Matrix[ZT] UinvT, int
flags)

…

const int enable_int_gram
const int enable_row_expo
const int enable_transform


ZT can be int or mpz_t, FT can be double, long double, double
double, quad double, dpe or mpfr_t. Thus, quite a few
combinations. For this we defined:

https://github.com/fplll/fpylll/blob/zt-long/src/fpylll/fplll/decl.pxd#L103

IF HAVE_QD:
ctypedef union mat_gso_core_t:
MatGSO[Z_NR[mpz_t], FP_NR[d_t]] *mpz_d
MatGSO[Z_NR[mpz_t], FP_NR[ld_t]] *mpz_ld
MatGSO[Z_NR[mpz_t], FP_NR[dpe_t]] *mpz_dpe
MatGSO[Z_NR[mpz_t], FP_NR[dd_t]] *mpz_dd
MatGSO[Z_NR[mpz_t], FP_NR[qd_t]] *mpz_qd
MatGSO[Z_NR[mpz_t], FP_NR[mpfr_t]] *mpz_mpfr
MatGSO[Z_NR[long], FP_NR[d_t]] *long_d
MatGSO[Z_NR[long], FP_NR[ld_t]] *long_ld
MatGSO[Z_NR[long], FP_NR[dpe_t]] *long_dpe
MatGSO[Z_NR[long], FP_NR[dd_t]] *long_dd
MatGSO[Z_NR[long], FP_NR[qd_t]] *long_qd
MatGSO[Z_NR[long], FP_NR[mpfr_t]] *long_mpfr
ELSE:
ctypedef union mat_gso_core_t:
MatGSO[Z_NR[mpz_t], FP_NR[d_t]] *mpz_d
MatGSO[Z_NR[mpz_t], FP_NR[ld_t]] *mpz_ld
MatGSO[Z_NR[mpz_t], FP_NR[dpe_t]] *mpz_dpe
MatGSO[Z_NR[mpz_t], FP_NR[mpfr_t]] *mpz_mpfr
MatGSO[Z_NR[long], FP_NR[d_t]] *long_d
MatGSO[Z_NR[long], FP_NR[ld_t]] *long_ld
MatGSO[Z_NR[long], FP_NR[dpe_t]] *long_dpe
MatGSO[Z_NR[long], FP_NR[mpfr_t]] *long_mpfr

Our Cython class uses this union:

https://github.com/fplll/fpylll/blob/zt-long/src/fpylll/fplll/gso.pxd#L6

cdef class MatGSO:
cdef fplll_gso_type_t _type
cdef mat_gso_core_t _core

And we then do runtime checks:

https://github.com/fplll/fpylll/blob/zt-long/src/fpylll/fplll/gso.pyx#L344

@property
def int_gram_enabled(self):
if self._type == gso_mpz_d:
return bool(self._core.mpz_d.enable_int_gram)
IF HAVE_LONG_DOUBLE:
if self._type == gso_mpz_ld:
return bool(self._core.mpz_ld.enable_int_gram)
if self._type == gso_mpz_dpe:
return bool(self._core.mpz_dpe.enable_int_gram)
IF HAVE_QD:
if self._type == gso_mpz_dd:
return bool(self._core.mpz_dd.enable_int_gram)
if self._type == gso_mpz_qd:
return bool(self._core.mpz_qd.enable_int_gram)
if self._type == gso_mpz_mpfr:
return bool(self._core.mpz_mpfr.enable_int_gram)

if self._type == gso_long_d:
return bool(self._core.long_d.enable_int_gram)
IF HAVE_LONG_DOUBLE:
if self._type == gso_long_ld:
return bool(self._core.long_ld.enable_int_gram)
if self._type == gso_long_dpe:
return bool(self._core.long_dpe.enable_int_gram)
IF HAVE_QD:
if self._type == gso_long_dd:
return bool(self._core.long_dd.enable_int_gram)
if self._type == gso_long_qd:
return bool(self._core.long_qd.enable_int_gram)
if self._type == gso_long_mpfr:
return bool(self._core.long_mpfr.enable_int_gram)

raise RuntimeError("MatGSO object '%s' has no core."%self)

Of course, the combinatorics of that approach explode eventually.
It would be nice to write, say, some sort of macro GSO_DISPATCH
which would essentially implement that switch statement for us and
would write the appropriate code e.g.

return GSO_DISPATCH(self, enable_int_gram)

Any ideas, tricks?

Cheers,
Martin

--

_pgp: https://keybase.io/martinralbrecht
_www: https://martinralbrecht.wordpress.com
_jab: martinr...@jabber.ccc.de
_otr: 47F43D1A 5D68C36F 468BAEBA 640E8856 D7951CCF

* Footnotes

[fn:1] https://github.com/fplll/fpylll

[fn:2] https://github.com/fplll/fplll

[Jeroen Demeyer]

Obvious answer: auto-generate your code. Either with a templating engine
like Jinja2 or just manually write the generator code.

We are using the latter approach for CyPari2:
https://github.com/defeo/cypari2

[Martin Albrecht]

Hi all, 

I played around with the various options I could think of and wrote up my impressions at 

  https://martinralbrecht.wordpress.com/2017/07/23/adventures-in-cython-templating/

Cheers,

Martin

[Robert Bradshaw]

On Wed, Jul 12, 2017 at 6:55 AM, Jeroen Demeyer <jdem...@cage.ugent.be> wrote:
> Obvious answer: auto-generate your code. Either with a templating engine
> like Jinja2 or just manually write the generator code.

+1, this is definitely the approach I'd take. You could auto-generate
the high-level dispatchers as well. Good point about editor support; I
suppose you could to a simple find-replace of TYPE to {{T}} before
passing to jinja2 (if you even need the latter) to keep the code
valid.

> We are using the latter approach for CyPari2:
> https://github.com/defeo/cypari2
>
>

> --
>
> --- You received this message because you are subscribed to the Google
> Groups "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to cython-users...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.