Module initialization on PyPy

Skip to first unread message

Alex Gaynor

Dec 13, 2021, 7:44:15 AM12/13/21
Hi all,

I'm one of the developers of PyCA cryptography. We've been using cffi
for many many years at this point.

Right now we're looking at doing something novel: rather than having
cffi emit a module that's normally imported, instead have it emit an
object file which is linked into another module, and that other module
is responsible for calling `PyInit_{module}` and adding it to the
module hierarchy. By putting both modules in a single .so, we ensure
that they link the same version of external libraries (in our case,

You can read about the motivation for this and see the PoC here

Amazingly, this all seems to work fine, but only on CPython. On PyPy,
we encounter linker errors, because cffi modules on PyPy do not
actually emit `PyInit_{module}` symbols! They have an entirely
different API for module initialization

What's the best way to accomplish this? Essentially we need some API
that can find a function ptr inside the same .so as the calling code
and return the freshly created module for it.

I suspect this requires a new API from cffi, but I'd appreciate any thoughts.


All that is necessary for evil to succeed is for good people to do nothing.

Matti Picus

Dec 18, 2021, 5:58:54 PM12/18/21
to python-cffi
Perhaps the PyPy-cffi backend could emit a `PyInit_{module}` function that would encapsulate the code you pointed to. I don't think this needs any changes in cffi itself, rather the PyPy-specific _cffi_backend could be adapted to emit this code. Of course this is all just an idea. I opened to discuss this further.

Alex Gaynor

Dec 18, 2021, 6:06:04 PM12/18/21
Thanks Matti!

I don't quite follow how cffi_start_python works so I'm not sure it's
sufficient, but if it works I'm happy with it :-)

If we'd like something even simpler, I think exposing
load_cffi1_module via cpyext (e.g. PyObject *PyPyLoadCFFIModule(char
*name, INITFUNCPTR initptr)) would be enough for what we'd like to do!

> --
> -- python-cffi: To unsubscribe from this group, send email to For more options, visit this group at
> ---
> You received this message because you are subscribed to the Google Groups "python-cffi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to
> To view this discussion on the web visit
Reply all
Reply to author
0 new messages