Using CFFI with bundled sources

[Tin Tvrtković]

Hello,

I'm trying to teach myself CFFI on a toy project. Assume I'd like to expose a small C library to Python (say, a JSON parser). I'd like to bundle the C code in my Python package to make it easier on users (like ujson, rapid json, lz4-cffi and a bunch of other libraries do). I'd also like to use the API, out-of-line approach (ffi.set_source). I'm working on Linux.

So I copy the C sources somewhere into my tree, and add a file or two to wrap it nicely on the C level. Then, in my setup.py I declare an Extension('libx', sources=..., include_dirs=..., extra_compile_args=...). Now 'python setup.py build' will make me a .so library, but called 'libx.pypy-26-x86_64-linux-gnu.so'. I don't know how to get rid of the suffix.

Then I prepare a x_build.py file with my FFI object, with my ffi.set_source("_x", custom_src=..., libraries=..., library_dirs=...) and my ffi.cdefs, and I put my libx into the libraries argument to set_source, but I need the 'pypy-26-x86_64-linux-gnu' part in there too, otherwise ld complains. This step gives me another .so which I can import and use it's lib and ffi members.

Am I doing everything right? How do I get rid of the suffix in my shared library? I realize this is a distutils/setuptools thing, but I thought maybe there's a common answer since this is probably a common use case. Or I could be fundamentally doing something wrong...

Other than this, I'm finding CFFI great fun to work with. :)

[Armin Rigo]

Hi Tin,

On Sun, Feb 14, 2016 at 3:25 AM, Tin Tvrtković <tinch...@gmail.com> wrote:
> Am I doing everything right? How do I get rid of the suffix in my shared
> library? I realize this is a distutils/setuptools thing, but I thought maybe
> there's a common answer since this is probably a common use case. Or I could
> be fundamentally doing something wrong...

You should be able in your case to produce only one .so, instead of
two. Remove the Extension('libx', sources=..., include_dirs=...,
extra_compile_args=...), and arrange for every C source to be compiled
into the cffi-generated .so. This should be as easy as copying the
same arguments to set_source():

ffi.set_source("_x", sources=..., include_dirs=..., extra_compile_args=...)

which asks set_source() to make a .so containing both the
cffi-generated C file plus all the ones listed in "sources", with the
given compilation args.


A bientôt,

Armin.

[Tin Tvrtković]

Thank you, Armin! That makes sense, is simpler and kinda obvious in retrospective. :)