How to build cffi without linking against libffi?

[f3flight (f3flight)]

Hi cffi community! I noticed that cffi wheels for Linux published on pypi.org do not link against libffi. Instead, they link against libpthread.so.0 (and both link against libc.so.6)

$ auditwheel -v show cffi-1.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl ... The following external shared libraries are required by the wheel: { "libc.so.6": "/lib/libc.so.6", "libpthread.so.0": "/lib/libpthread.so.0" }

At the same time, libffi is not bundled in cffi wheels on pypi.org:

$ zipinfo -1 cffi-1.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl | grep '.so' _cffi_backend.cpython-310-x86_64-linux-gnu.so

However, if I build cffi myself on my hosts, it links against libffi.so.8 or libffi.so.6 (depending on the OS):

$ auditwheel -v show cffi-1.15.1/dist/cffi-1.15.1-cp310-cp310-linux_x86_64.whl ... The following external shared libraries are required by the wheel: { "libc.so.6": "/lib/libc.so.6", "libffi.so.8": "/lib/libffi.so.8.1.0" }

How can I build cffi without linking against libffi?
Do pypi.org cffi wheels work without libffi installed? Is there a quick way to test the library with some python snippet/script to ensure it actually works?

Disclaimer: I have not used the library myself and it's just a part of dependency tree and we're running into issues where cffi wheel built on one Linux distro doesn't work on another due to linking to libfii v6 on one and libffi v8 on another.

[Armin Rigo]

Hi,

On Tue, 9 May 2023 at 02:18, f3flight (f3flight) <f3fl...@gmail.com> wrote:
> Do pypi.org cffi wheels work without libffi installed?

I'm pretty sure they don't work without libffi installed. I have no
idea why libffi is not mentioned in the wheels.


Armin

[f3flight (f3flight)]

Answering my own question and asking a bit more - I reviewed this - https://foss.heptapod.net/pypy/cffi/-/blame/branch/default/.github/workflows/ci.yaml#L79
Looks like it was written by Matt Davis @nitzmahone but I don't know the email so not sure how to reach out.
Based on what I see here, the wheels are built against a locally downloaded hardcoded version of libffi, and static linking is used (which is not default), which would explain why I see no linking to libffi and also no embedding of libffi into the package. It also means that libffi is *not* required in the system for cffi wheels to work - they are self-sufficient and staically link = copy all necessary libffi code into _cffi_backend.so library.

[f3flight (f3flight)]

That is true of course only for wheels build by this workflow, for example those published on pypi.org. If someone were to build wheels on their host, they would have a dynamically linked library instead and the issue which I ran into. I think the build env flags and settings should better go into the python code (setup.py for example) so that the experience and build results are the same for everyone. Anyways, mystery solved.

[Armin Rigo]

Hi !

On Wed, 10 May 2023 at 03:40, f3flight (f3flight) <f3fl...@gmail.com> wrote:
> static linking is used

Ah right, yes. It's a far saner approach to making portable binaries
for Linux, particularly for libraries like libffi that are small.

> That is true of course only for wheels build by this workflow, for example those published on pypi.org. If someone were to build wheels on their host, they would have a dynamically linked library instead and the issue which I ran into. I think the build env flags and settings should better go into the python code (setup.py for example) so that the experience and build results are the same for everyone. Anyways, mystery solved.

I'd consider pull requests to move some of that logic into setup.py,
if it makes sense to Matt Davis, the release manager
<matt...@redhat.com>. I guess you don't need many flag changes? I
thought the main issue was having a static `libffi.a` first in the
path where gcc searches for libraries, or something, which doesn't
really look like something to configure in setup.py.




Armin Rigo