How to deal with transitive linker dependencies

[Tobias Diez]

Hi everyone,

I have a question concerning the best practices of dealing with transitive dependencies.

The setup is roughly like this:

- There is a dynamic C library (pari in my case)

- Cython project A makes use of this library (cypari2 for me)

- Cython project B uses Cython project A (sagemath here)

It looks like B also needs to link against the C library for it to work (**). But how does one do this given the various Python packaging methods. If you build Cython project A locally and link against a locally installed version of the C library, it's easy. But what if Cython project A comes i wheel that bundles the C library?

---
** The issue that I'm currently experience when I'm not explicitly linking against the C library is the following:
- Cython project A makes use of "cdef extern from foo.h" in a pxd file to declare a function fct

- Cython project B cimports that pxd from Cython project A

- Upon runtime I get ImportError: project-b.so: undefined symbol: fct

[Oscar Benjamin]

Hi Tobias,

This is not really a Cython question so much. The issue arises for any
projects with C extension modules trying to use a shared library.
Another example of this would be NumPy, SciPy and OpenBLAS. There is a
description of issues in that case here:

https://pypackaging-native.github.io/key-issues/native-dependencies/blas_openmp/

There is not really a good solution for this right now and as you can
see NumPy and SciPy handle this by distributing separate OpenBLAS
libraries in each wheel. I don't know if that is even an option in the
sagemath case since you might need projects A and B to be using the
exact same pari.

The first problem is that in general there is no way to express ABI
constraints among wheels in PyPI-land. You can work around this if you
control both projects (cypari2 and sagemath) and use exact version
pins for dependency constraints in the wheels (sagemath pins an exact
version of cypari2). I think there will be situations where this work
around does not work though such as if someone combines packages built
locally with wheels from PyPI.

There is a discussion related to this here:
https://discuss.python.org/t/shipping-common-libraries-in-a-dedicated-wheel/76407

That also links to this proposal which lays out in some detail the
mechanics of what you need to do to make this work:
https://github.com/njsmith/wheel-builders/blob/pynativelib-proposal/pynativelib-proposal.rst

--
Oscar


On Thu, 18 Sept 2025 at 13:23, 'Tobias Diez' via cython-users

> --
>
> ---
> You received this message because you are subscribed to the Google Groups "cython-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/cython-users/4585ff87-ed47-489f-8527-fca578b8d539n%40googlegroups.com.