Generate cffi bindings from user-defined library location

[Ghislain Vaillant]

Dear python-cffi community,

I played around with CFFI and successfully generated bindings to the FFTW library here:

https://github.com/ghisvail/fftw-cffi

Right now, the FFTW library is discovered via pkg-config (using the python-pkgconfig module) directly in the
bindings generator script:

https://github.com/ghisvail/fftw-cffi/blob/master/builders/build_bindings.py

and the corresponding ffi object is injected to the setup via the `cffi_modules` parameter:

https://github.com/ghisvail/fftw-cffi/blob/master/setup.py

Although this set-up is more flexible than a plain call to `ffi.set_sources(..., libraries=['fftw3', 'fftw3_threads'])`,
I could provide true support for arbitrary install of FFTW by letting the user call:

python setup.py build -I <path-to-FFTW-includes> -L <path-to-FFTW-libraries>

Does it sound sensible, or is there a simpler solution that I did not think of? Do you guys know of an example,
which achieves a similar thing?

Thanks,
Ghis

[Mark Santcroos]

Hi Ghislain,

> On 29 Mar 2016, at 20:09 , Ghislain Vaillant <ghis...@gmail.com> wrote:
> Right now, the FFTW library is discovered via pkg-config (using the python-pkgconfig module) directly in the
> bindings generator script:

> https://github.com/ghisvail/fftw-cffi/blob/master/setup.py

I use a similar approach, as in relying on pkg-config:
https://github.com/open-mpi/ompi/blob/master/orte/bindings/python/src/orte-cffi/build.py

AFAIK I couldn't use the pkg-config python package so had to roll my own :(

> Although this set-up is more flexible than a plain call to `ffi.set_sources(..., libraries=['fftw3', 'fftw3_threads'])`,
> I could provide true support for arbitrary install of FFTW by letting the user call:

Isn't pkg-config generic enough or are there installations of FFTW that don't include pkg-config?

Gr,

Mark

[Ghislain Vaillant]

Le mercredi 30 mars 2016 08:34:13 UTC+1, Mark Santcroos a écrit :

Hi Ghislain,

> On 29 Mar 2016, at 20:09 , Ghislain Vaillant <ghis...@gmail.com> wrote:
> Right now, the FFTW library is discovered via pkg-config (using the python-pkgconfig module) directly in the
> bindings generator script:
> https://github.com/ghisvail/fftw-cffi/blob/master/setup.py

I use a similar approach, as in relying on pkg-config:
https://github.com/open-mpi/ompi/blob/master/orte/bindings/python/src/orte-cffi/build.py

AFAIK I couldn't use the pkg-config python package so had to roll my own :(

I have had a look and it is pretty well made. Well done. So far, the pkgconfig Python module works although you
need an extra conversion step to convert the sets in the dictionary returned by pkgconfig to lists / tuples, otherwise
distutils is not happy apparently.

> Although this set-up is more flexible than a plain call to `ffi.set_sources(..., libraries=['fftw3', 'fftw3_threads'])`,
> I could provide true support for arbitrary install of FFTW by letting the user call:

Isn't pkg-config generic enough or are there installations of FFTW that don't include pkg-config?

I can think of 2 use cases: 1) Windows, where usage of pkg-config is not as straightforward, and
2) bundling, where someone can have its own version of FFTW and want the ffi to link with it using rpath.

Ghis

[Mark Santcroos]

> On 30 Mar 2016, at 11:46 , Ghislain Vaillant <ghis...@gmail.com> wrote:
> I have had a look and it is pretty well made. Well done.

Thanks!

> So far, the pkgconfig Python module works although you
> need an extra conversion step to convert the sets in the dictionary returned by pkgconfig to lists / tuples, otherwise
> distutils is not happy apparently.

Roughly true, but there was a couple of additional minor details that prevented me from using it. I should probably find the time to send a PR instead of having my own.

> > Although this set-up is more flexible than a plain call to `ffi.set_sources(..., libraries=['fftw3', 'fftw3_threads'])`,
> > I could provide true support for arbitrary install of FFTW by letting the user call:
>
> Isn't pkg-config generic enough or are there installations of FFTW that don't include pkg-config?
>
> I can think of 2 use cases: 1) Windows, where usage of pkg-config is not as straightforward, and

ACK, but that was/is not an issue for me.

> 2) bundling, where someone can have its own version of FFTW and want the ffi to link with it using rpath.

That could be solved by adding that location to the PKG_CONFIG_PATH right?

Mark

[Ghislain Vaillant]

Le mercredi 30 mars 2016 10:55:58 UTC+1, Mark Santcroos a écrit :

> On 30 Mar 2016, at 11:46 , Ghislain Vaillant <ghis...@gmail.com> wrote:
> I have had a look and it is pretty well made. Well done.

 I should probably find the time to send a PR instead of having my own.

That would be awesome.
 

> > Although this set-up is more flexible than a plain call to `ffi.set_sources(..., libraries=['fftw3', 'fftw3_threads'])`,
> > I could provide true support for arbitrary install of FFTW by letting the user call:
>
> Isn't pkg-config generic enough or are there installations of FFTW that don't include pkg-config?
>
> I can think of 2 use cases: 1) Windows, where usage of pkg-config is not as straightforward, and

ACK, but that was/is not an issue for me.

> 2) bundling, where someone can have its own version of FFTW and want the ffi to link with it using rpath.

That could be solved by adding that location to the PKG_CONFIG_PATH right?

Assuming the bundle contains it yes. If not, you need to have some sort of way to

What I meant to say is that support for pkg-config is nice but (optional) control via:

python setup build_ext -I[...] -L[...]  -R[...]

would be even nicer and cover pretty much all use cases.

Anyone?

Any example of integration of build_ext with CFFI?

Ghis