automatic cdef generation / "float complex"

[Tom Krauss]

I recently started using CFFI for some wrapping at work and it's going really well. Great library, very useful tool, thanks for all your efforts!

I would like to ask the community about a couple things. A quick search of this list and google didn't result in anything, although perhaps I missed it - please refer me to better sources if they are available!

1) Any advice for dealing with "complex float" or "complex double"? For example:

#include <stdint.h>

#include <complex.h>

int32_t process_samples(complex float *samples, uint32_t nsamp);

Currently, this gives a "cannot parse" error. Our solution is to use float * or void * instead in the CDEF, and then use "ffi.cast" prior to calling these functions. However, that's suboptimal in a number of ways (can't use the "natural" type in python, have to change the cdef to be different from the actual header)

2) Are there any techniques or tools to automatically generate the CDEF from a header file? Let me explain what I am asking. In each case, when I create a C header, I then need to define a CDEF for it. I cannot use just the header, as it starts with "#include" of some headers ("directives not supported yet"), and there may be some functions that I'd like to exclude (e.g. with "complex float *" arguments, see item 1 above). What I envision might help here is some comment-based markup of the original header, that would allow a tool to parse the file and extract only the desired CDEF portions, into a new file that could be read in for the FFI.cdef() function. I'd rather not have to make changes to the .h file, and THEN make those same changes to the CDEF string / file. I'm thinking of writing such a tool, if it doesn't exist - but please let me know if I am missing something big that would preclude the need for it!

Thanks for your help,

  Tom Krauss

[Daniel Holth]

A very simple program that takes preprocessor output and keeps only certain files: https://bitbucket.org/dholth/pysdl2-cffi/src/tip/builder/filtercdefs.py

[Tom Krauss]

Nice! Thanks for sharing Daniel, this is a great start. The "gcc -E" output is pretty interesting (preprocessor output) - I haven't looked at this much. In your script, the "stdin" in the allowed_files pattern is matching stdint.h in my code. Assuming this wasn't your intention, how does "stdin" show up in the preprocessor filename output?

(I'm using "gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609" on Ubuntu 16.04, if that is relevant - I assume you are on Windows of some variant?)

[Armin Rigo]

Hi Tom,

On 1 January 2017 at 18:25, Tom Krauss <t...@kraussfamily.org> wrote:
> 1) Any advice for dealing with "complex float" or "complex double"?

Not really. It's unsupported yet.

I just noticed that complexes are supported by libffi now. I think
this was added in 2014; when cffi started in 2012, they were not
supported. That means that we could give it a try now :-)


A bientôt,

Armin.

[Tom Krauss]

Thanks for the response Armin.

What's involved in giving it a try? :-) I'd like to try to help add support for this.

[Daniel Holth]

I invoke it like so with input from stdin. The defines remove other
non-cffi-parseable things. You've found a bug, the regex should look
for the end of the string, but it works for me.
https://bitbucket.org/dholth/pysdl2-cffi/src/tip/cdefs.sh

cpp -D _SDL_endian_h -D SDL_FORCE_INLINE= -D DECLSPEC= -D SDLCALL=
-DDOXYGEN_SHOULD_IGNORE_THIS= -I /usr/include/SDL2 <
/usr/include/SDL2/SDL.h | ./builder/filtercdefs.py > _sdl/sdl.h

[Armin Rigo]

Hi Tom,

On 2 January 2017 at 16:29, Tom Krauss <t...@kraussfamily.org> wrote:
> What's involved in giving it a try? :-) I'd like to try to help add support
> for this.

Great :-) The best would be to join #pypy on irc.freenode.net for
more interactive discussion.

In two words, check out https://bitbucket.org/cffi/cffi and look at
how the "float/double/long double" types are implemented:
c/_cffi_backend.c, grep for PRIMITIVE_FLOAT. I guess a good first
step would be to support only "float _Complex " and "double _Complex",
with a new CT_PRIMITIVE_COMPLEX. There is already a skipped test in
c/test_c.py: test_complex_types(), that I wrote long ago (needs
checking, maybe some details changed; compare with test_float_types).
All in all it's not a whole lot of work, but we need to make sure
about the interaction between <cdata 'double _Complex'> and regular
Python complex objects.

The next step is the mapping to libffi types, which should occur in
new_primitive_type(). You need to add a test in test_c.py to check
calling functions with complex arguments or return types, something
similar to the various test_call_function_NUMBER.

Note that the C type name is e.g. "double _Complex", I think. This is
similar to "_Bool", which is the primitive name, and which usually is
typedef'ed to "bool" in C. This and other details need to be added to
the Python logic in the cffi/ package. Grep around for "_Bool" to see
an example. Maybe grepping around for "long double" is also a good
idea: "long double" is a type that needs a few special cases here and
there.


A bientôt,

Armin.