Can cffi support macro constant of char and const char*?

[mt]

Currently cffi only supports macro constant as ... or integer literal.

Can it be extended to char and string literals?

Like

#define X '<'

#define Y "<<"

[Armin Rigo]

Hi,

Would you like

#define Y ...

to work if Y happens to be a string literal in the actual headers?
That's the hard question here. I'm unsure how to implement that---we
would need some C code that is able to work if the expression Y
expands to an integer *or* to a string literal.

If you don't want this, and instead only want `#define Y "<<"` to
work, then it's not hard to implement, apart from a small mess because
we need to parse the string constant following the C rules. But then,
(1) it doesn't give a lot of benefits because you can always just
write `Y = "<<"` as regular Python code, and (2) it's strange that
this would work but `#define Y ...` wouldn't.


Armin Rigo

[mt]

Thanks!

I need to process some existing headers with constants defined as string literals. I can write corresponding Python string constant instead.

By the way, I have read your previous opposition of switching c parser from pycparser to libclang.

What do you think of adding a new libclang-based c parser that has compatible API with current pycparser-based parser (instead of replacing it)? libclang can be an optional dependency and does not influence existing code.

I skimmed through the code and thought the internally exposed API of parser is not too difficult to mock.

[Armin Rigo]

Hi,

On Thu, 6 Apr 2023 at 11:08, mt <firstl...@gmail.com> wrote:
> By the way, I have read your previous opposition of switching c parser from pycparser to libclang.
> What do you think of adding a new libclang-based c parser that has compatible API with current pycparser-based parser (instead of replacing it)? libclang can be an optional dependency and does not influence existing code.

Why do you want that?


Armin

[mt]

Because the partial support for real-world C header files of pycparser makes generating C bindings autonomously impossible.

1. Julia has Clang.jl and Rust has bindgen. Clang-based ffi generator has been proved to be robust and successful. Python should enjoy out-of-the-box C FFI generation as the only "officially" supported language binding of libclang.

2. libclang binding can be installed from PyPI directly accross multiple platforms. (https://github.com/sighingnow/libclang)

[Armin Rigo]

Hi,

On Thu, 6 Apr 2023 at 13:10, mt <firstl...@gmail.com> wrote:
> Because the partial support for real-world C header files of pycparser makes generating C bindings autonomously impossible.

You know that you cannot drop an existing, unmodified C header into
ffi.cdef() and be done with it, right? Of course you can't if the
headers use non-C99 extensions of the language, but it's also true if
they don't.

There are some 3rd-party projects that use CFFI by doing various kinds
of preprocessing on the real C header until it is suitable for their
use case to put in a cdef(). For the case you have in mind, maybe you
could parse the headers using libclang, extract the parts that make
sense, and emit sanitized C code again, which you can put in a cdef().
Within the scope of one project, it's possible to do this. But it's
not included with CFFI because the details are really dependent on the
project; I believe it would be hard to find a one-size-fits-all
solution. I prefer to keep this part into the hands of 3rd-party
projects, with maintainers closer to their particular problem than I
am.


Armin Rigo

[Armin Rigo]

Hi again,

Another answer would be that if you are looking for something like
Clang.jl or bindgen, then sure, but that's a bit different than CFFI's
core idea. I think a successful Python project that uses the same
idea could be done and be interesting---probably reusing parts of
CFFI's source and user API, too. I think doing it properly is a
different project than CFFI, though. You are welcome to give it a
try, and of course copy whatever parts of CFFI you want for it.

Maybe look at other projects first? E.g. cppyy?


Armin Rigo

[Paulo Meira]

Hi, all,

I'm not affiliated and have never used it, but I'd like to point that there is a related project that does something close to what mt's proposing (but doesn't seem to use that libclang package on PyPI itself):

https://github.com/aguinet/dragonffi

"DragonFFI is a C Foreign Function Interface (FFI) library written in C++ and based on Clang/LLVM. It allows any language to call C functions throught the provided APIs and bindings."

Besides maturity and platform support, an obvious downside is that it's quite heavy compared to CFFI.

Well, since I'm already posting this, let me share some personal experience:

I maintain (somewhat simple) bindings for a few languages (Python with CFFI, Julia, MATLAB, C#) and the situation in Julia is not all flowers either. Julia sometimes move too fast, the libclang API is unstable, and in the end it can be quite annoying to track which API changed and so on.

On libclang/LLVM in general, the lack of stability for certain APIs has been mentioned by others as an issue too throughout the years (to the point that some smaller projects die due to lack of updates).

Never had a single issue with CFFI though :)

Regards,

Paulo Meira

[Armin Rigo]

Hi Paulo,

On Thu, 6 Apr 2023 at 15:32, 'Paulo Meira' via python-cffi
<pytho...@googlegroups.com> wrote:
> On libclang/LLVM in general, the lack of stability for certain APIs has been mentioned by others as an issue too throughout the years (to the point that some smaller projects die due to lack of updates).

Thanks for the feedback. I don't have direct experience in the case
we're discussing. However, I am reluctant to promote clang/llvm
solutions based on (admittedly long past) experiences with the PyPy
project. Over the years, someone within PyPy (me in one case, other
people in other cases) tried to use clang or llvm to do various
things. It always ended up being killed for one reason or another.
In retrospect I'm glad we killed them, for various reasons including
stability over the years.


A bientôt,
Armin