Ways to compile .pyx to .c

[Nikolaus Rath]

Hello,

I am currently calling Cython.Compiler.Main.compile directly to compile
my .pyx files to .c:

from Cython.Compiler.Main import compile as cython_compile
from Cython.Compiler.Options import extra_warnings

directives = dict(extra_warnings)
directives['embedsignature'] = True
directives['language_level'] = 3
directives['warn.maybe_uninitialized'] = False

options = {'include_path': [ os.path.join(basedir, 'Include') ],
'recursive': False, 'verbose': True, 'timestamps': False,
'compiler_directives': directives, 'warning_errors': True,
'compile_time_env': {} }

for sysname in ('linux', 'freebsd', 'darwin'):
print('compiling capi.pyx to capi_%s.c...' % (sysname,))
options['compile_time_env']['TARGET_PLATFORM'] = sysname
options['output_file'] = os.path.join(basedir, 'src', 'llfuse',
'capi_%s.c' % (sysname,))
res = cython_compile(os.path.join(basedir, 'src', 'llfuse', 'capi.pyx'),
full_module_name='llfuse.capi', **options)
if res.num_errors != 0:
raise SystemExit('Cython encountered errors.')

However, a few versions ago this started to produce a warning about the
"warning_errors" option being unknown.

Looking at Cython.Compiler.Options (and the `cython --help` output),
this option still seems to exist, but I suppose something about the
internals changed. I tried to figure out how things worked from the
source, but I got rather confused by the way cython treats handles its
options.

So, a few questions:

* Is this API not intended to be used externally, i.e. should I switch
to using cythonize() or calling the "cython" program?

* What am I doing wrong when passing the 'warning_errors' option? Why
is this option treated differently than the others?

* It looks to me as if there is also a second-kind of
distutils/setuptools integration that does not require to explicitly
use cythonize(). How do these two methods relate to each other?

* When using cythonize(), how do I pass compiler options?

* When using the command line cython, is there a way to specify the
compile time environment?


Thanks,
-Nikolaus
--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

»Time flies like an arrow, fruit flies like a Banana.«

[Robert Bradshaw]

Correct. 

 

 * What am I doing wrong when passing the 'warning_errors' option? Why
   is this option treated differently than the others?

 * It looks to me as if there is also a second-kind of
   distutils/setuptools integration that does not require to explicitly
   use cythonize(). How do these two methods relate to each other?

 * When using cythonize(), how do I pass compiler options?

You should be able to pass them in as extra keyword arguments to cythonize()

 

 * When using the command line cython, is there a way to specify the
   compile time environment?


Thanks,
-Nikolaus

--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nikolaus Rath]

On Aug 08 2015, Robert Bradshaw <robe...@gmail.com> wrote:
>> * Is this API not intended to be used externally, i.e. should I switch
>> to using cythonize() or calling the "cython" program?
>>
>
> Correct.
>

>> * When using cythonize(), how do I pass compiler options?
>
> You should be able to pass them in as extra keyword arguments to
> cythonize()

Hmm. This work for some (e.g. compile_time_env), but not for others:

$ cat setup.py
#!/usr/bin/env python

from distutils.core import setup
from Cython.Build import cythonize

directives = {'embedsignature': True }
setup(
name = 'Test',
ext_modules = cythonize('mylib.pyx',
compile_time_env={ 'TARGET_PLATFORM': 'foo' },
directives = directives,
warning_errors=True,
warning_extra=True)
)

$ python3 setup.py build_ext --inplace
/usr/lib/python3/dist-packages/Cython/Compiler/Main.py:506: UserWarning: got unknown compilation options, please remove: warning_errors, warning_extra, directives
warnings.warn(message)
Compiling mylib.pyx because it changed.
Cythonizing mylib.pyx
[...]


Am I doing something wrong?


Best,

[Nikolaus Rath]

Hi Robert,

On Aug 08 2015, Robert Bradshaw <robe...@gmail.com> wrote:

>> * Is this API not intended to be used externally, i.e. should I switch
>> to using cythonize() or calling the "cython" program?
>>
>
> Correct.
>

>> * It looks to me as if there is also a second-kind of
>> distutils/setuptools integration that does not require to explicitly
>> use cythonize(). How do these two methods relate to each other?

Could you also comment on this question? I'm a confused because neither
the Cython nor the setuptools documentation say anything about this, yet
both provide part of the support. Is this a deprecated feature that
should not be used either?

I'm looking at e.g.
https://bitbucket.org/pypa/setuptools/src/tip/setuptools/command/build_ext.py
and https://github.com/cython/cython/blob/master/Cython/Distutils/build_ext.py.


Best,

[Nikolaus Rath]

On Aug 12 2015, Nikolaus Rath <Niko...@rath.org> wrote:
> On Aug 08 2015, Robert Bradshaw <robe...@gmail.com> wrote:
>>> * Is this API not intended to be used externally, i.e. should I switch
>>> to using cythonize() or calling the "cython" program?
>>>
>>
>> Correct.
>>
>>> * When using cythonize(), how do I pass compiler options?
>>
>> You should be able to pass them in as extra keyword arguments to
>> cythonize()
>
> Hmm. This work for some (e.g. compile_time_env), but not for others:

[...]

> ext_modules = cythonize('mylib.pyx',
> compile_time_env={ 'TARGET_PLATFORM': 'foo' },
> directives = directives,
> warning_errors=True,
> warning_extra=True)

[...]

> /usr/lib/python3/dist-packages/Cython/Compiler/Main.py:506: UserWarning: got unknown compilation options, please remove: warning_errors, warning_extra, directives
> warnings.warn(message)

There also seems to be a third category of options that get accepted
without warning, but then completely ignored. This happens for
output_file='something.c'.

[Nikolaus Rath]

On Aug 12 2015, Nikolaus Rath <Niko...@rath.org> wrote:
> On Aug 12 2015, Nikolaus Rath <Niko...@rath.org> wrote:
>> On Aug 08 2015, Robert Bradshaw <robe...@gmail.com> wrote:
>>>> * Is this API not intended to be used externally, i.e. should I switch
>>>> to using cythonize() or calling the "cython" program?
>>>>
>>>
>>> Correct.
>>>
>>>> * When using cythonize(), how do I pass compiler options?
>>>
>>> You should be able to pass them in as extra keyword arguments to
>>> cythonize()
>>
>> Hmm. This work for some (e.g. compile_time_env), but not for others:
> [...]
>> ext_modules = cythonize('mylib.pyx',
>> compile_time_env={ 'TARGET_PLATFORM': 'foo' },
>> directives = directives,
>> warning_errors=True,
>> warning_extra=True)
> [...]
>> /usr/lib/python3/dist-packages/Cython/Compiler/Main.py:506: UserWarning: got unknown compilation options, please remove: warning_errors, warning_extra, directives
>> warnings.warn(message)
>
> There also seems to be a third category of options that get accepted
> without warning, but then completely ignored. This happens for
> output_file='something.c'.

Really no one able to help?


Best,

[Stefan Behnel]

Nikolaus Rath schrieb am 12.08.2015 um 22:36:

> On Aug 08 2015, Robert Bradshaw wrote:
>>> * Is this API not intended to be used externally, i.e. should I switch
>>> to using cythonize() or calling the "cython" program?
>>>
>>
>> Correct.
>>
>>> * When using cythonize(), how do I pass compiler options?
>>
>> You should be able to pass them in as extra keyword arguments to
>> cythonize()
>
> Hmm. This work for some (e.g. compile_time_env), but not for others:
>
> $ cat setup.py
> #!/usr/bin/env python
>
> from distutils.core import setup
> from Cython.Build import cythonize
>
> directives = {'embedsignature': True }
> setup(
> name = 'Test',
> ext_modules = cythonize('mylib.pyx',
> compile_time_env={ 'TARGET_PLATFORM': 'foo' },

A "target platform" sounds like something I'd want to handle at C compile
time, but you may know better.

> directives = directives,
> warning_errors=True,
> warning_extra=True)
> )
>
> $ python3 setup.py build_ext --inplace
> /usr/lib/python3/dist-packages/Cython/Compiler/Main.py:506: UserWarning: got unknown compilation options, please remove: warning_errors, warning_extra, directives
> warnings.warn(message)

Ok, so "warning_errors" happens to be a compiler option that you need to
set directly as "Cython.Compiler.Options.warning_errors". "warning_extra"
doesn't exist at all. "directives" is really called "compiler_directives".

The current state of the compiler configuration in Cython is an
underdocumented mess. The global settings in Options.py were originally the
only way to configure Pyrex (typically from the command line). The compiler
directives (anything you can set from a source file) were added much later
but we never cleaned up those options that would better be represented as
directives. And the arguments to cythonize() are a big bunch of options,
parameters and directives some of which are not even spelled out but rather
passed on to other places that may or may not handle them. Until recently,
you didn't even get the warning above to see that things were not going the
expected way. We originally wanted to make that a hard error, but that
would have broken lots of Cython packages out there. With the warning, it's
at least possible to reach a sane state eventually.

So, the best place to learn something about Cython compiler configuration
is currently Options.py, I guess.

Stefan

[Daniele Nicolodi]

On 18/08/15 18:18, Stefan Behnel wrote:
> The current state of the compiler configuration in Cython is an
> underdocumented mess. The global settings in Options.py were originally the
> only way to configure Pyrex (typically from the command line). The compiler
> directives (anything you can set from a source file) were added much later
> but we never cleaned up those options that would better be represented as
> directives. And the arguments to cythonize() are a big bunch of options,
> parameters and directives some of which are not even spelled out but rather
> passed on to other places that may or may not handle them. Until recently,
> you didn't even get the warning above to see that things were not going the
> expected way. We originally wanted to make that a hard error, but that
> would have broken lots of Cython packages out there. With the warning, it's
> at least possible to reach a sane state eventually.

Hello Stephan,

what would be the preferred way to clean up this part of the code?

I would be happy to tackle the problem.

Cheers,
Daniele

[Stefan Behnel]

Daniele Nicolodi schrieb am 18.08.2015 um 19:05:
> On 18/08/15 18:18, Stefan Behnel wrote:
>> The current state of the compiler configuration in Cython is an
>> underdocumented mess. The global settings in Options.py were originally the
>> only way to configure Pyrex (typically from the command line). The compiler
>> directives (anything you can set from a source file) were added much later
>> but we never cleaned up those options that would better be represented as
>> directives. And the arguments to cythonize() are a big bunch of options,
>> parameters and directives some of which are not even spelled out but rather
>> passed on to other places that may or may not handle them. Until recently,
>> you didn't even get the warning above to see that things were not going the
>> expected way. We originally wanted to make that a hard error, but that
>> would have broken lots of Cython packages out there. With the warning, it's
>> at least possible to reach a sane state eventually.
>

> what would be the preferred way to clean up this part of the code?
>
> I would be happy to tackle the problem.

Thanks for asking.

First of all, there is some documentation for the compiler directives in

docs/src/reference/compilation.rst

It's incomplete and could use an update. The directives themselves are in
Cython/Compiler/Options.py -> "directive_defaults", with scope restrictions
(where can I use them?) defined in "directive_scopes". Few of them are
documented in the code, so you might not get very far with figuring out
what they do.

Next, the cythonize() function lives in Cython/Build/Dependencies.py. You
can see there that "**options" are being passed into CompilationOptions
where they magically end up in the object "__dict__". That makes it rather
unclear what happens (thus this ML thread). Making at least some of the
more important options visible in the cythonize() signature would help users.

And then there are the global names at the top of Options.py. I updated the
documentation for some of them recently, but in fact it would be better if
those that can be used at a per-module level were turned into module-scoped
compiler directives. Definitely the docstring related options and all
optimiser flags. They would be kept in Options for backwards compatibility,
but the new directives would then override them.

Converting an option into a directive means replacing all usages in the
compiler code (usually just the "Cython.Compiler" package) by references to
the directives (search for usages of some of their names to see how they
work), and then initialising the directive value to the Option value at
compilation start (because existing user code will set the Option before
running Cython).

Not all of the Options would go away, I guess, such as "embed" and the
"annotate" options, but getting rid of anything that users may want to set
at a per-module level rather than a per-python-runtime level would already
help a lot.

Any help with these tasks would be appreciated. If you have questions or
get stuck, please ask.

Stefan

[Nikolaus Rath]

On Aug 18 2015, Stefan Behnel <stef...@behnel.de> wrote:
>> $ cat setup.py
>> #!/usr/bin/env python
>>
>> from distutils.core import setup
>> from Cython.Build import cythonize
>>
>> directives = {'embedsignature': True }
>> setup(
>> name = 'Test',
>> ext_modules = cythonize('mylib.pyx',
>> compile_time_env={ 'TARGET_PLATFORM': 'foo' },
>
> A "target platform" sounds like something I'd want to handle at C compile
> time, but you may know better.

If there's a way to do that, I'd be interesting. I'm currently running
cython once for every target (FreeBSD, Linux, OS-X) to generate three
different .c output files. The "main" C file then contains

#ifdef __gnu_linux__
#include "capi_linux.c"
#elif __FreeBSD__
#include "capi_freebsd.c"
#elif __APPLE__ && __MACH__
#include "capi_darwin.c"
#else
#error "Unable to determine system (Linux/FreeBSD/Darwin)"
#endif

This works, but having just one C file would of course be nicer. The
(first) problem that I couldn't solve without Cython defines was a case
distinction in the .pxd files, e.g.:

cdef extern from "fuse_lowlevel.h" nogil:
[...]
IF TARGET_PLATFORM == 'darwin':
ctypedef void(*setxattr_fn_t)(fuse_req_t req, fuse_ino_t ino, const_char *name,
const_char *value, size_t size, int flags,
uint32_t position)
ctypedef void(*getxattr_fn_t)(fuse_req_t req, fuse_ino_t ino, const_char *name,
size_t size, uint32_t position)
ELSE:
ctypedef void(*setxattr_fn_t)(fuse_req_t req, fuse_ino_t ino, const_char *name,
const_char *value, size_t size, int flags)
ctypedef void(*getxattr_fn_t)(fuse_req_t req, fuse_ino_t ino, const_char *name,
size_t size)


Is there a way to handle this differently?

[Nikolaus Rath]

On Aug 18 2015, Stefan Behnel <stef...@behnel.de> wrote:

>>>> * When using cythonize(), how do I pass compiler options?
>>>
>>> You should be able to pass them in as extra keyword arguments to
>>> cythonize()
>>
>> Hmm. This work for some (e.g. compile_time_env), but not for others:
>>
>> $ cat setup.py
>> #!/usr/bin/env python
>>
>> from distutils.core import setup
>> from Cython.Build import cythonize
>>
>> directives = {'embedsignature': True }
>> setup(
>> name = 'Test',
>> ext_modules = cythonize('mylib.pyx',
>> compile_time_env={ 'TARGET_PLATFORM': 'foo' },

>> directives = directives,
>> warning_errors=True,
>> warning_extra=True)
>> )
>>
>> $ python3 setup.py build_ext --inplace
>> /usr/lib/python3/dist-packages/Cython/Compiler/Main.py:506: UserWarning: got unknown compilation options, please remove: warning_errors, warning_extra, directives
>> warnings.warn(message)
>
> Ok, so "warning_errors" happens to be a compiler option that you need to
> set directly as "Cython.Compiler.Options.warning_errors". "warning_extra"
> doesn't exist at all. "directives" is really called "compiler_directives".
>
> The current state of the compiler configuration in Cython is an
> underdocumented mess. The global settings in Options.py were originally the
> only way to configure Pyrex (typically from the command line). The compiler
> directives (anything you can set from a source file) were added much later
> but we never cleaned up those options that would better be represented as
> directives. And the arguments to cythonize() are a big bunch of options,
> parameters and directives some of which are not even spelled out but rather
> passed on to other places that may or may not handle them. Until recently,
> you didn't even get the warning above to see that things were not going the
> expected way. We originally wanted to make that a hard error, but that
> would have broken lots of Cython packages out there. With the warning, it's
> at least possible to reach a sane state eventually.
>
> So, the best place to learn something about Cython compiler configuration
> is currently Options.py, I guess.

Hmm. Is there anything wrong with setting *everything* in Options.py? It
seems it has everything that I want:

from Cython.Compiler import Options
from Cython.Build import cythonize

Options.directive_defaults.update(Options.extra_warnings)
Options.directive_defaults['embedsignature'] = True
Options.fast_fail = True
Options.warning_errors = True

setup(
name = 'Test',
ext_modules = cythonize('mylib.pyx',

compile_time_env={ 'TARGET_PLATFORM': 'foo'}))

[Nikolaus Rath]

On Aug 18 2015, Stefan Behnel <stef...@behnel.de> wrote:

Ok, so it seems I have the following options:

1. Call the "cython" program. Unfortunately this does not allow me to
specify the compile time environment (and tricks like putting the
definitions in a top-level .pyx file that includes the others doesn't
work, because the definitions are not propagated into cinclude'd .pxd
files).

2. Call the "cythonize" function in Python. This requires me to depend
on implementation details that may presumably be subject to change
from one Cython version to the other (set
Cython.Compiler.Options.warning_errors, and pass
..Options.warning_options as compiler directives). Furthermore, I
cannot specify the output file (so I have to add boilerplate code to
work with temporary files), and an apparently simple call like
cythonize('file.pyx') actually round-trips the file name through a
setuptools Extension object (which means that if setuptools isn't
Cython aware, cythonize() becomes a silent no-op).

3. Call "cython_compile". This is again an implementation detail that
may change, but it allows me to specify all the options as well as
the output file, and does not pass anything through setuptools.


Since (1) is a no-go because I can't specify the compile time
environment, and (2) and (3) in-practice both depend on undocumented
behavior, it seems to me I'm actually better off with (3).

[Stefan Behnel]

I would use a C header file that aliases them with a macro that discards
the position argument if not available. Then declare and use the macro in
your Cython code.

I would also #define global flags in that file for each platform (or
otherwise distinguishing factor) so that you can easily write conditional code.

Stefan

[Stefan Behnel]

If you can live with everything being global settings that impact all
compiled modules and usages of Cython in the current Python runtime, then
yes, that's ok. If not, then not.

Stefan

[Stefan Behnel]

Nikolaus Rath schrieb am 20.08.2015 um 04:33:
> Ok, so it seems I have the following options:
>
> 1. Call the "cython" program. Unfortunately this does not allow me to
> specify the compile time environment (and tricks like putting the
> definitions in a top-level .pyx file that includes the others doesn't
> work, because the definitions are not propagated into cinclude'd .pxd
> files).
>
> 2. Call the "cythonize" function in Python. This requires me to depend
> on implementation details that may presumably be subject to change
> from one Cython version to the other (set
> Cython.Compiler.Options.warning_errors, and pass
> ..Options.warning_options as compiler directives). Furthermore, I
> cannot specify the output file (so I have to add boilerplate code to
> work with temporary files), and an apparently simple call like
> cythonize('file.pyx') actually round-trips the file name through a
> setuptools Extension object (which means that if setuptools isn't
> Cython aware, cythonize() becomes a silent no-op).
>
> 3. Call "cython_compile". This is again an implementation detail that
> may change, but it allows me to specify all the options as well as
> the output file, and does not pass anything through setuptools.
>
>
> Since (1) is a no-go because I can't specify the compile time
> environment, and (2) and (3) in-practice both depend on undocumented
> behavior, it seems to me I'm actually better off with (3).

2) seems the least likely to suffer from unexpected incompatible changes in
Cython.

Regarding the output file name, I would rather try to drop the target
platform dependency in your code and resolve it all into one Cython module.

Stefan

[Nikolaus Rath]

I don't think that works (but admittedly you couldn't have known seeing
just this part). These are not functions that I'm calling, but the
prototypes for function pointers that I have to supply as callbacks to a
C function.

So, with some more context:

,----

| cdef extern from "fuse_lowlevel.h" nogil:

| IF TARGET_PLATFORM == 'darwin':
| ctypedef void(*setxattr_fn_t)(fuse_req_t req, fuse_ino_t ino, const_char *name,
| const_char *value, size_t size, int flags,

| ELSE:
| ctypedef void(*setxattr_fn_t)(fuse_req_t req, fuse_ino_t ino, const_char *name,
| const_char *value, size_t size, int flags)
|

| struct fuse_lowlevel_ops:
| void (*init) (void *userdata, fuse_conn_info *conn)
| setxattr_fn_t setxattr
|
|
| cdef void fuse_setattr (fuse_req_t req, fuse_ino_t ino, struct_stat *stat,
| int to_set, fuse_file_info *fi) with gil:
| # code
|
|
| cdef void fuse_setxattr_darwin (fuse_req_t req, fuse_ino_t ino, const_char *cname,
| const_char *cvalue, size_t size, int flags,
| uint32_t position) with gil:
| # code
|
|
| cdef void init_fuse_ops(fuse_lowlevel_ops* fuse_ops):
| '''Initialize fuse_lowlevel_ops structure'''
|
| string.memset(fuse_ops, 0, sizeof(fuse_lowlevel_ops))
| fuse_ops.init = fuse_init
| IF TARGET_PLATFORM == 'darwin':
| fuse_ops.setxattr = fuse_setxattr_darwin
| ELSE:
| fuse_ops.setxattr = fuse_setxattr
`----


Is there any solution for that?

[Robert Bradshaw]

In this case you can do

if TARGET_PLATFORM == 'darwin':
fuse_ops.setxattr = <void*>fuse_setxattr_darwin
else:
fuse_ops.setxattr = <void*>fuse_setxattr

[Nikolaus Rath]

Aeh, yes, but the question was if there is a way to get rid of the
TARGET_PLATFORM entirely and use the C preprocessor instead.

What would be the advantage of the above over what I'm doing now?

[Stefan Behnel]

The cast should better say <setxattr_fn_t> instead of <void*>. The C
compiler will always see the correct platform specific declaration in the
header file.

> Aeh, yes, but the question was if there is a way to get rid of the
> TARGET_PLATFORM entirely and use the C preprocessor instead.
>
> What would be the advantage of the above over what I'm doing now?

That you can take the decision at C compilation time (or even runtime, if
necessary) rather than having to take it at Cython translation time. I.e.
replace the "IF" around the assignment with a normal Cython "if".

Stefan

[Nikolaus Rath]

But then we're back to the original question of how to define
setxattr_fn_t in Cython (without using 'IF TARGET_PLATFORM).

>> Aeh, yes, but the question was if there is a way to get rid of the
>> TARGET_PLATFORM entirely and use the C preprocessor instead.
>>
>> What would be the advantage of the above over what I'm doing now?
>
> That you can take the decision at C compilation time (or even runtime,
> if necessary) rather than having to take it at Cython translation
> time. I.e. replace the "IF" around the assignment with a normal
> Cython "if".

Maybe I'm being dense, but if I replace "IF" with "if" then the case
distinction happens at runtime, not C compilation time. Arguably this is
better than at Cython compile time, but it still doesn't answer the
question of how to use the Preprocessor instead of Cython (which was the
start of this thread).

Please be patient with me one more time :-).

[Stefan Behnel]

If you don't call it yourself, it doesn't matter (to Cython). And if you
want to call it, define and call the macro instead (with all arguments,
discarding the last one inside of the macro if necessary).

>>> Aeh, yes, but the question was if there is a way to get rid of the
>>> TARGET_PLATFORM entirely and use the C preprocessor instead.
>>>
>>> What would be the advantage of the above over what I'm doing now?
>>
>> That you can take the decision at C compilation time (or even runtime,
>> if necessary) rather than having to take it at Cython translation
>> time. I.e. replace the "IF" around the assignment with a normal
>> Cython "if".
>
> Maybe I'm being dense, but if I replace "IF" with "if" then the case
> distinction happens at runtime, not C compilation time.

Except when the C compiler can evaluate it at compile time, e.g. because if
compares a value that you conditionally #define in an external header file
with a constant, e.g. some enum or #defined macro. And that's really all
you need to do in this case. You'd usually prefer integer or enum values
over string names, though.

Stefan

[Nikolaus Rath]

On Aug 31 2015, Stefan Behnel <stef...@behnel.de> wrote:
> [....]

Ok, so if I understand correctly, you're proposing something like this:

,----
| cdef extern from *:
| const_char* TARGET_PLATFORM

|
| cdef extern from "fuse_lowlevel.h" nogil:

| struct fuse_lowlevel_ops:
| void (*init) (void *userdata, fuse_conn_info *conn)
|

| # Not valid for Darwin
| void (*setxattr)(fuse_req_t req, fuse_ino_t ino, const_char *name,

| const_char *value, size_t size, int flags)
|
|

| cdef void fuse_setattr (fuse_req_t req, fuse_ino_t ino, struct_stat *stat,
| int to_set, fuse_file_info *fi) with gil:
| # code
|
|
| cdef void fuse_setxattr_darwin (fuse_req_t req, fuse_ino_t ino, const_char *cname,
| const_char *cvalue, size_t size, int flags,
| uint32_t position) with gil:
| # code
|
|
| cdef void init_fuse_ops(fuse_lowlevel_ops* fuse_ops):
| '''Initialize fuse_lowlevel_ops structure'''
|
| string.memset(fuse_ops, 0, sizeof(fuse_lowlevel_ops))
| fuse_ops.init = fuse_init

| if TARGET_PLATFORM == 'darwin':
| fuse_ops.setxattr = <void*> fuse_setxattr_darwin
| else:
| fuse_ops.setxattr = <void*> fuse_setxattr

`----

which I would then compile with -DTARGET_PLATFORM=darwin or
-DTARGET_PLATFORM=linux.

Did I understand you correctly?

This would have the advantage that I no longer have to use undocumented
APIs to set TARGET_PLATFORM in Cython, but it has the drawback that if I
make an error in the signature of fuse_setxattr_darwin, neither Cython
nor the C compiler would be able to detect it.

So is this really an improvement over what I'm doing at the moment?

I don't think I can cast to the actual type of fuse_ops.setxattr instead
of void*. Fuse_lowlevel.h doesn't export it as a separate
ctypedef, so I would need to do either:

| if TARGET_PLATFORM == 'darwin':
| fuse_ops.setxattr = <void (*setxattr)(fuse_req_t, fuse_ino_t, const_char*,
| const_char*, size_t, int)> fuse_setxattr_darwin
| # ...

..in which case the C compiler will give a type error under darwin (I'm
casting to the wrong type), or

| if TARGET_PLATFORM == 'darwin':
| fuse_ops.setxattr = <void (*setxattr)(fuse_req_t, fuse_ino_t, const_char*,
| const_char*, size_t, int, uint32_t)> fuse_setxattr_darwin
| # ...

..in which case Cython will give a type error.


Don't get me wrong, I actually agree that generating a single C file for
all platforms would be a much better solution. But looking at the
practical implications, they fall a bit short of my expectations. But
maybe (hopefully) I'm just still not doing it right.

[Nikolaus Rath]

I think I finally found the right solution, thanks for prompting me to
keep looking. The trick is to use an extra .c file:

,----[ macros.c ]---
| #ifdef __gnu_linux__
| # define ASSIGN_DARWIN(x,y)
| # define ASSIGN_NODARWIN(x,y) ((x) = (y))
| #elif #elif __APPLE__ && __MACH__
| # define ASSIGN_DARWIN(x,y) ((x) = (y))
| # define ASSIGN_NODARWIN(x,y)
| #else
| # error Unable to determine system
| #endif
`----


,----
| cdef extern from "macros.c" nogil:
| void ASSIGN_DARWIN(void*, void*)
| void ASSIGN_NODARWIN(void*, void*)

|
| cdef extern from "fuse_lowlevel.h" nogil:
| struct fuse_lowlevel_ops:
| void (*init) (void *userdata, fuse_conn_info *conn)
|
| # Not valid for Darwin
| void (*setxattr)(fuse_req_t req, fuse_ino_t ino, const_char *name,
| const_char *value, size_t size, int flags)
|
|
| cdef void fuse_setattr (fuse_req_t req, fuse_ino_t ino, struct_stat *stat,
| int to_set, fuse_file_info *fi) with gil:
| # code
|
|
| cdef void fuse_setxattr_darwin (fuse_req_t req, fuse_ino_t ino, const_char *cname,
| const_char *cvalue, size_t size, int flags,
| uint32_t position) with gil:
| # code
|
|
| cdef void init_fuse_ops(fuse_lowlevel_ops* fuse_ops):
| '''Initialize fuse_lowlevel_ops structure'''
|
| string.memset(fuse_ops, 0, sizeof(fuse_lowlevel_ops))
| fuse_ops.init = fuse_init

| ASSIGN_DARWIN(fuse_ops.setxattr, &fuse_setxattr_darwin)
| ASSIGN_NODARWIN(fuse_ops.setxattr, &fuse_setxattr)
`----


This still prevents type checking by Cython, but now type checking is
still done by the C compiler. Problem solved :-).