Declaring and defining function pointers

[Mikael Djurfeldt]

Dear all,

I'm a cython novice who doesn't have all cython concepts clear
yet---please bear with me.

I have a tricky module dependency problem where I want to call a C
function foo () in module B from module A. However, I cannot load
module B until earliest just before the call to foo ().

One solution to the problem would be to do some home-made linking by
defining a function pointer foo_ptr in module A and let module B fill
in this pointer with foo once it's loaded. I was, thus, wondering if
such a pointer could be declared from within cython. I tried first to
declare the pointer type with:

ctypedef (foo*) (int x)

This leads to the backtrace included below.

How should I go about defining my pointer-field in module A and how do
I fill it in with foo in module B?

Best regards,
Mikael Djurfeldt

Backtrace:

Traceback (most recent call last):
File "/usr/bin/cython", line 8, in <module>
main(command_line = 1)
File "/usr/lib/pymodules/python2.5/Cython/Compiler/Main.py", line 733, in main
result = compile(sources, options)
File "/usr/lib/pymodules/python2.5/Cython/Compiler/Main.py", line
710, in compile
return compile_multiple(source, options)
File "/usr/lib/pymodules/python2.5/Cython/Compiler/Main.py", line
680, in compile_multiple
result = run_pipeline(source, options)
File "/usr/lib/pymodules/python2.5/Cython/Compiler/Main.py", line
542, in run_pipeline
err, enddata = context.run_pipeline(pipeline, source)
File "/usr/lib/pymodules/python2.5/Cython/Compiler/Main.py", line
197, in run_pipeline
data = phase(data)
File "/usr/lib/pymodules/python2.5/Cython/Compiler/Main.py", line
500, in parse
tree = context.parse(source_desc, scope, pxd = 0, full_module_name
= full_module_name)
File "/usr/lib/pymodules/python2.5/Cython/Compiler/Main.py", line
432, in parse
tree = Parsing.p_module(s, pxd, full_module_name)
File "Parsing.py", line 2508, in Cython.Compiler.Parsing.p_module
(/build/buildd-cython_0.11.2-1-i386-RNSlk8/cython-0.11.2/Cython/Compiler/Parsing.c:47394)
File "Parsing.py", line 2520, in Cython.Compiler.Parsing.p_module
(/build/buildd-cython_0.11.2-1-i386-RNSlk8/cython-0.11.2/Cython/Compiler/Parsing.c:47189)
File "Parsing.py", line 1560, in
Cython.Compiler.Parsing.p_statement_list
(/build/buildd-cython_0.11.2-1-i386-RNSlk8/cython-0.11.2/Cython/Compiler/Parsing.c:29947)
File "Parsing.py", line 1565, in
Cython.Compiler.Parsing.p_statement_list
(/build/buildd-cython_0.11.2-1-i386-RNSlk8/cython-0.11.2/Cython/Compiler/Parsing.c:29773)
File "Parsing.py", line 1480, in Cython.Compiler.Parsing.p_statement
(/build/buildd-cython_0.11.2-1-i386-RNSlk8/cython-0.11.2/Cython/Compiler/Parsing.c:29647)
File "Parsing.py", line 1488, in Cython.Compiler.Parsing.p_statement
(/build/buildd-cython_0.11.2-1-i386-RNSlk8/cython-0.11.2/Cython/Compiler/Parsing.c:28278)
File "Parsing.py", line 2288, in
Cython.Compiler.Parsing.p_ctypedef_statement
(/build/buildd-cython_0.11.2-1-i386-RNSlk8/cython-0.11.2/Cython/Compiler/Parsing.c:42659)
AttributeError: 'CComplexBaseTypeNode' object has no attribute 'name'

[Stefan Behnel]

Hi,

Mikael Djurfeldt wrote:
> I have a tricky module dependency problem where I want to call a C
> function foo () in module B from module A. However, I cannot load
> module B until earliest just before the call to foo ().
>
> One solution to the problem would be to do some home-made linking by
> defining a function pointer foo_ptr in module A and let module B fill
> in this pointer with foo once it's loaded. I was, thus, wondering if
> such a pointer could be declared from within cython. I tried first to
> declare the pointer type with:
>
> ctypedef (foo*) (int x)
>
> This leads to the backtrace included below.

> [...]

> Cython.Compiler.Parsing.p_ctypedef_statement
(/build/buildd-cython_0.11.2-1-i386-RNSlk8/cython-0.11.2/Cython/Compiler/Parsing.c:42659)
> AttributeError: 'CComplexBaseTypeNode' object has no attribute 'name'

Looks like a bug to me.

> How should I go about defining my pointer-field in module A and how do
> I fill it in with foo in module B?

Cython has support for providing and calling public C-API functions
(declared with the "api" modifier). The question is if you can use this
mechanism if you require to delay the import. IIRC, external cimports are
executed early during module initialisation.

Could you explain why you cannot do the import of module B when loading
module A?

Stefan

[Mikael Djurfeldt]

On Tue, Oct 13, 2009 at 4:47 PM, Stefan Behnel <stef...@behnel.de> wrote:
>> Cython.Compiler.Parsing.p_ctypedef_statement
> (/build/buildd-cython_0.11.2-1-i386-RNSlk8/cython-0.11.2/Cython/Compiler/Parsing.c:42659)
>> AttributeError: 'CComplexBaseTypeNode' object has no attribute 'name'
>
> Looks like a bug to me.

Right. Even if my ctypedef is wrong, I shouldn't get a backtrace.
BTW, is my declaration correct?

>> How should I go about defining my pointer-field in module A and how do
>> I fill it in with foo in module B?
>
> Cython has support for providing and calling public C-API functions
> (declared with the "api" modifier). The question is if you can use this
> mechanism if you require to delay the import. IIRC, external cimports are
> executed early during module initialisation.

The idea to fill in a pointer variable afterwards should work if I
only can sort out how to declare things. A worst case would be to
link with an external library providing providing the pointer
variable. :) In my case, the overhead due to indirection is no
problem.

> Could you explain why you cannot do the import of module B when loading
> module A?

Module A is a Python binding for MUSIC. It contains a wrapper, Setup,
for the C++ level constructor, cxx_Setup. A typical usage case would
be to import A and then call Setup (...), which calls new_cxx_Setup,
which calls MPI_Init. When using the MUSIC binding together with
mpi4py, mpi4py has to be loaded after A, otherwise mpi4py will call
MPI_Init at module initialization. Everything is fine so far. Now,
the class Setup contains a method communicator () which is supposed to
return an mpi4py Intracomm object (Python level) constructed from the
C++-level object returned from cxx_Setup.communicator (). I know how
to make the Intracomm object through a factory function (as you have
previously suggested to me) but the only way for this factory function
to be able to construct the object is that it has cimported mpi4py's
MPI.pxd which loads mpi4py. My solution was to put the factory
function in module B (mentioned above) and load this module in
Setup.communicator (). Obviously, this does not work either, because
the factory function takes a C++-level argument and needs to be
cimported. It is at this point that I'm thinking that I could define
a C level function pointer in module A which is initialized to NULL.
Setup.communicator () can be compiled against this pointer and
dynamically load module B which fills it in...

[Mikael Djurfeldt]

On Tue, Oct 13, 2009 at 4:21 PM, Mikael Djurfeldt <mik...@djurfeldt.com> wrote:
> ctypedef (foo*) (int x)

That is a typo. I of course mean:

ctypedef (*foo) (int x)

but that leads to the same bug.

[sturlamolden]

But you did not declare a return value. You cannot expect Cython to
accept invalid C declarations.

Either:

cdef void (*foo)(int)

Or:

ctypedef void (*foo_t)(int)
cdef foo_t foo


Sturla Molden