build failure when cython files are not "cythonized" beforehand.

[Chris Colbert]

Cython version 0.14.1-1, Python 2.7, OSX

I have the following setup.py;

setup.py

------------

from distutils.core import setup

from distutils.extension import Extension

from Cython.Distutils import build_ext

ext_modules = [Extension('messages', ['messages.pyx']),

               Extension('signals', ['signals.pyx']),

               Extension('dispatchers', ['dispatchers.pyx'])]

setup(cmdclass = {'build_ext': build_ext},

      ext_modules = ext_modules)

If I execute the following shell commands, everything builds fine:

new-host:c_event_sys chris$ rm -rf build *.c *.so

new-host:c_event_sys chris$ cython signals.pyx

new-host:c_event_sys chris$ cython dispatchers.pyx

new-host:c_event_sys chris$ cython messages.pyx

new-host:c_event_sys chris$ python setup.py build_ext -i

<yaddy yadda output. finished fine>

However, if I don't pre-cythonize the files, I get this error:

new-host:c_event_sys chris$ rm -rf build *.c *.so

new-host:c_event_sys chris$ python setup.py build_ext -i

running build_ext

cythoning messages.pyx to messages.c

building 'messages' extension

creating build

creating build/temp.macosx-10.5-i386-2.7

gcc -fno-strict-aliasing -fno-common -dynamic -arch i386 -isysroot /Developer/SDKs/MacOSX10.5.sdk -DNDEBUG -g -O3 -arch i386 -isysroot /Developer/SDKs/MacOSX10.5.sdk -I/Library/Frameworks/Python.framework/Versions/7.0/include -I/Library/Frameworks/Python.framework/Versions/7.0/include/python2.7 -c messages.c -o build/temp.macosx-10.5-i386-2.7/messages.o

gcc -g -L/usr/local/lib -L/Library/Frameworks/Python.framework/Versions/7.0/lib -bundle -undefined dynamic_lookup -g -L/usr/local/lib -L/Library/Frameworks/Python.framework/Versions/7.0/lib -arch i386 build/temp.macosx-10.5-i386-2.7/messages.o -o /Users/chris/Development/traits4/c_event_sys/messages.so

ld: warning: directory '/usr/local/lib' following -L not found

ld: warning: directory '/usr/local/lib' following -L not found

cythoning signals.pyx to signals.c

Error compiling Cython file:

------------------------------------------------------------

...

# pxd imports

from cpython.weakref cimport PyWeakref_NewRef, PyWeakref_GET_OBJECT

from messages cimport Message

^

------------------------------------------------------------

signals.pyx:3:0: 'messages.pxd' not found

Error compiling Cython file:

------------------------------------------------------------

...

# pxd imports

from cpython.weakref cimport PyWeakref_NewRef, PyWeakref_GET_OBJECT

from messages cimport Message

^

------------------------------------------------------------

signals.pyx:3:0: 'Message.pxd' not found

Error compiling Cython file:

------------------------------------------------------------

...

# pxd imports

from cpython.weakref cimport PyWeakref_NewRef, PyWeakref_GET_OBJECT

from messages cimport Message

                     ^

------------------------------------------------------------

signals.pyx:3:22: Name 'Message' not declared in module 'messages'

Error compiling Cython file:

------------------------------------------------------------

...

        

        remove_indices.reverse()

        for i in remove_indices:

            heap.pop(i)

    cpdef emit(self, Message message):

                    ^

------------------------------------------------------------

signals.pyx:41:21: 'Message' is not a type identifier

building 'signals' extension

gcc -fno-strict-aliasing -fno-common -dynamic -arch i386 -isysroot /Developer/SDKs/MacOSX10.5.sdk -DNDEBUG -g -O3 -arch i386 -isysroot /Developer/SDKs/MacOSX10.5.sdk -I/Library/Frameworks/Python.framework/Versions/7.0/include -I/Library/Frameworks/Python.framework/Versions/7.0/include/python2.7 -c signals.c -o build/temp.macosx-10.5-i386-2.7/signals.o

signals.c:1:2: error: #error Do not use this file, it is the result of a failed Cython compilation.

error: command 'gcc' failed with exit status 1

new-host:c_event_sys chris$ ls

__init__.py     messages.c      setup.py        test.py

build           messages.pxd    signals.c

dispatchers.pxd messages.pyx    signals.pxd

dispatchers.pyx messages.so     signals.pyx

new-host:c_event_sys chris$ 

Clearly, messages.pxd is in the local directory. Any ideas to why it's not being found?

[Chris Colbert]

gah, so even though pre cythonizing lets the modules build properly, I get errors on import:

Traceback (most recent call last):

  File "test.py", line 1, in <module>

    from signals import Signal, KillSignalException

  File "messages.pxd", line 3, in init traits4.c_event_sys.signals (signals.c:2083)

    cdef class Message:

ImportError: No module named traits4.c_event_sys.messages

[Chris Colbert]

code repository is here:

https://github.com/enthought/traits4/tree/master/c_event_sys

It's not apparent if i'm doing anything wrong in the build-process, and there shouldn't be any circular imports going on.

Hopeful someone has some insight. 

[Chris Colbert]

ok, removing the __init__.py from the directory solves the problem in that things build and run fine. But why would having an __init__.py effect a python setup.py build_ext -i command?

[Stefan Behnel]

Chris Colbert, 21.02.2011 02:28:

>>>> new-host:c_event_sys chris$ ls
>>>> __init__.py messages.c setup.py test.py
>>>> build messages.pxd signals.c
>>>> dispatchers.pxd messages.pyx signals.pxd
>>>> dispatchers.pyx messages.so signals.pyx
>>>> new-host:c_event_sys chris$
>>>>

>>>> *Clearly, messages.pxd is in the local directory. Any ideas to why it's
>>>> not being found?*

>
> ok, removing the __init__.py from the directory solves the problem in that
> things build and run fine. But why would having an __init__.py effect a
> python setup.py build_ext -i command?

Cython needs to know the package structure at compile time in order to find
corresponding .pxd files and to properly generate importing code at the C
level. If you put an __init__.py file into a directory, that makes it a
package.

Stefan