cimport numpy: Parent module '' not loaded, cannot perform relative import

[Julien Delafontaine]

Hello,
I have a cython extension "myextension.pyx" with only 2 lines it it:

    import numpy as np
    cimport numpy as np

It compiles and builds without error, but when I "import myextension" from python I get this :

    cimport numpy as np
    E   SystemError: Parent module '' not loaded, cannot perform relative import

My setup.py says:
    ext_modules=[
        Extension("a/b/myextension", ["a/b/myextension.c"],
                  include_dirs=[numpy.get_include()]),

    ],

and I build with these commands:

    cython a/b/myextension.pyx
    python3 setup.py build_ext --inplace

I am working with Python3 on OSX, with the latest numpy (1.10.2) and cython 0.23.4.

What can I try to solve this problem ?

[Oscar Benjamin]

On 5 January 2016 at 09:59, Julien Delafontaine <mura...@gmail.com> wrote:
> It compiles and builds without error, but when I "import myextension" from
> python I get this :
>
> cimport numpy as np
> E SystemError: Parent module '' not loaded, cannot perform relative
> import
>
> My setup.py says:
> ext_modules=[
> Extension("a/b/myextension", ["a/b/myextension.c"],
> include_dirs=[numpy.get_include()]),
> ],
>
> and I build with these commands:
>
> cython a/b/myextension.pyx
> python3 setup.py build_ext --inplace
>
> I am working with Python3 on OSX, with the latest numpy (1.10.2) and cython
> 0.23.4.
>
> What can I try to solve this problem ?

Are the folders a and b packages? Shouldn't you be importing it as
a.b.myextension rather than myextension?

--
Oscar

[Julien Delafontaine]

a and b have an __init__.py.
But when building with --inplace, it creates the .so at where we run the command (at the root of my project, where setup.py is). And a print statement on top of the .pyx gets printed before the "cimport numpy" throws.

[Oscar Benjamin]

Are you sure? It should build it right next to the .pyx file. Try
deleting all .so and .c files to confirm this.

BTW the normal way to do this is with cythonize:

http://docs.cython.org/src/reference/compilation.html#compiling-with-distutils

Then you don't need the two steps.

--
Oscar

[Julien Delafontaine]

Certain. I just deleted the .so, ran setup.py build_ext --inplace again, and the .so reappeared next to setup.py. If I don't add '--inplace', then the .so is in somewhere in /build.

It looked like on the latest OSX the cythonize() version ignores "include_dirs=[numpy.get_include()]". Maybe this is fixed - or I was wrong.

[Julien Delafontaine]

Update: cythonize() now works and builds the .so in the same folder as the .pyx. Then importing a.b.myextension works. Thanks !

[Julien Delafontaine]

No... Here we are. When running build_ext,

    a/b/myextension.c:250:10: fatal error: 'numpy/arrayobject.h' file not found

So numpy.get_include() was ignored. In setup.py:

    ext_modules = cythonize("a/b/myextension.pyx", include_path=[numpy.get_include()]),

Should I open an issue ?

[Julien Delafontaine]

I had to specify manually where the numpy headers are. Looks like an OSX issue:

    https://github.com/hmmlearn/hmmlearn/issues/43

[Julien Delafontaine]

Solution:

    export CFLAGS="-I $(python3 -c 'import numpy; print(numpy.get_include())') $CFLAGS"

[Oscar Benjamin]

On 5 January 2016 at 13:05, Julien Delafontaine <mura...@gmail.com> wrote:
> No... Here we are. When running build_ext,
>
> a/b/myextension.c:250:10: fatal error: 'numpy/arrayobject.h' file not
> found
>
> So numpy.get_include() was ignored. In setup.py:
>
> ext_modules = cythonize("a/b/myextension.pyx",
> include_path=[numpy.get_include()]),
>
> Should I open an issue ?

Are you using include_path or include_dirs? I think it should be
include_dirs for C-level .h headers and include_path for .pxd files
that you cimport. Maybe I'm getting confused as well though...

--
Oscar

[Julien Delafontaine]

It is include_path in the docs, with cythonize(). But I also put include_dirs in setup(), just in case. Cython sees it and does something like

   all_imports = include_path + [default_imports]

(if include_path is a string for instance, this line throws an error) but it gets lost afterwards.

[Oscar Benjamin]

On 5 January 2016 at 15:03, Julien Delafontaine <mura...@gmail.com> wrote:
> It is include_path in the docs, with cythonize(). But I also put
> include_dirs in setup(), just in case. Cython sees it and does something
> like
>
> all_imports = include_path + [default_imports]
>
> (if include_path is a string for instance, this line throws an error) but it
> gets lost afterwards.

Could you post a minimal reproducing example of a .pyx and setup.py
and the versions of cython, python and numpy that you're using?

--
Oscar

[Julien Delafontaine]

Sure. Note that on Linux I could make it work without specifying CFLAGS, so it really is an OSX(-python) issue.
I have to remove the .c from the distribution and recompile on that machine, though - I suppose it is expected.

apply_bitwise.pyx:
------------------------

import numpy as np
cimport numpy as np

setup.py:
------------

import os
from setuptools import setup, find_packages
from Cython.Build import cythonize
import numpy

setup(
    name='whatever',
    version='0.3.2',
    packages=find_packages(exclude=['doc*'']),
    ext_modules = cythonize("a/b/apply_bitwise.pyx", include_path=[numpy.get_include()]),
    include_dirs = [numpy.get_include()],
    include_package_data=True,
    license='GPL-2',
    description='asdf',
)

Versions:
-------------
OSX El Capitan (10.11.2)
Python 3.4.3 (installed with Homebrew I believe)
GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)
>>> cython.__version__
'0.23.4'
>>> numpy.__version__
'1.10.2'

[Oscar Benjamin]

On 5 January 2016 at 16:16, Julien Delafontaine <mura...@gmail.com> wrote:
> Sure. Note that on Linux I could make it work without specifying CFLAGS, so
> it really is an OSX(-python) issue.

Then I have no way of testing it. Sorry...

--
Oscar

[Chris Barker]

OK,

This setup.py works for me:

from setuptools import setup, Extension

from Cython.Build import cythonize
import numpy

setup(
    name='whatever',
    version='0.3.2',

    ext_modules=cythonize(Extension("apply_bitwise",
                                    ["apply_bitwise.pyx"],
                                    include_dirs=[numpy.get_include()],
                                    )),
)

cythonize is a great idea, but not really feature complete, or well documented. I've resorted to always making an Extension object, and passing that to cythonize.

(OS-X 10.9)

Interesting that it works on Linux -- different Cython version?

-Chris

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris....@noaa.gov

[Julien Delafontaine]

Same cython version on Linux (0.23.4).
Indeed it works with a full Extension definition instead of cythonize().