Multiple Cython files in a package

[Stefan]

Hi folks,

I'm trying to make a package with a number of Cython files. The documentation is not the most helpful:
http://docs.cython.org/src/userguide/source_files_and_compilation.html#multiple-cython-files-in-a-package

Here's my sample project.
src/
    setup.py
    fish/
        compiled.pyx
        __init__.py

#compiled.pyx
def testfun(data):
    return data + 5

#__init__.py
__all__ = ["compiled"]

#setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext

INCLUDES = ['.', './fish']

ext_modules=[
    Extension("fish.compiled",
             sources = ["fish/compiled.pyx"],
             include_dirs=INCLUDES
             ),
]

setup(
   name = "fish",
   version = "0.0.1",
   cmdclass = {"build_ext": build_ext},
   ext_modules = ext_modules,
)



Now I do, from the shell:
sage setup.py install

This puts a file "compiled.so" into the site-packages/fish/ directory.

Then, from the Sage prompt:

sage: from fish.compiled import testfun

which results in an

ImportError: No module named compiled

Finally, if I replace "fish.compiled" by "compiled" in setup.py, then "from compiled import testfun" works just fine...

What am I missing?

Cheers,

Stefan.

[Robert Bradshaw]

On Thu, May 19, 2011 at 7:53 AM, Stefan <stefan...@gmail.com> wrote:
> Hi folks,
>
> I'm trying to make a package with a number of Cython files. The
> documentation is not the most helpful:
> http://docs.cython.org/src/userguide/source_files_and_compilation.html#multiple-cython-files-in-a-package
>
> Here's my sample project.
> src/
>     setup.py
>     fish/
>         compiled.pyx
>         __init__.py
>
> #compiled.pyx
> def testfun(data):
>     return data + 5
>
> #__init__.py
> __all__ = ["compiled"]
>
> #setup.py
> from distutils.core import setup
> from distutils.extension import Extension
> from Cython.Distutils import build_ext
>
> INCLUDES = ['.', './fish']

Just have ['.']

> ext_modules=[
>     Extension("fish.compiled",
>              sources = ["fish/compiled.pyx"],
>              include_dirs=INCLUDES
>              ),
> ]
>
> setup(
>    name = "fish",
>    version = "0.0.1",
>    cmdclass = {"build_ext": build_ext},
>    ext_modules = ext_modules,
> )
>
>
>
> Now I do, from the shell:
> sage setup.py install
>
> This puts a file "compiled.so" into the site-packages/fish/ directory.
>
> Then, from the Sage prompt:
>
> sage: from fish.compiled import testfun
>
> which results in an
>
> ImportError: No module named compiled
>
> Finally, if I replace "fish.compiled" by "compiled" in setup.py, then "from
> compiled import testfun" works just fine...
>
> What am I missing?

What directory are you running this from?

- Robert

[Stefan]

It's ~/Documents/coding/python/src if you must know. Sage lives in ~/sage-4.6.2 (so the compiled.so file lands in ~/sage-4.6.2/local/lib/python2.6/site-packages/fish/ using the commands as above). After changing setup.py as detailed, the "compiled.so" file lands in ~/sage-4.6.2/local/lib/python2.6/site-packages/ as expected. I can access it from the latter location, but not from the former.

Cheers,

Stefan.

[Stefan]

I have found the source of my problem (and the solution). After installing the package, I started Python immediately, from the directory containing setup.py. This caused the "import" statement to search through the subdirectory of my source, rather than the subdirectory of site-packages where the compiled.so file went. It didn't find said file, and gave an error.

Solution: start Python (or Sage) from a different directory, and "from fish.compiled import testfun" works perfectly.

Thanks for listening and thinking along!

Stefan.