Compiling Several pyx's in one "setup.py" file?
Shalom Rav
I am using one "setup.py" file to compile one cython "pyx" file:
__________________________________________________________
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
import numpy
setup(
cmdclass = {'build_ext': build_ext},
include_dirs = [numpy.get_include()],
ext_modules = [Extension("file1", ["file1.pyx"])]
)
___________________________________________________________
Is it possible to use one "setup.py" file to compile multiple "pyx"
files? if so, how? thanks in advance.
Konstantin Khlyzov
Hello,
you can do it by appending more items ti ext_modules:
---------------------------------------------------
setup(
cmdclass = {'build_ext': build_ext},
include_dirs = [numpy.get_include()],
ext_modules = [Extension("file1", ["file1.pyx"]),
Extension("file2", ["file2.pyx"]), Extension("file3", ["file3.pyx"])]
)
---------------------------------------------------
so you'll have three compiled files file1.so (or file1.pyd on Win),
file2.so, file3.so
But — I wonder how one can compile _several_ .pyx-files into _one_
.so-file? It's easy to develop a package through several modules, but
finally I'd like to have just one extension file. Is it possible?
Does anyone know?
Keith Goodman
<kostik...@gmail.com> wrote:
> But — I wonder how one can compile _several_ .pyx-files into _one_
> .so-file? It's easy to develop a package through several modules, but
> finally I'd like to have just one extension file. Is it possible?
I think you just make a pyx file that contains:
include "file1.pyx"
include "file2.pyx"
include "file3.pyx"
That file will compile into one .so file.
Robert Bradshaw
Of course this will all be a single module, as if you copy-pasted each
of file1, file2, file3 literally into a larger file, and may or may
not be what you want. Why do you need a single .so file?
- Robert
Shalom Rav
This is great -- thank you.
On Dec 14, 12:41 pm, Keith Goodman <kwgood...@gmail.com> wrote:
> On Tue, Dec 14, 2010 at 8:55 AM, Konstantin Khlyzov
>
Konstantin Khlyzov
> On Tue, Dec 14, 2010 at 9:41 AM, Keith Goodman <kwgo...@gmail.com> wrote:
>> On Tue, Dec 14, 2010 at 8:55 AM, Konstantin Khlyzov
>> <kostik...@gmail.com> wrote:
>>
>>> But — I wonder how one can compile _several_ .pyx-files into _one_
>>> .so-file? It's easy to develop a package through several modules, but
>>> finally I'd like to have just one extension file. Is it possible?
>>
>> I think you just make a pyx file that contains:
>>
>> include "file1.pyx"
>> include "file2.pyx"
>> include "file3.pyx"
>>
>> That file will compile into one .so file.
That's good! I only found TODO section in docs about this, right here:
http://docs.cython.org/src/userguide/source_files_and_compilation.html#multiple-cython-files-in-a-package.
> Of course this will all be a single module, as if you copy-pasted each
> of file1, file2, file3 literally into a larger file, and may or may
> not be what you want. Why do you need a single .so file?
>
In my case, I just split code (for easier development) between modules
like constants.py, helpers.py, utils.py, classes.py and so on, then
making from them all import *. And finally I want a single library
file, without need to import or cimport constants and helpers each
time. And, being included, those constants and helpers can be cdef'd
(which means faster performance, right?) — I don't need them in the
Python scope.
Probably, there is a couple of things I should be aware of doing so,
but I know nothing about it. Well, it's just my second week with
Cython :-)
Thanks in advance, if you can cast some light upon it.
Regards,
Konstantin.
Robert Bradshaw
<kostik...@gmail.com> wrote:
> 2010/12/15 Robert Bradshaw <robe...@math.washington.edu>:
>> On Tue, Dec 14, 2010 at 9:41 AM, Keith Goodman <kwgo...@gmail.com> wrote:
>>> On Tue, Dec 14, 2010 at 8:55 AM, Konstantin Khlyzov
>>> <kostik...@gmail.com> wrote:
>>>
>>>> But — I wonder how one can compile _several_ .pyx-files into _one_
>>>> .so-file? It's easy to develop a package through several modules, but
>>>> finally I'd like to have just one extension file. Is it possible?
>>>
>>> I think you just make a pyx file that contains:
>>>
>>> include "file1.pyx"
>>> include "file2.pyx"
>>> include "file3.pyx"
>>>
>>> That file will compile into one .so file.
>
> That's good! I only found TODO section in docs about this, right here:
> http://docs.cython.org/src/userguide/source_files_and_compilation.html#multiple-cython-files-in-a-package.
>
>> Of course this will all be a single module, as if you copy-pasted each
>> of file1, file2, file3 literally into a larger file, and may or may
>> not be what you want. Why do you need a single .so file?
>>
>
> In my case, I just split code (for easier development) between modules
> like constants.py, helpers.py, utils.py, classes.py and so on, then
> making from them all import *. And finally I want a single library
> file, without need to import or cimport constants and helpers each
> time.
If it's just about keeping the files you're editing small, and you
don't care about compilation time (or your project is small enough)
then that should work fine.
> And, being included, those constants and helpers can be cdef'd
> (which means faster performance, right?) — I don't need them in the
> Python scope.
That's what the cimport keyword is for.
> Probably, there is a couple of things I should be aware of doing so,
> but I know nothing about it. Well, it's just my second week with
> Cython :-)
>
> Thanks in advance, if you can cast some light upon it.
I guess I would say that one file = one module is an organizational
pattern from Python that it typically used with Cython as well, but if
it doesn't fit you needs then including everything into a single file
works as well. Note, however, that if you include a file more than
once (either in the same module, or into two different modules) you
literally get two copies of all the contents (e.g. two functions, two
distinct classes, etc.) That's something to watch out for if you ever
feel the need to go to more than a single module.
- Robert
Konstantin Khlyzov
> On Tue, Dec 14, 2010 at 11:17 AM, Konstantin Khlyzov
>> (which means faster performance, right?) — I don't need them in the
>> Python scope.
>
> That's what the cimport keyword is for.
Yes, I read about it, but when I made .pxd-file for a .pyx-file, I got
an error " 'pos_konst.pxd' not found " at the line "from pos_konst
cimport * ", though the file actually had been created in the same
directory and looked as that:
---------------------------------------------------------
# -*- coding: utf--*-
cdef char la, ne, o, a # and so on
---------------------------------------------------------
and all of the cdef'd vars were assigned something in pos_konst.pyx.
Why would that be wrong? Should I have cdef them again in .pyx-file?
(I didn't).
Stefan Behnel
> On Tue, Dec 14, 2010 at 8:55 AM, Konstantin Khlyzov wrote:
>
>> But — I wonder how one can compile _several_ .pyx-files into _one_
>> .so-file? It's easy to develop a package through several modules, but
>> finally I'd like to have just one extension file. Is it possible?
>
> I think you just make a pyx file that contains:
>
> include "file1.pyx"
> include "file2.pyx"
> include "file3.pyx"
>
> That file will compile into one .so file.
The commonly used extension for included files is ".pxi", not ".pyx", to
distinguish them from complete modules.
Stefan
Robert Bradshaw
Actually, one current defect of .pxd files is that the can only
declare functions, classes, and extern declarations, not variables
(yet), so it wouldn't work for all your constant definitions. You also
have to include '.' in include_dirs, see
http://wiki.cython.org/PackageHierarchy
- Robert