Unable to Cythonise .pyx file

[Damian Vuleta]

I apologise in advance if this is an old topic. I'm new to Cython and am probably overlooking something basic, but I haven't been able to find help elsewhere.

I have a short Python module I wish to Cythonise:

# Define Jumble
def Jumble(TestList[], SequenceX):
    TLen = len(TestList)
    J = 0
    for I in range(TLen):
        J = (J + int(SequenceX[I:I+6])) % TLen
        TestList[I], TestList[J] = TestList[J], TestList[I]
    return TestList

# Define De-Jumble
def DeJumble(TestList, SequenceX):
    TLen = len(TestList)
    J = 0
    for I in range(TLen):
        J = (J + int(SequenceX[I:I+6])) % TLen
    for I in reversed(range(TLen)):
        TestList[I], TestList[J] = TestList[J], TestList[I]
        J = (J - int(SequenceX[I:I+6])) % TLen
    return TestList

(TestList is a list of integers and SequenceX is a text string of numbers.)

I create a basic setup.py file:

from distutils.core import setup
from Cython.Build import cythonize

setup(
    ext_modules = cythonize("JumDe.pyx")
)

and run python3 setup.py build_ext --inplace from Terminal. It produces a .so file (albeit with a lot of warning messages) and I can successfully import it and use it in my main program, with a noticeable speed increase.

When I rename the module with a .pyx suffix, change the setup file and run it again, up comes this error message:

Traceback (most recent call last):
  File "setup.py", line 5, in <module>
    ext_modules = cythonize("JumDe.pyx")
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/Cython/Build/Dependencies.py", line 754, in cythonize
    aliases=aliases)
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/Cython/Build/Dependencies.py", line 649, in create_extension_list
    for file in nonempty(extended_iglob(filepattern), "'%s' doesn't match any files" % filepattern):
  File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/Cython/Build/Dependencies.py", line 103, in nonempty
    raise ValueError(error_msg)
ValueError: 'JumDe.pyx' doesn't match any files

This happens whether I put C type declarations in the module or not. So what am I overlooking?

Using Python 3.4.3, Cython 0.22.1, Mac OS X 10.10.4

[Stefan Behnel]

Damian Vuleta schrieb am 28.07.2015 um 08:12:
> I apologise in advance if this is an old topic. I'm new to Cython and am
> probably overlooking something basic, but I haven't been able to find help
> elsewhere.
>
> I have a short Python module I wish to Cythonise:
>
> # Define Jumble

> def Jumble(TestList[], SequenceX): ...
> # setup.py

> setup(
> ext_modules = cythonize("JumDe.pyx")
> )
>
> and run python3 setup.py build_ext --inplace from Terminal. It produces a
> .so file (albeit with a lot of warning messages) and I can successfully
> import it and use it in my main program, with a noticeable speed increase.
>
> When I rename the module with a .pyx suffix, change the setup file and run
> it again, up comes this error message:
>
> Traceback (most recent call last):
> File "setup.py", line 5, in <module>
> ext_modules = cythonize("JumDe.pyx")

> [...]

> raise ValueError(error_msg)
> ValueError: 'JumDe.pyx' doesn't match any files

It cannot find the file "JumDe.pyx" that you told it to compile. What is
the name of the file that you want to compile?

Stefan

[Toni Barth]

Am 28.07.2015 um 08:12 schrieb Damian Vuleta:

from distutils.core import setup
from Cython.Build import cythonize

setup(
    ext_modules = cythonize("JumDe.pyx")
)

Hello,

as far as I know, the correct call to Cythonize would be the following:
setup(
    ext_modules = cythonize("*",["JumDe.pyx"])
)

The first argument should mention the package name, the second one the related pyx files. But I never compiled something without a package around it, so it can be different for simple modules.

Best Regards.

Toni

[Stefan Behnel]

'Toni Barth' via cython-users schrieb am 28.07.2015 um 08:18:
> Am 28.07.2015 um 08:12 schrieb Damian Vuleta:
>>
>> from distutils.core import setup
>> from Cython.Build import cythonize
>>
>> setup(
>> ext_modules = cythonize("JumDe.pyx")
>> )
>>
> Hello,
>
> as far as I know, the correct call to Cythonize would be the following:
> setup(
> ext_modules = cythonize("*",["JumDe.pyx"])
> )
>
> The first argument should mention the package name, the second one the
> related pyx files.

No, that's wrong. The first argument is the list of modules to compile, and
can be a simple glob pattern string for convenience, as in the two examples
above (although "*" isn't the ideal pattern here...). The second argument
is the list of file patterns to *exclude*.

It's usually best to use keyword arguments for anything but the first
argument of cythonize(). Otherwise it's just too unclear what happens.

Stefan

[Toni Barth]

Am 28.07.2015 um 08:26 schrieb Stefan Behnel:
> 'Toni Barth' via cython-users schrieb am 28.07.2015 um 08:18:
>> Am 28.07.2015 um 08:12 schrieb Damian Vuleta:
>>> from distutils.core import setup
>>> from Cython.Build import cythonize
>>>
>>> setup(
>>> ext_modules = cythonize("JumDe.pyx")
>>> )
>>>
>> Hello,
>>
>> as far as I know, the correct call to Cythonize would be the following:
>> setup(
>> ext_modules = cythonize("*",["JumDe.pyx"])
>> )
>>
>> The first argument should mention the package name, the second one the
>> related pyx files.
> No, that's wrong. The first argument is the list of modules to compile, and
> can be a simple glob pattern string for convenience, as in the two examples
> above (although "*" isn't the ideal pattern here...).

Oh yeah, sorry, mixed that up with the Extension class constructor.
Thanks for clarifying.

[Damian Vuleta]

Sorry, mistake there: my first setup script read

from distutils.core import setup
from Cython.Build import cythonize
setup(

    ext_modules = cythonize("JumDe.py")
)

as the original file (which compiled successfully) was JumDe.py. It was after I changed the file's name to JumDe.pyx and edited the setup script that the process failed.

[Damian Vuleta]

I found the problem. I'd saved the file as .pyx in IDLE. I tried using Cython directly from Terminal and it came up with an error message saying the file was called JumDe.pyx.py, even though it showed as JumDe.pyx in the folder. Using 'Get Info', I was able to change the file's name and Cython can now recognise it.

My next question (yes, I'm full of them, or full of something): I know that I could produce a better result by adding type declarations in the .pyx file. Variables I, J and TLen are clearly long, but how would I classify a list like TestList and a string of characters like SequenceX?

[Chris Barker]

On Tue, Jul 28, 2015 at 7:02 PM, Damian Vuleta <dami...@gmail.com> wrote:

I found the problem. I'd saved the file as .pyx in IDLE. I tried using Cython directly from Terminal and it came up with an error message saying the file was called JumDe.pyx.py, even though it showed as JumDe.pyx in the folder.

yeah, Windows hiding extensions is evil, truly evil -- you can turn that off -- you really should. But glad you figured it out.

 

My next question (yes, I'm full of them, or full of something): I know that I could produce a better result by adding type declarations in the .pyx file. Variables I, J and TLen are clearly long, but how would I classify a list like TestList and a string of characters like SequenceX?

I'm pretty sure you can typedef a variable as a list in cyton, but it won't buy you much.

string of characters is a bit tricky -- do you need unicode support? and do you need both py2 and py3?

I _think_ -- I"ll leave it as an excricse on for the reader to workout the details, you want to use:

bytes if it's a py2 stringa, nd only one-byte-per charactor charactors

unicode if it's a py3 string and/ or needs to support unicode.

Cython shoudl give you some boost with these.

 This should help with more info:

http://docs.cython.org/src/tutorial/strings.html

-Chris

 

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris....@noaa.gov

[Damian Vuleta]

Thanks for your help, everyone. Cython is starting to make sense to me.