Cython Noob having trouble getting off the ground

[Brad Hoehne]

Hi,

I've been writing python scripts for about a year or so and have a few things that I'd like to "optimize" (physics simulations, highly repetitive game logic etc,.) with Cython.  However, I can't get it to work.

I've googled around and found out about how to install Cython and use it to compile python and cython "typed" code. After watching a number of tutorials extolling its virtues and proclaiming how easy it is, I've taken the plunge.

Here's the details:

- I'm on a Mac, latest OS 15.3.2
- Using VSCode to write/debug code
- I've Homebrew installed C with homebrew (GCC)
- Version of python: 3.13.2

- I've "pip3 installed" Cython - I've installed the Cython extension in VS code - I've created a setup.py file which goes in the same folder as my project files (both .py and .pyx) Here's what that setup.py looks like:

from setuptools import setup
from Cython.Build import cythonize
import time

setup(
name = "L_app",
ext_modules=cythonize("license.pyx"),
annotate=True
)

I have two frustrating problems:

FIRST PROBLEM:

As a learning project, I'm writing file that searches through 3 letter combinations in all english words looking for matches. I have a version that works well in Python. It's a game that I call the "license plate game". I'd like to optimize it so that it works as fast as possible and I can run through all possibilities much more quickly. As a test, I've decided to "cythonize" a raw python.pyx file for the main "loop" of the code, which searches through all 3-letter combinations looking for matches. That file is called license.pyx.

When I run:

>>> python3 setup.py build_ext --inplace

... nothing happens!

Seriously, no time passes. I just get the prompt back and no files appear.

In the terminal I'm in the directory in which setup.py is in.

>>> cythonize -a setup.py

...seems to work just fine. The files appear. I get a longwinded file called:

>>> checkplate.cpython-313-darwin.so
So I can, it seems, compile the file. (Incidentally, the C file is weirdly long, 3000+ lines!). I'd prefer to go the setup.py route, as it would be nice to put all the compiling instructions in a single, easy to edit, place rather than putting it all on the command line.  But if I have to use "cythonize", I can deal with it.

This leads to the SECOND PROBLEM:

(which I figure is related to the fact that I'm a noob and am missing some key step)

I'm trying to import the cythonized file into my license.py game like this:

import cython
from checkplate.cpython-313-darwin.so import check_plate

(check_plate is the name of the function in the module that I'm making, I'm starting with just pure python, but I've also tried a cpdef tag with typing just to see if that helped)

and... intellisense puts little squiggly lines below the names of cython and python. I cannot run or compile the program.

when I pip3 list the modules I have installed, the latest version of cython (3.1.0)appears on that list.

The checkplate.cpython-313-darwin.so file is in the same directory as licence.py

For the life of me, I can't figure out what I'm doing wrong or what step I've missed. I've googled everything I can think of and there's always this crucial last step missing in whatever tutorials I encounter (or they just assume that everything will work hunky-dory)

Any thoughts?

Thanks, B.

[David Woods]

3000 line C file is about right.

> import cython
> from checkplate.cpython-313-darwin.so import check_plate

You don't need `import cython`.
You should just do `from checkplate import check_plate`. Don't include the version tag or the .so - this is handled magically

[Prakhar Goel]

A few thoughts:

The generated C files contain a lot of boilerplate, etc... ignore them unless you're very deep into the weeds. Certainly, 3k lines of generated C is not that unusual.

You don't need to import Cython to use compiled modules. The modules are self-contained.

You only need the package name in the import statement. I.e. import checkplate.

Paths, etc... need to match up. I.e. the compiled module needs to be somewhere Python can find it and the files for the cythonize setup.py file also need to be discoverable.

As it is, your example clearly has a bunch of things slapped together (e.g. license.pyx vs checkplate?). You'll have better luck if you're u can point to the actual code you're using so we have something concrete to help you with.

-- PG

--

---
You received this message because you are subscribed to the Google Groups "cython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cython-users...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/cython-users/8e633d2b-647f-4ce4-96e7-dbb3574e522an%40googlegroups.com.

[Keith Wipf]

You don't import checkplate.cpython-313-darwin.so, you import checkplate. Python automatically appends the platform-specific and .so bits for you.

Also I'm not sure but in your setup.py, the argument to ext_modules should possibly be a list even if there's only one thing in it, also try that with the argument to cythonize.

--

[Brad Hoehne]

Hi Gang,

Thank you for reaching out.   So, here's what I'm doing (not my ultimate goal, but part of my learning process for cython.  This is to preempt any "why are you doing > this< in cython" queries.  I have some stuff that I would actually like to optimize once I figure out how to get it to work).   I'm taking this tiny program, which wrote in about 15 minute, and wanted to use cython to speed it up:

#licence plate game. Find if three letters exist in any english word in order but not necessarily consecutively

import time

# from checkplate import check_plate

def main():

count = 0

wordlist = []

plate = input("What are the 3 letters of the license plate? ")

start_time = time.perf_counter() # start timer

with open("/Users/bradhoehne_1/documents/python_learn/words_alpha.txt") as wordlist:

for wrd in wordlist:

finding = check_plate(wrd, plate)

if finding != None:

count = count + 1

print(f"{finding.strip()}")

print ("\nCount:", count, "words")

end_time = time.perf_counter() #end timer

print (f"Total Time: {end_time - start_time:0.6f} seconds")

#end main

def check_plate(word, lets):

original_word = word

if lets[0] in word:

index = word.find(lets[0])

word = word[index+1:]

if lets[1] in word:

index = word.find(lets[1])

word = word[index+1:]

if lets[2] in word:

return original_word

else:

return

else:

return

else:

return

if __name__ == "__main__":

main()

So to do so, I've taken the "check_plate" def out of this module and reimported it like this:

#licence plate game. Find if three letters exist in any english word in order but not necessarily consecutively

import time

from checkplate import check_plate

def main():

count = 0

wordlist = []

plate = input("What are the 3 letters of the license plate? ")

start_time = time.perf_counter() # start timer

with open("/Users/bradhoehne_1/documents/python_learn/words_alpha.txt") as wordlist:

for wrd in wordlist:

finding = check_plate(wrd, plate)

if finding != None:

count = count + 1

print(f"{finding.strip()}")

print ("\nCount:", count, "words")

end_time = time.perf_counter() #end timer

print (f"Total Time: {end_time - start_time:0.6f} seconds")

#end main

Cythonizing the checkplate.pyx file:

VS Code still doesn't recognize the file as a module.  Note the squiggly line...

Just to confirm that I have everything that is needed:

If I run the license_game.py game as seen above:



I'm stumped...

Thank you for your help so far

Brad

[da-woods]

Only answering a tiny bit of this, but: I wouldn't get too worried by what vscode recognises as a module. I think it tends to ignore extension modules because it's hard to introspect them without executing them which it treats as a security risk. So worry when it fails to import, not just because of the red underlines.

To view this discussion visit https://groups.google.com/d/msgid/cython-users/951a785b-7ae7-468e-86c4-0113f1687707n%40googlegroups.com.

[Prakhar Goel]

Recommended making LicenseGame a proper module by adding a __init__.py file in there and then move checkplate.pyx in there as well. After that, you should be able to import it as from .checkplate import check_plate. For debugging, also include the output of:

import sys

print(sys.path)

-- PG

To view this discussion visit https://groups.google.com/d/msgid/cython-users/951a785b-7ae7-468e-86c4-0113f1687707n%40googlegroups.com.

[Brad Hoehne]

Hi, 

Here's the result of print(sys.path):

>>> import sys
>>> print(sys.path)
['', '/Library/Frameworks/Python.framework/Versions/3.13/lib/python313.zip', '/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13', '/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/lib-dynload', '/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages']

I created a blank __init__.py file and placed it in the same directory as checkplate.pyx and license_game.py.  I still get:

AttributeError: module 'checkplate' has no attribute 'check_plate'

I'm sort of a "module" noob, but have written a lot of python code, so there maybe something that I'm misunderstanding.  As I understand it __init__.py just has to exist, not have anything in it.  Is this correct?

Brad

[Peter Schay]

Hi!

Here is a great talk on modules, packages, and imports by Python guru David Beazley: https://youtu.be/0oTh1CXRaQ0?t=1443

Also he has a section on modules and packages in his awesome Python course: https://www.oreilly.com/library/view/python-programming-language/9780134217314/

[Prakhar Goel]

Did you move checkplate into the same directory as the rest of your code? What's the output of print(checkplate)?

An empty __init__.py file should be sufficient assuming it's in the right place.

In general, you're giving partial information which makes this whole process rather inefficient. At the very least, include the complete directory layout with the commands you ran and the output you got. A git repo would be ideal.

-- PG

To view this discussion visit https://groups.google.com/d/msgid/cython-users/dede0b85-2ed2-48ee-adc3-762d8a8809ffn%40googlegroups.com.