Cython with buildozer

[Hubert Soyer]

Hi everybody!

Buildozer is a great tool to get kivy running on Android because it manages all the nasty building and linking.

For parts of my program where performance is an issue, I would like to use cython to get a speed up.

So I wrote the pyx files the I need and on my desktop it's easy to get the program running cause I can do the cython compiling easily on my own.

However, now I want to deploy my app to Android and am not sure what to do with the pyx cython files.

Is there a way to handle them with Buildozer?

If not, can you hint me towards a point of entry in the docs or code so I can learn how to get pyx files working with Android?

Thank you in advance!

Best,

Hubert

[Hubert Soyer]

A small update:

I found out that Cython can automatically import and compile pyx files if I specify the line

import pyximport; pyximport.install()

as described here: http://docs.cython.org/src/userguide/source_files_and_compilation.html#pyximport

However, that just causes my app to crash on startup. Not sure if I am on the right track there.

[Hubert Soyer]

I think I have to create a custom recipe for python-for-android that compiles my cython module.

I am now trying to solve the same problem that stardt posted on StackOverflow:

http://stackoverflow.com/questions/21359510/custom-recipes-for-python-for-android

Buildozer will erase any custom template I make.

Is there a folder where I can put custom recipes so they don't get deleted but are still used by buildozer?

[Hubert Soyer]

I was finally able to figure out how I can leverage cython in my android kivy app.

Here some notes in case other people face the same problem in the future:

#!/bin/bash

VERSION_processimage=0.1

DEPS_processimage=(android)

MD5_processimage=

BUILD_processimage=/home/ogh/projects/Android/myapp

RECIPE_processimage=$RECIPES_PATH/processimage

function prebuild_processimage() {

true

}

function shouldbuild_processimage() {

if [ -d "$SITEPACKAGES_PATH/processimage" ]; then

DO_BUILD=0

fi

}

function build_processimage() {

cd $BUILD_processimage

push_arm

export LDFLAGS="$LDFLAGS -L$LIBS_PATH"

export LDSHARED="$LIBLINK"

# fake try to be able to cythonize generated files
        cython $BUILD_processimage/processimage.pyx

$HOSTPYTHON setup.py build_ext --inplace

try $HOSTPYTHON setup.py install -O2

unset LDSHARED

pop_arm

}

function postbuild_processimage() {

true

}

The important bit was to run cython in the recipe to create the .c file from my .pyx file.

Since the python distribution that will be deployed to android does not contain a cython module a .pyx file can not be processed during the deployment process.

The setup.py file that translates the .pyx file to .c looks like this:

from distutils.core import setup

from distutils.extension import Extension

setup(

    name='processimage',

    ext_modules = [Extension("processimage", ["processimage.c"], extra_compile_args=['-std=c99'] )]

)

It just takes the processimage.c file and makes it a python extension.

Make sure to put the recipe in a subfolder of the recipe folder in the ./.buildozer/android/platform/python-for-android/recipes/ subfolder of your app folder.

[Jacek Nowak]

Can you specify precisely how you did this? I understand that processimage.pyx is your cython libary. Where do you put this file? In the main app dir or in a processimage subdirectory, or in the site-packages dir for python? You wrote before that buildozer removes your recipe.sh file before it runs it - how did you overcome this? Did you put processimage in your buildozer.spec as a requirement for the app?

I'm trying to do a very similar thing and cannot get there.

[Hubert Soyer]

The processimage.pyx is located in the main directory of my app, but in principle you can put it anywhere, you just need to specify in the recipy where.

Look at the line: BUILD_processimage=/home/ogh/projects/Android/myapp

in the recipy. That is where buildozer looks for the cython file.

I didn't really resolve the "buildozer removes recipe" problem. Once you run buildozer, it will create a .buildozer directory in your app's main directory where it stores a cached version of all your dependencies. I simply added the recipy there and added and commited it with git locally.

Yes, I did put processimage in my buildozer.spec.

[Jacek Nowak]

Thanks a lot, I made it. The key tip was "added and commited it with git locally" - I did not do this. Adding the recipe.sh to local git also preserves it and buildozer no longer removes the recipe before processing (or removes it and restores after that - I'm not sure).

[Bill Janssen]

Hubert, I'm glad you got this working.

However, this seems like maybe something that should happen automatically?  That is, if your project contains .pyx files, they should be processed by Cython when buildozer runs the app.  You shouldn't have to write a recipe for your application.  That's what the buildozer.spec file is supposed to be.

Bill

[David Roundy]

I agree. Bulldozer would be so much more useful if we could use cython in our own projects without having to hack Buildozer itself. Is there any chance of this happening?

[Brent Picasso]

Hello,

I found this thread very helpful, thank you for the information.

I am able to create a simple cython extension in the root folder of the project. However, when I place it in a subfolder, the extension cannot be found under android. When I access it from desktop (linux) using pyximport() - it does work correctly - I get an error in the android console about cannot load module named 'testit'. Again, if the module is in the root directory, it does work correctly.

Any help / thoughts would be very welcome. Thank you!

Here's my recipe and setup script:

recipe:

#!/bin/bash

VERSION_racecaptureapp=0.1

DEPS_racecaptureapp=(kivy android)

MD5_racecaptureapp=

BUILD_racecaptureapp=/home/brent/git-projects/RaceCapture_App

RECIPE_racecaptureapp=$RECIPES_PATH/racecaptureapp

function prebuild_racecaptureapp() {

    true

}

function shouldbuild_racecaptureapp() {

if [ -d "$SITEPACKAGES_PATH/racecaptureapp" ]; then

DO_BUILD=0

fi

}

function build_racecaptureapp() {

    echo "build racecapture app"

cd $BUILD_racecaptureapp

push_arm

export LDFLAGS="$LDFLAGS -L$LIBS_PATH"

export LDSHARED="$LIBLINK"

export PYTHONPATH=$SITEPACKAGES_PATH:$BUILDLIB_PATH

# fake try to be able to cythonize generated files

$HOSTPYTHON setup.py build_ext

try find . -iname '*.pyx' -exec $CYTHON {} \;

try $HOSTPYTHON setup.py build_ext -v

try find build/lib.* -name "*.o" -exec $STRIP {} \;

    try $HOSTPYTHON setup.py install -O2 --root=$BUILD_PATH/python-install --install-lib=lib/python2.7/site-packages

unset LDSHARED

pop_arm

}

function postbuild_racecaptureapp() {

true

}

setup script

from distutils.core import setup

from distutils.extension import Extension

setup(

    name='racecaptureapp',

    ext_modules = [Extension("blah.testit", ["blah/testit.c"], 

                             extra_compile_args=['-std=c99']

                              )]

)

[Brent Picasso]

code for extension (testit.pyx):

cdef class Foooo(object):

    cdef int tick

    def __init__(self, **kwargs):

        self.tick = kwargs.get('tick', 0)

        print('tick: ' + str(self.tick))

In main.py:

    from blah.testit import Foooo

    def build(self):

        x = Foooo(tick=1234)

....

--
You received this message because you are subscribed to a topic in the Google Groups "Kivy users support" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kivy-users/2dnyeFDWNTw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kivy-users+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Brent Picasso

Autosport Labs

Technology for Race and Street

[Brent Picasso]

Figured it out, mostly. Even though buildozer was correctly packaging my specific .so into the main libpymodules.so bundle, my main project still had the same package structure in place. At runtime, this package was found first, the .so wasn't found there, and python gave up without looking in libpymodules.so.

To get myself moving forward, I'm cheating and building the .so in place and allowing the .so extension in the buildozer spec file.

I see now how if my extensions were outside of the main project the library would be found correctly. However, I'd imagine it could be desirable to have cython optimizations right inside of the app.

Is there a way for python to search libpymodules.so first, or is there something else I can configure in my recipe or buildozer configuration?

-Brent

To unsubscribe from this group and all its topics, send an email to kivy-users+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[K leo]

Hi all,

After 5 year, I wonder what the status of Cython with buildozer is at the moment. 

-Brent

To unsubscribe from this group and all its topics, send an email to kivy-...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.