Buildozer(Conda Python3 Environment) + kivy, numpy, scipy, scikit-image, opencv, matplotlib

1,780 views
Skip to first unread message

OpenSourceEngineer

unread,
Apr 12, 2018, 1:59:54 AM4/12/18
to Kivy users support
Introduction
*************
I would like to make a computer vision android application based on python3. It seems to be possible to do that using buildozer but and cannot manage to make it work.
I can create APK for the basics hello world application available in kivy documentation(https://kivy.org/docs/examples/gen__application__app_with_build__py.html) with "requirements = python3crystax,kivy" but if I add numpy, scipy, scikit-image, opencv, matplotlib the process end with errors.

From what I understood each library listed in requirements should have its own recipe to be build correctly especially if the library is not 100% made in python. As numpy is a mix of C and python the recipe seems to be mandatory.

I saw that there are plenty of recipes already available in python-for-android (p4a) and my understanding is that buildozer is working with this list of recipes.


opencv, numpy recipes are here but their look like designed for old versions of opencv (version = '2.4.10.1') and numpy (depends = ['python2'])

Most of the recipes looks designed for python2 and I suppose this is why I cannot manage to build an APK.

Environment
****************
I am using the following conda environment:

# Name                    Version                   Build  Channel
bokeh                    
0.12.15                  py36_0  
bzip2                    
1.0.6                h9a117a8_4  
ca
-certificates           2018.03.07                    0  
cairo                    
1.14.12              h77bcde2_0  
certifi                  
2018.1.18                py36_0  
chardet                  
3.0.4                     <pip>
click                    
6.7              py36h5253387_0  
cloudpickle              
0.5.2                    py36_1  
colorama                  
0.3.9                     <pip>
cycler                    
0.10.0           py36h93f1223_0  
Cython                    0.25.2                    <pip>
cytoolz                  
0.9.0.1          py36h14c3975_0  
dask                      
0.17.2                   py36_0  
dask
-core                 0.17.2                   py36_0  
dbus                      
1.13.2               hc3f9b76_0  
decorator                
4.2.1                    py36_0  
distributed              
1.21.6                   py36_0  
docutils                  
0.14                      <pip>
expat                    
2.2.5                he0dffb1_0  
ffmpeg                    
3.4                  h7264315_0  
fontconfig                
2.12.6               h49f89f6_0  
freetype                  
2.8                  hab7d2ae_1  
glib                      
2.53.6               h5d9569c_2  
graphite2                
1.3.10               hf63cedd_1  
gst
-plugins-base          1.12.4               h33fb286_0  
gstreamer                
1.12.4               hb53b477_0  
harfbuzz                  
1.7.6                hc5b324e_0  
hdf5                      
1.10.1               h9caa474_1  
heapdict                  
1.0.0                    py36_2  
icu                      
58.2                 h9c2bf20_1  
idna                      
2.6                       <pip>
imageio                  
2.3.0                    py36_0  
intel
-openmp              2018.0.0                      8  
jasper                    
1.900.1              hd497a04_4  
jinja2                    
2.10             py36ha16c418_0  
jpeg                      
9b                   h024ee3a_2  
Kivy                      1.10.0                    <pip>
Kivy-Garden               0.1.4                     <pip>
kiwisolver                
1.0.1            py36h764f252_0  
libedit                  
3.1                  heed3624_0  
libffi                    
3.2.1                hd88cf55_4  
libgcc
-ng                 7.2.0                hdf63c60_3  
libgfortran
-ng            7.2.0                hdf63c60_3  
libopus                  
1.2.1                hb9ed12e_0  
libpng                    
1.6.34               hb9fc6fc_0  
libprotobuf              
3.5.2                h6f1eeef_0  
libstdcxx
-ng              7.2.0                hdf63c60_3  
libtiff                  
4.0.9                h28f6b97_0  
libvpx                    
1.6.1                h888fd40_0  
libxcb                    
1.13                 h1bed415_0  
libxml2                  
2.9.8                hf84eae3_0  
locket                    
0.2.0            py36h787c0ad_1  
markupsafe                
1.0              py36hd9260cd_1  
matplotlib                
2.2.2            py36h0e671d2_1  
mkl                      
2018.0.2                      1  
mkl_fft                  
1.0.1            py36h3010b51_0  
mkl_random                
1.0.1            py36h629b387_0  
msgpack
-python            0.5.6            py36h6bb024c_0  
ncurses                  
6.0                  h9df7e31_2  
networkx                  
2.1                      py36_0  
numpy                    
1.14.2           py36hdbf6ddf_1  
olefile                  
0.45.1                   py36_0  
opencv                    
3.3.1            py36h0a11808_0  
openssl                  
1.0.2o               h20670df_0  
packaging                
17.1                     py36_0  
pandas                    
0.22.0           py36hf484d3e_0  
partd                    
0.3.8            py36h36fd896_0  
pcre                      
8.41                 hc27e229_1  
pillow                    
5.0.0            py36h3deb7b8_0  
pip                      
9.0.3                    py36_0  
pixman                    
0.34.0               hceecf20_3  
plyer                    
1.3.0                     <pip>
psutil                    
5.4.3            py36h14c3975_0  
pygame                    
1.9.3                     <pip>
Pygments                  2.2.0                     <pip>
pyparsing                
2.2.0            py36hee85983_1  
pyqt                      
5.9.2            py36h751905a_0  
python                    
3.6.5                hc3d631a_0  
python
-dateutil           2.7.2                    py36_0  
pytz                      
2018.3                   py36_0  
pywavelets                
0.5.2            py36he602eb0_0  
pyyaml                    
3.12             py36hafb9ca4_1  
qt                        
5.9.4                h4e5bff0_0  
readline                  
7.0                  ha6073c6_4  
requests                  
2.18.4                    <pip>
scikit
-image              0.13.1           py36h14c3975_1  
scipy                    
1.0.1            py36hfc37229_0  
setuptools                
39.0.1                   py36_0  
sip                      
4.19.8           py36hf484d3e_0  
six                      
1.11.0           py36h372c433_1  
sortedcontainers          
1.5.9                    py36_0  
sqlite                    
3.22.0               h1bed415_0  
tblib                    
1.3.2            py36h34cf8b6_0  
tk                        
8.6.7                hc745277_3  
toolz                    
0.9.0                    py36_0  
tornado                  
5.0.1                    py36_1  
urllib3                  
1.22                      <pip>
virtualenv                
15.2.0                    <pip>
wheel                    
0.31.0                   py36_0  
xz                        
5.2.3                h55aa19d_2  
yaml                      
0.1.7                had09818_2  
zict                      
0.1.3            py36h3a3bf81_0  
zlib                      
1.2.11               ha838bed_2  

I installed buildozer following given "Installing Buildozer with python3 support". Surprisingly buildozer do not appear in "conda list" of my environment but it appears in the root conda environment. Maybe the "sudo pip install -e ." is not the good way to proceed with conda ? Anyway, buildozer is working.

Here is my builddozer.spec file:

[app]


# (str) Title of your application
title
= helloWord


# (str) Package name
package.name = helloWordApp


# (str) Package domain (needed for android/ios packaging)
package.domain = open.source.engineer


# (str) Source code where the main.py live
source
.dir = .


# (list) Source files to include (let empty to include all the files)
source
.include_exts = py,png,jpg,kv,atlas


# (list) List of inclusions using pattern matching
#source.include_patterns = assets/*,images/*.png


# (list) Source files to exclude (let empty to not exclude anything)
#source.exclude_exts = spec


# (list) List of directory to exclude (let empty to not exclude anything)
#source.exclude_dirs = tests, bin


# (list) List of exclusions using pattern matching
#source.exclude_patterns = license,images/*/*.jpg


# (str) Application versioning (method 1)
version
= 0.1


# (str) Application versioning (method 2)
# version.regex = __version__ = ['"](.*)['"]
# version.filename = %(source.dir)s/main.py


# (list) Application requirements
# comma seperated e.g. requirements = sqlite3,kivy
requirements
= python3crystax,kivy
# (str) Custom source folders for requirements
# Sets custom source for any requirements with recipes
# requirements.source.kivy = ../../kivy


# (list) Garden requirements
#garden_requirements =


# (str) Presplash of the application
#presplash.filename = %(source.dir)s/data/presplash.png


# (str) Icon of the application
#icon.filename = %(source.dir)s/data/icon.png


# (str) Supported orientation (one of landscape, portrait or all)
orientation
= portrait


# (list) List of service to declare
#services = NAME:ENTRYPOINT_TO_PY,NAME2:ENTRYPOINT2_TO_PY


#
# OSX Specific
#


#
# author = © Copyright Info


# change the major version of python used by the app
osx
.python_version = 3


# Kivy version to use
osx
.kivy_version = 1.9.1


#
# Android specific
#


# (bool) Indicate if the application should be fullscreen or not
fullscreen
= 0


# (string) Presplash background color (for new android toolchain)
# Supported formats are: #RRGGBB #AARRGGBB or one of the following names:
# red, blue, green, black, white, gray, cyan, magenta, yellow, lightgray,
# darkgray, grey, lightgrey, darkgrey, aqua, fuchsia, lime, maroon, navy,
# olive, purple, silver, teal.
#android.presplash_color = #FFFFFF


# (list) Permissions
#android.permissions = INTERNET


# (int) Android API to use
#android.api = 19


# (int) Minimum API required
#android.minapi = 9


# (int) Android SDK version to use
#android.sdk = 20


# (str) Android NDK version to use (Changed because: NDK version was set as r9c, but checking the NDK dir claims it is 10.3.2.)
#android.ndk = 10.3.2.


# (bool) Use --private data storage (True) or --dir public storage (False)
#android.private_storage = True


# (str) Android NDK directory (if empty, it will be automatically downloaded.)
#Requirement to use python 3
android
.ndk_path =~/logiciel/crystalx/crystax-ndk-10.3.2


# (str) Android SDK directory (if empty, it will be automatically downloaded.)
#android.sdk_path =


# (str) ANT directory (if empty, it will be automatically downloaded.)
#android.ant_path =


# (bool) If True, then skip trying to update the Android sdk
# This can be useful to avoid excess Internet downloads or save time
# when an update is due and you just want to test/build your package
# android.skip_update = False


# (str) Android entry point, default is ok for Kivy-based app
#android.entrypoint = org.renpy.android.PythonActivity


# (list) Pattern to whitelist for the whole project
#android.whitelist =


# (str) Path to a custom whitelist file
#android.whitelist_src =


# (str) Path to a custom blacklist file
#android.blacklist_src =


# (list) List of Java .jar files to add to the libs so that pyjnius can access
# their classes. Don't add jars that you do not need, since extra jars can slow
# down the build process. Allows wildcards matching, for example:
# OUYA-ODK/libs/*.jar
#android.add_jars = foo.jar,bar.jar,path/to/more/*.jar


# (list) List of Java files to add to the android project (can be java or a
# directory containing the files)
#android.add_src =


# (list) Android AAR archives to add (currently works only with sdl2_gradle
# bootstrap)
#android.add_aars =


# (list) Gradle dependencies to add (currently works only with sdl2_gradle
# bootstrap)
#android.gradle_dependencies =


# (list) Java classes to add as activities to the manifest.
#android.add_activites = com.example.ExampleActivity


# (str) python-for-android branch to use, defaults to stable
#p4a.branch = stable


# (str) OUYA Console category. Should be one of GAME or APP
# If you leave this blank, OUYA support will not be enabled
#android.ouya.category = GAME


# (str) Filename of OUYA Console icon. It must be a 732x412 png image.
#android.ouya.icon.filename = %(source.dir)s/data/ouya_icon.png


# (str) XML file to include as an intent filters in <activity> tag
#android.manifest.intent_filters =


# (str) launchMode to set for the main activity
#android.manifest.launch_mode = standard


# (list) Android additionnal libraries to copy into libs/armeabi
#android.add_libs_armeabi = libs/android/*.so
#android.add_libs_armeabi_v7a = libs/android-v7/*.so
#android.add_libs_x86 = libs/android-x86/*.so
#android.add_libs_mips = libs/android-mips/*.so


# (bool) Indicate whether the screen should stay on
# Don't forget to add the WAKE_LOCK permission if you set this to True
#android.wakelock = False


# (list) Android application meta-data to set (key=value format)
#android.meta_data =


# (list) Android library project to add (will be added in the
# project.properties automatically.)
#android.library_references =


# (str) Android logcat filters to use
#android.logcat_filters = *:S python:D


# (bool) Copy library instead of making a libpymodules.so
#android.copy_libs = 1


# (str) The Android arch to build for, choices: armeabi-v7a, arm64-v8a, x86
android
.arch = armeabi-v7a


#
# Python for android (p4a) specific
#


# (str) python-for-android git clone directory (if empty, it will be automatically cloned from github)
#p4a.source_dir =


# (str) The directory in which python-for-android should look for your own build recipes (if any)
#p4a.local_recipes =


# (str) Filename to the hook for p4a
#p4a.hook =


# (str) Bootstrap to use for android builds
# p4a.bootstrap = sdl2


# (int) port number to specify an explicit --port= p4a argument (eg for bootstrap flask)
#p4a.port =




#
# iOS specific
#


# (str) Path to a custom kivy-ios folder
#ios.kivy_ios_dir = ../kivy-ios


# (str) Name of the certificate to use for signing the debug version
# Get a list of available identities: buildozer ios list_identities
#ios.codesign.debug = "iPhone Developer: <lastname> <firstname> (<hexstring>)"


# (str) Name of the certificate to use for signing the release version
#ios.codesign.release = %(ios.codesign.debug)s




[buildozer]


# (int) Log level (0 = error only, 1 = info, 2 = debug (with command output))
log_level
= 2


# (int) Display warning if buildozer is run as root (0 = False, 1 = True)
warn_on_root
= 1


# (str) Path to build artifact storage, absolute or relative to spec file
# build_dir = ./.buildozer


# (str) Path to build output (i.e. .apk, .ipa) storage
# bin_dir = ./bin


#    -----------------------------------------------------------------------------
#    List as sections
#
#    You can define all the "list" as [section:key].
#    Each line will be considered as a option to the list.
#    Let's take [app] / source.exclude_patterns.
#    Instead of doing:
#
#[app]
#source.exclude_patterns = license,data/audio/*.wav,data/images/original/*
#
#    This can be translated into:
#
#[app:source.exclude_patterns]
#license
#data/audio/*.wav
#data/images/original/*
#




#    -----------------------------------------------------------------------------
#    Profiles
#
#    You can extend section / key with a profile
#    For example, you want to deploy a demo version of your application without
#    HD content. You could first change the title to add "(demo)" in the name
#    and extend the excluded directories to remove the HD content.
#
#[app@demo]
#title = My Application (demo)
#
#[app:source.exclude_patterns@demo]
#images/hd/*
#
#    Then, invoke the command line with the "demo" profile:
#
#buildozer --profile demo android debug

Buildozer without numpy requirements
********************************************
It works but the APK file is not successfully saved in the bin folder. I found it in the following folder:
/.buildozer/android/platform/build/dists/helloWordApp/bin

requirements in buildozer.spec:
requirements = python3crystax,kivy

Buildozer result:

[DEBUG]:    BUILD SUCCESSFUL
[DEBUG]:    Total time: 3 seconds
[INFO]:    <- directory context /media/username/data/racine/Loisir/IT/pythonProgram/androidApp/.buildozer/android/platform/python-for-android-new-toolchain
[INFO]:    # Copying APK to current directory
[INFO]:    # Found APK file: /media/username/data/racine/Loisir/IT/pythonProgram/androidApp/.buildozer/android/platform/build/dists/helloWordApp/bin/helloWord-0.1-debug.apk
[DEBUG]:   -> running cp /media/username/data/racine/Loisir/IT/pythonProgram/androidApp/.buildozer/android/platform/build/dists/helloWordApp/bin/helloWord-0.1-debug.apk ./
b
'Could not find hostpython, will not compile to .pyo (this is normal with python3)\nNo compiled python is present to zip, skipping.\nthis should only be the case if you are using the CrystaX python\n'Traceback (most recent call last):
 
File "/home/username/anaconda3/bin/buildozer", line 11, in <module>
    load_entry_point
('buildozer', 'console_scripts', 'buildozer')()
 
File "/home/username/logiciel/buildozer/buildozer/scripts/client.py", line 13, in main
   
Buildozer().run_command(sys.argv[1:])
 
File "/home/username/logiciel/buildozer/buildozer/__init__.py", line 1058, in run_command
   
self.target.run_commands(args)
 
File "/home/username/logiciel/buildozer/buildozer/target.py", line 92, in run_commands
    func
(args)
 
File "/home/username/logiciel/buildozer/buildozer/target.py", line 104, in cmd_debug
   
self.buildozer.build()
 
File "/home/username/logiciel/buildozer/buildozer/__init__.py", line 212, in build
   
self.target.build_package()
 
File "/home/username/logiciel/buildozer/buildozer/targets/android.py", line 828, in build_package
    copyfile
(join(apk_dir, apk), join(self.buildozer.bin_dir, apk_dest))
 
File "/home/username/anaconda3/lib/python3.6/shutil.py", line 120, in copyfile
   
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory: '/media/username/data/racine/Loisir/IT/pythonProgram/androidApp/.buildozer/android/platform/build/dists/helloWordApp/build/outputs/apk/helloWordApp-debug.apk'

Buildozer with numpy requirements
****************************************
With numpy in requirements I have the following error:

[INFO]:    No existing dists meet the given requirements!

Requirements in buildozer.spec:
requirements = python3crystax,kivy, numpy

Details:

----------

id: 202 or "extra-google-webdriver"

    Type: Extra

    Desc: Google Web Driver, revision 2

          By Google Inc.

          Install path: extras/google/webdriver


# Run (expect) '/home/username/.buildozer/android/platform/android-sdk-20/tools/android update sdk -u -a -t build-tools-28.0.0rc0'

# Cwd None

Picked up JAVA_TOOL_OPTIONS:  -Dfile.encoding=UTF-8

Picked up JAVA_TOOL_OPTIONS:  -Dfile.encoding=UTF-8

Picked up JAVA_TOOL_OPTIONS:  -Dfile.encoding=UTF-8

Refresh Sources:

 Fetching https://dl.google.com/android/repository/addons_list-2.xml

 Validate XML

 Parse XML

 Fetched Add-ons List successfully

 Refresh Sources

 Fetching URL: https://dl.google.com/android/repository/repository-11.xml

 Validate XML: https://dl.google.com/android/repository/repository-11.xml

 Parse XML:    https://dl.google.com/android/repository/repository-11.xml

 Fetching URL: https://dl.google.com/android/repository/addon.xml

 Validate XML: https://dl.google.com/android/repository/addon.xml

 Parse XML:    https://dl.google.com/android/repository/addon.xml

 Fetching URL: https://dl.google.com/android/repository/glass/addon.xml

 Validate XML: https://dl.google.com/android/repository/glass/addon.xml

 Parse XML:    https://dl.google.com/android/repository/glass/addon.xml

 Fetching URL: https://dl.google.com/android/repository/extras/intel/addon.xml

 Validate XML: https://dl.google.com/android/repository/extras/intel/addon.xml

 Parse XML:    https://dl.google.com/android/repository/extras/intel/addon.xml

 Fetching URL: https://dl.google.com/android/repository/sys-img/android/sys-img.xml

 Validate XML: https://dl.google.com/android/repository/sys-img/android/sys-img.xml

 Parse XML:    https://dl.google.com/android/repository/sys-img/android/sys-img.xml

 Fetching URL: https://dl.google.com/android/repository/sys-img/android-wear/sys-img.xml

 Validate XML: https://dl.google.com/android/repository/sys-img/android-wear/sys-img.xml

 Parse XML:    https://dl.google.com/android/repository/sys-img/android-wear/sys-img.xml

 Fetching URL: https://dl.google.com/android/repository/sys-img/android-tv/sys-img.xml

 Validate XML: https://dl.google.com/android/repository/sys-img/android-tv/sys-img.xml

 Parse XML:    https://dl.google.com/android/repository/sys-img/android-tv/sys-img.xml

 Fetching URL: https://dl.google.com/android/repository/sys-img/google_apis/sys-img.xml

 Validate XML: https://dl.google.com/android/repository/sys-img/google_apis/sys-img.xml

 Parse XML:    https://dl.google.com/android/repository/sys-img/google_apis/sys-img.xml

 Fetching URL: https://dl.google.com/android/repository/sys-img/google_apis_playstore/sys-img.xml

 Validate XML: https://dl.google.com/android/repository/sys-img/google_apis_playstore/sys-img.xml

 Parse XML:    https://dl.google.com/android/repository/sys-img/google_apis_playstore/sys-img.xml

Error: Ignoring unknown package filter 'build-tools-28.0.0rc0'

Warning: The package filter removed all packages. There is nothing to install.

        Please consider trying to update again without a package filter.

# Check that aidl can be executed

# Search for Aidl

# Run '/home/username/.buildozer/android/platform/android-sdk-20/build-tools/23.0.1/aidl'

# Cwd None

# Android packages installation done.

# Check application requirements

# Check garden requirements

# Compile platform

# Run '/home/username/anaconda3/bin/python -m pythonforandroid.toolchain create --dist_name=helloWordApp --bootstrap=sdl2 --requirements=python3crystax,kivy,numpy --arch armeabi-v7a --copy-libs --color=always --storage-dir=/media/username/data/racine/Loisir/IT/pythonProgram/androidApp/.buildozer/android/platform/build'

# Cwd /media/username/data/racine/Loisir/IT/pythonProgram/androidApp/.buildozer/android/platform/python-for-android-new-toolchain

[INFO]:    Will compile for the following archs: armeabi-v7a

[INFO]:    Found Android API target in $ANDROIDAPI

[INFO]:    Available Android APIs are (19)

[INFO]:    Requested API target 19 is available, continuing.

[INFO]:    Found NDK dir in $ANDROIDNDK

[INFO]:    Got NDK version from $ANDROIDNDKVER

[WARNING]: NDK version was set as r9c, but checking the NDK dir claims it is 10.3.2.

[WARNING]: The build will try to continue, but it may fail and you should check that your setting is correct.

[WARNING]: If the NDK dir result is correct, you don't need to manually set the NDK ver.

[INFO]:    Using Crystax NDK r9c

[INFO]:    Found virtualenv at /home/username/anaconda3/envs/visionEnv/bin/virtualenv

[INFO]:    Found the following toolchain versions: ['4.9', '5', 'clang3.6', 'clang3.7']

[INFO]:    Picking the latest gcc toolchain, here 5

[INFO]:    No existing dists meet the given requirements!

[INFO]:    No dist exists that meets your requirements, so one will be built.

[ERROR]:   Didn't find any valid dependency graphs.

[ERROR]:   This means that some of your requirements pull in conflicting dependencies.

[ERROR]:   Exiting.

# Command failed: /home/username/anaconda3/bin/python -m pythonforandroid.toolchain create --dist_name=helloWordApp --bootstrap=sdl2 --requirements=python3crystax,kivy,numpy --arch armeabi-v7a --copy-libs --color=always --storage-dir=/media/username/data/racine/Loisir/IT/pythonProgram/androidApp/.buildozer/android/platform/build

#

# Buildozer failed to execute the last command

# The error might be hidden in the log above this error

# Please read the full log, and search for it before

# raising an issue with buildozer itself.

# In case of a bug report, please add a full log with log_level = 2



The python-for-numpy command is executed on conda root environment but I am using a diferent one:

/home/username/anaconda3/bin/python -m pythonforandroid.toolchain
create
--dist_name=helloWordApp --bootstrap=sdl2
--requirements=python3crystax,kivy,numpy --arch armeabi-v7a
--copy-libs --color=always
--storage-

My environment:
(visionEnv) Username:/media/username/data/racine/Loisir/IT/pythonProgram/androidApp$

Solution ? Tutorials ?
************************
I search the web to understand what is going wrong did not find the answer.
Does any one could provide me instruction to make it works ?

It would be nice to have a complete tutorial about how to make an APK based on python3, viky, numpy, scipy, scikit-image,... Lots of people are working with those libraries, I would make sense to make sure people can use buildozer with those libraries.

I wonder if there a problem of compatibility with conda...

A recipe may be the solution but even after looking at the documentation I have no idea about how to make numpy recipe and how to integrate it to buildozer...

阿奇蒙

unread,
May 23, 2018, 12:33:01 PM5/23/18
to Kivy users support

As far as  I know, replace the numpy recipe of your p4a with the one on https://github.com/frmdstryr/p4a-numpy will fix the numpy dist issue for a while.But you will get compile error after that, BTW, I am still working on this because I need numpy too.
I will back here as soon as I find the solution.
Reply all
Reply to author
Forward
0 new messages