Python 3 and buildozer

[Piet Stevens]

Hi, sorry for the double post - after having posted in the Kivy development thread, I realized it was in the wrong place. My apologies. Piet

Hello, I am new to Kivy - thank you for all the work: very impressive.

I have made a skeleton app following along with the Kivy crash course videos and want to generate an app for Android.

PC: ubuntu 16.04
Python: 3.5.2 (installed in a virtual environment)
Kivy: 1.10

Not sure how to check the buildozer version.

I have 2 questions:
a) when running buildozer, I get an error in the tools\depsort.py file - seems to be python 2. I manually changed the last line from  'print main(sys.argv)' to 'print(main(sys.argv))'. The apk is generated but apparently not pushed to my android device (I used 'buildozer android debug deploy'). I manually copied it to the android device, installed it and it ran. Question: how to avoid the py error in depsort?

b) during the build, I noticed that buildozer is pulling in android 2.7 libraries and not 3.5. How to change that?

I looked around and found this: https://blog.kivy.org/2016/01/python-for-android-now-supports-python-3%C2%A0apks/

Do I need to stop using buildozer and make manual changes to my tools? Or is there any better instruction on how to generate a .apk with python 3?

Thank you,

Piet

[Chu Bun]

I'm in the same boat, and have been struggleing for a week now.  I assume you meant python not android 2.7 and 3.5 libraries.  When running buildozer, do you see warnings about missing autoconf, libtool, ....?  On my system (buildozer VM from Kivy Web site), these are missing.  After installing these missing packages, buildozer seems to use the correct libraries.  But my problems didn't end there.  In the end I decided to reverse back to Python 2 and wait for official support for python 3 in buildozer.

[Piet Stevens]

Hi, I get no autoconf or libtool warnings. What I meant was python-for-android 2.7 libraries instead of python-for-android 3.5 libraries. I am running buildozer 0.33dev.

Strange thing is so far, my apk on my Android phone works. But I want to get everything with the right libraries to avoid problems in the future.

[Alexander Taylor]

When you run `buildozer android debug`, you invoke the old python-for-android toolchain that does not support python3 at all. If you run instead `buildozer android_new debug`, it will use the new/current toolchain, which does support it. This latter method will be made the default sometime soonish.

Likewise, the old toolchain does not support building APKs with python3. The current toolchain does support it, but you must use the CrystaX NDK (https://www.crystax.net/en/android/ndk), then add `python3crystax` to your requirements line.

On Monday, 22 May 2017 10:36:53 UTC+1, Piet Stevens wrote:

Hi, sorry for the double post - after having posted in the Kivy development thread, I realized it was in the wrong place. My apologies. Piet

Hello, I am new to Kivy - thank you for all the work: very impressive.

I have made a skeleton app following along with the Kivy crash course videos and want to generate an app for Android.

PC: ubuntu 16.04
Python: 3.5.2 (installed in a virtual environment)
Kivy: 1.1

[Piet Stevens]

Thank you for your help. In the mean-time, it seems buildozer was upgraded to 0.34 and android_new is now android - coincidence?

In any case, I have downloaded ChrstaX NDK and added it in the .spec file.

Unfortunately, when I run in my virtual environment, I receive this error:
-----------------------------------------------------------------------------------------------------------------------------------
(kivyinstall) pstevens@quiet8:~/PycharmProjects/PSAndGPSDataAnnotator$ pip -V
pip 9.0.1 from /home/pstevens/kivyinstall/local/lib/python3.5/site-packages (python 3.5)
(kivyinstall) pstevens@quiet8:~/PycharmProjects/PSAndGPSDataAnnotator$ pip install colorama
Requirement already satisfied: colorama in /home/pstevens/kivyinstall/lib/python3.5/site-packages
(kivyinstall) pstevens@quiet8:~/PycharmProjects/PSAndGPSDataAnnotator$ pip install sh
Requirement already satisfied: sh in /home/pstevens/kivyinstall/lib/python3.5/site-packages
(kivyinstall) pstevens@quiet8:~/PycharmProjects/PSAndGPSDataAnnotator$ pip install jinja2
Requirement already satisfied: jinja2 in /home/pstevens/kivyinstall/lib/python3.5/site-packages
Requirement already satisfied: MarkupSafe>=0.23 in /home/pstevens/kivyinstall/lib/python3.5/site-packages (from jinja2)
(kivyinstall) pstevens@quiet8:~/PycharmProjects/PSAndGPSDataAnnotator$ buildozer android debug
# Check configuration tokens
# Ensure build layout
# Check configuration tokens
# Preparing build
# Check requirements for android
# Run 'dpkg --version'
# Cwd None
Debian 'dpkg' package management program version 1.18.4 (amd64).
This is free software; see the GNU General Public License version 2 or
later for copying conditions. There is NO warranty.
# Search for Git (git)
#  -> found at /usr/bin/git
# Search for Cython (cython)
#  -> found at /home/pstevens/kivyinstall/bin/cython
# Search for Java compiler (javac)
#  -> found at /usr/lib/jvm/java-8-oracle/bin/javac
# Search for Java keytool (keytool)
#  -> found at /usr/lib/jvm/java-8-oracle/jre/bin/keytool
# Install platform
# Run 'pip install -q --user "appdirs" "colorama>=0.3.3" "sh>=1.10" "jinja2" "six"'
# Cwd None
Can not perform a '--user' install. User site-packages are not visible in this virtualenv.
# Command failed: pip install -q --user "appdirs" "colorama>=0.3.3" "sh>=1.10" "jinja2" "six"
#
# Buildozer failed to execute the last command
# The error might be hidden in the log above this error
# Please read the full log, and search for it before
# raising an issue with buildozer itself.
# In case of a bug report, please add a full log with log_level = 2
------------------------------------------------------------------------------------------------------------------------------------------------------------------------

So, I tried to then re-install kivy etc... while not using a virtual environment.

Then, I run into the error that colorama, sh, jinja2, etc... packages are not found:
--------------------------------------------------------------------------------------------------------------
....
# Check that aidl can be executed
# Search for Aidl
# Run '/home/pstevens/.buildozer/android/platform/android-sdk-20/build-tools/23.0.1/aidl'
# Cwd None
# Android packages installation done.
# Check application requirements
# Check garden requirements
# Compile platform
# Run '/usr/bin/python -m pythonforandroid.toolchain create --dist_name=myapp1 --bootstrap=sdl2 --requirements=python3crystax,kivy --arch armeabi-v7a --copy-libs --color=always --storage-dir=/home/pstevens/PycharmProjects/PSAndGPSDataAnnotator/.buildozer/android/platform/build'
# Cwd /home/pstevens/PycharmProjects/PSAndGPSDataAnnotator/.buildozer/android/platform/python-for-android-master
ERROR: The colorama Python module could not be found, please install version 0.3.3 or higher
ERROR: The sh Python module could not be found, please install version 1.10 or higher
ERROR: The jinja2 Python module could not be found, please install it.
python-for-android is exiting due to the errors above.
--------------------------------------------------------------------------------------------------------------------------------------------------------

Yet, when I do a manual 'pip install', pip tells me they are already installed:

pstevens@quiet8:~/PycharmProjects/PSAndGPSDataAnnotator$ pip install --user colorama
Requirement already satisfied: colorama in /home/pstevens/.local/lib/python3.5/site-packages
pstevens@quiet8:~/PycharmProjects/PSAndGPSDataAnnotator$ pip install --user sh
Requirement already satisfied: sh in /home/pstevens/.local/lib/python3.5/site-packages
pstevens@quiet8:~/PycharmProjects/PSAndGPSDataAnnotator$ pip install --user jinja2
Requirement already satisfied: jinja2 in /usr/lib/python3/dist-packages
Requirement already satisfied: MarkupSafe in /usr/lib/python3/dist-packages (from jinja2)
pstevens@quiet8:~/PycharmProjects/PSAndGPSDataAnnotator$ pip install --user six
Requirement already satisfied: six in /usr/lib/python3/dist-packages

I would rather get this to work in my virtual environment. Any help would be welcome.

Thank you,

Piet

EDIT: OK, I have found a way to run more of buildozer in my virtual environment, as follows:
1. I installed buildozer to the virtual environment.
2. I edited site-packages/buildozer-0.34.dev0-py3.5.egg/buildozer/targets/android.py and changed line 529 from cmd('pip install -q --user {}'.format(" ".join(pip_deps))) to cmd('pip install -q {}'.format(" ".join(pip_deps))). Then, the build continues. Maybe the writers of buildozer can check for $VIRTUAL_ENV and not include --user when that is not empty? I do not feel confident enough to make such a change myself.

Unfortunately, the build does not run to the end. Now I run into this:
--------------------------------------------------------------------------------------------------------------------------------------------------------
[INFO]:    Building pyjnius for armeabi-v7a
[INFO]:    jnius apparently isn't already in site-packages
[INFO]:    Cythonizing anything necessary in pyjnius
Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/pstevens/PycharmProjects/PSAndGPSDataAnnotator/.buildozer/android/platform/python-for-android-master/pythonforandroid/toolchain.py", line 905, in <module>
    main()
  File "/home/pstevens/PycharmProjects/PSAndGPSDataAnnotator/.buildozer/android/platform/python-for-android-master/pythonforandroid/toolchain.py", line 902, in main
    ToolchainCL()
  File "/home/pstevens/PycharmProjects/PSAndGPSDataAnnotator/.buildozer/android/platform/python-for-android-master/pythonforandroid/toolchain.py", line 508, in __init__
    getattr(self, args.subparser_name.replace('-', '_'))(args)
  File "/home/pstevens/PycharmProjects/PSAndGPSDataAnnotator/.buildozer/android/platform/python-for-android-master/pythonforandroid/toolchain.py", line 147, in wrapper_func
    build_dist_from_args(ctx, dist, args)
  File "/home/pstevens/PycharmProjects/PSAndGPSDataAnnotator/.buildozer/android/platform/python-for-android-master/pythonforandroid/toolchain.py", line 190, in build_dist_from_args
    build_recipes(build_order, python_modules, ctx)
  File "/home/pstevens/PycharmProjects/PSAndGPSDataAnnotator/.buildozer/android/platform/python-for-android-master/pythonforandroid/build.py", line 565, in build_recipes
    recipe.build_arch(arch)
  File "/home/pstevens/PycharmProjects/PSAndGPSDataAnnotator/.buildozer/android/platform/python-for-android-master/pythonforandroid/recipe.py", line 975, in build_arch
    self.build_cython_components(arch)
  File "/home/pstevens/PycharmProjects/PSAndGPSDataAnnotator/.buildozer/android/platform/python-for-android-master/pythonforandroid/recipe.py", line 986, in build_cython_components
    '-c', 'import site; print("\\n".join(site.getsitepackages()))')
  File "/home/pstevens/kvnew/lib/python3.5/site-packages/sh.py", line 1427, in __call__
    return RunningCommand(cmd, call_args, stdin, stdout, stderr)
  File "/home/pstevens/kvnew/lib/python3.5/site-packages/sh.py", line 774, in __init__
    self.wait()
  File "/home/pstevens/kvnew/lib/python3.5/site-packages/sh.py", line 792, in wait
    self.handle_command_exit_code(exit_code)
  File "/home/pstevens/kvnew/lib/python3.5/site-packages/sh.py", line 815, in handle_command_exit_code
    raise exc
sh.ErrorReturnCode_1:

  RAN: /home/pstevens/kvnew/bin/python3.5 -c import site; print("\n".join(site.getsitepackages()))

  STDOUT:


  STDERR:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
AttributeError: module 'site' has no attribute 'getsitepackages'
-------------------------------------------------------------------------------------------------------------------------------------------------------
Not sure what to do - there is a web-site (going back to 2012 but with more recent comments, too) https://github.com/pypa/virtualenv/issues/355 indicating that this error also has to do with virtual environments. Will need to do more searching. Or maybe someone has an idea?

[Piet Stevens]

Just a report out on further developments, in case someone finds this useful.

I have not been able to get it to run in a virtual environment. pyjnius build did not finish without an error, still having to do with site.py to provide site.getsitepackages() (in a virtualenv, getsitepackages() returns empty and that generates a fatal error).

If I do not use a virtualenv, then I could get it to work thanks to Alexander Taylor's help (thank you). The clue was to ensure that the python that was running the p4a build was also python3. I used update-alternatives for that (found it here: https://askubuntu.com/questions/14615/how-do-i-make-the-terminal-run-python-3-1).