Building and Signing apk from buildozer

[Horace Johnson]

Hello,

Let me start by saying I am normally a Windows user so I'm fairly new to Linux Ubuntu.

Here's my set-up....

Host Machine: windows 7 home p. 64 bit amd

Guest Machine: vm virtualbox, linux unbuntu 14.04 (guess auditions installed)

I have python 2.7, buildozer and dependencies all installed.  My cython is 0.21 because it's all over the net that 0.21 works with buildozer.  My buildozer is the latest version.

Here's the thing...  When I ran buildozer -v android debug, buildozer automated the whole process, downloading the sdk and ndk.  In my .spec file for requirements, I specified kivy=master.  Hope using "master" didn't break anything.

In the end, I do get a debug apk in the bin folder, which is inside the project root folder.  Now this is where it gets kind of strange...

When I run buildozer andorid release, the process is faster because the sdk and ndk were already downloaded but at the end, I don't get an apk file in my bin folder.  In fact, I have to navigate inside the .buildozer folder all the way to another bin folder and inside it I find the release apk.

- why was the release apk not placed inside the bin folder sitting in my project root?  Does the fact it wasn't mean something went wrong?

Going back to the debug apk, I can successfully use the keytool to create a .keystore file and run a successful jarsinger command.  The problem is, I can't do the last step which is....

$ ~/.buildozer/android/platform/android-sdk-21/build-tools/22.0.1/zipalign -v 4 ./<ProjectDir>/bin/<ProjApkName>-<ApkVersion>-release-unsigned.apk ./<ProjectDir>/bin/<ProjApkName>.apk

For some reason, I don't have that buildozer/android/platform/android-sdk-21/build-tools/22.0.1/zipalign -v 4 path.  In fact, there's no sdk or ndk inside my .buildozer folder.  There's no kivy folder inside it for that matter as well.

I'm assuming it was installed because I can make the debug apk; wouldn't it not work if the sdk and ndk were not present?  Not to mention it took along time to download the sdk and ndk.

I looked everywhere on my guess machine for android-sdk and ndk and there is no such folders on the virtual system!...?

I hope you can see why I'm starting to pull my hair out.

What should I do?  Should I manually download android studio, tools and sdk for linux and install, and then use the zipalign from there?

You know something, I know I saw the terminal stating that it was using zipalign on my debug apk.  I believe it was printed to the terminal as [zipalign].

Can someone please set me on the right path for buidling the release apk and get zipalign usable?  It would be greatly appreciated.  I've been at this for a week now because the original buildozer vdi is too small of a hard drive so I had to create another, larger vdi, download ubuntu and then fix all the issues with the ubuntu system.

I can't sleep knowing my sdk and ndk paths are invisible and toying with me.

[Horace Johnson]

I do get this one error only when I do an apk release build but not when I do a debug.  The release apk is generated inside the .buildozer folder and never gets moved/copied to the bin folder inside project root folder.

Traceback (most recent call last):

  File "/usr/local/bin/buildozer", line 9, in <module>

    load_entry_point('buildozer==0.33dev', 'console_scripts', 'buildozer')()

  File "/usr/local/lib/python2.7/dist-packages/buildozer-0.33dev-py2.7.egg/buildozer/scripts/client.py", line 13, in main

    Buildozer().run_command(sys.argv[1:])

  File "/usr/local/lib/python2.7/dist-packages/buildozer-0.33dev-py2.7.egg/buildozer/__init__.py", line 1029, in run_command

    self.target.run_commands(args)

  File "/usr/local/lib/python2.7/dist-packages/buildozer-0.33dev-py2.7.egg/buildozer/target.py", line 90, in run_commands

    func(args)

  File "/usr/local/lib/python2.7/dist-packages/buildozer-0.33dev-py2.7.egg/buildozer/target.py", line 107, in cmd_release

    self.buildozer.build()

  File "/usr/local/lib/python2.7/dist-packages/buildozer-0.33dev-py2.7.egg/buildozer/__init__.py", line 211, in build

    self.target.build_package()

  File "/usr/local/lib/python2.7/dist-packages/buildozer-0.33dev-py2.7.egg/buildozer/targets/android.py", line 765, in build_package

    copyfile(join(dist_dir, 'bin', apk), join(self.buildozer.bin_dir, apk))

  File "/usr/lib/python2.7/shutil.py", line 82, in copyfile

    with open(src, 'rb') as fsrc:

IOError: [Errno 2] No such file or directory: u'/home/kivy/Desktop/Doc140/.buildozer/android/platform/python-for-android/dist/doc140/bin/Doc140-0.1-release.apk'

kivy@kivy-VirtualBox:~/Desktop/Doc140$ 

[Horace Johnson]

Ok...  I found out that my sdk and ndk are in the hidden paths of ~/.buildozer

However, I don't think this works....

$ ~/.buildozer/android/platform/android-sdk-20/build-tools/19.1.0/zipalign -v 4 ~/Desktop/Dock140/bin/Doc140-1.0-release-unsigned.apk

nor this....

$ ~/.buildozer/android/platform/android-sdk-20/build-tools/19.1.0/zipalign -v 4 ~/Desktop/Dock140/bin/Doc140-1.0-release-unsigned.apk-1.0

I received a message once saying something about unable to unzip the apk.  The apk file is not a zip file to begin with.....or is it suppose to be?

Above, when I said buildozer android release was failing to put the final apk in the root bin folder, that was true.  However, the terminal did say the file was created successfully and even printed the time as 1 min 5 etc..

Then after that message, path to resource files start to print to the screen...!?  That's when the "no such directory or file" error pops up and my final apk is inside the .buildozer folder instead of in the bin folder in the project folder root.

Anyway, as far as zipalign goes, what am I doing wrong?

[Horace Johnson]

OMG!  Can't believe I figured out so much on my own!

I got the zipalign to work, yay!

I just have one worry now...can someone please answer this question...which was asked about as well.,,,

the command, buildozer android release....

Is it normal for the release apk to build in .buildozer/android/platform-for-android/dist/myproj/ and not move to the bin folder in main root ...?

I ask this because, when I build the debug version (buildozer android debug), that apk ends up in the bin folder that's located just inside the main project folder (not inside .buildozer).

Actually I have another question.  I couldn't run the debug apk on my android (didn't try release version).  Is it because I didn't sign it?  The terminal output for the debug apk shows that it signs and zipalign the debug apk.  Is there something else needed to test your apk on an android device (phone)?

[Horace Johnson]

Do I need a Google Developers Account and link with it, in order to test my debug apk?

[ZenCODE]

You can test and load your APK without signing, but need to enable developers mode and the 'Install from unknown sources' options on the device. Google it, as it changes for each version of android...

[Horace Johnson]

Yeah, I do have "from unknown source" selected.  Buildozer signs the debug apk with a debug signing.  I noticed the size of my apk is different when copying from Linux to Windows.  The apk goes from 506 mb to 483 mb.  Could data lose be causing the install to fail?

[Horace Johnson]

finally got my guest to pickup my usb ports and that allowed me to use the deploy logcat command.

This is the error...although it's not much to go on.

----------------------------------------------------------------

# Android packaging done!

# APK Doc140-0.1-debug.apk available in the bin directory

# Run '/home/kivy/.buildozer/android/platform/android-sdk-20/platform-tools/adb devices'

# Cwd None

* daemon not running. starting it now on port 5037 *

* daemon started successfully *

List of devices attached 

786A893474AC device

# Deploy on 786A893474AC

# Run '/home/kivy/.buildozer/android/platform/android-sdk-20/platform-tools/adb install -r /home/kivy/Desktop/Doc140/bin/Doc140-0.1-debug.apk'

# Cwd /home/kivy/.buildozer/android/platform

protocol failure

error: device not found

- waiting for device -

rm failed for /data/local/tmp/Doc140-0.1-debug.apk, No such file or directory

# Command failed: /home/kivy/.buildozer/android/platform/android-sdk-20/platform-tools/adb install -r /home/kivy/Desktop/Doc140/bin/Doc140-0.1-debug.apk

# 

# Buildozer failed to execute the last command

# The error might be hidden in the log above this error

# Please read the full log, and search for it before

# raising an issue with buildozer itself.

# In case of a bug report, please add a full log with log_level = 2

[Horace Johnson]

After doing a lot of testing using the compass example from kivy files, I've made some new discoveries.  First I noticed the compass app installed as a debug app.  I then added my cab folder (which holds my graphics for my project) which is around 500 mb in size to the compass project and did another android build.  The larger sized apk did not install!  I then deleted some of the folders in "cab" to shrink the size down and then re-built the compass debug apk again.  The apk did install with the small size.

So now I'm concluding it's a storage issue but that was strange because my device's internal storage is around 2.2 gb (but way less for apps).  My SD card is 16 gb with 9 gb free.  When I looked to see where the compass app installed, it was on internal storage!  My internal storage will only allow a little over 100 mb for apps!

So the problem is the fact, I was not installing to my SD card like I thought I was.

Which brings me to this question, which could finally solve this, how do you install to the SD card and not internal storage?

[Horace Johnson]

Man....  Looks like there's no real control over the manifest.xml file that gets auto generated by buildozer, therefore no way to change the "auto" value to one pointing to external storage.  "Gawd" developers...  Why would you do that?

I guess my only option is to include my cab folder with the sub directories but have them all empty and have the apt download all the graphics from a server and place them in the correct sub folders.

This.....   This is devastating.

[Horace Johnson]

You apparently need a lot more free space on internal storage than the size of the apk.  I was able to get my apk to install after freeing up a lot of internal space.  Now I can start the app but it stops with the message "could not extract private data".  I'm taking a "smartmouth" guess an assuming this error is because I'm running out of space during the extraction...hence why the data couldn't be extracted.

I tried to move the file to SD card but that puzzles me the most.  The app wouldn't move to SD card.  Anyone knows why it can't move to SD card?  9 gb free space is on the SD so it's not a space issue there.

Looks like I'm going to have to go through the trouble of creating a reduced graphics set for hand-helds devices.  I didn't wanna go through that much trouble but that'll reduce the overall package size.  I still wanna move my app to the SD card...that issue is freaking me out.

I had success pushing the smaller test compass app to external sd using apktool but didn't have the same success with my larger package.  Doesn't it have something to do with .spec file...android.private_storage = ?

[Horace Johnson]

I created my hand-held graphic cab and have no issues installing or moving to SD.  However, my app just closes after start with no error message.  Is there a way to make the app (kivy) write an error log to the resource directory so you can read it?

[ZenCODE]

You can set the log_dir using the config. Make sure to do this first thing, right at the top of main.py.

https://kivy.org/docs/api-kivy.config.html

[Horace Johnson]

Let me add that in.  In the mean time...  I have pinpointed the first section of code which cause the crash.

----code--------------------

MMpath = IPlatPath.GetPath(AppDir, ['cab', 'inc'])

sys.path.append(MMpath)

import a_sync, g_wind, g_ui, b_grnd, lv_objs, lv_objs_a, eff_sfx, my_debug, a_actor, s_sys

-------------------------------------------------------------------------------------

No surprise...  I figured after making an apk, trying to import my own modules from my own custom folder would be insufficient.  Can anyone tell me how to correctly add modular paths and import?

[Horace Johnson]

Correct?

Config.set('log_dir', abspath(''))

Config.set('log_enabled', '1')

[Horace Johnson]

Got it on the desktop working as,

Config.set('kivy', 'log_dir', abspath(''))

Config.set('kivy', 'log_enabled', '1')

Config.write()

Don't know if this will work as an apk.

[Horace Johnson]

[WARNING] stderr: ImportError: No module named a_sync

Either my style of trying to import the modules is wrong, or it's the fact I used the pyc module files instead of py...  ??

[ZenCODE]

Try putting an __init__.py file alongside your main.py and make sure a_sync.py is here too. Or is a_sync a python 3 import and you're using 2?

[Horace Johnson]

a_sync and all my modules are inside the inc folder which is inside my cab folder which is inside the root folder with main.py

I'm trying to import as...

import a_sync, next-module, next-module, etc...

Apparently, import doesn't work after building and apk.

Using my modules by name is import because I address variables through the module names, e.g. a_sync.GUI

I wish I could just add my modules to site-packages but buildozer pulls python from a tar file and adds it to the project so I don't see that happening.

I don't use __init__.py

I just import each module one by one.

I believe this issue is the only thing stopping me from successfully running my program as an apk on android.

[Horace Johnson]

Correction...

I do this in my app...

self.GameSync = a_sync.AppSync()

[Horace Johnson]

I was thinking about adding my modules to the root folder with main.py and the then call import because appending to sys.path isn't helping.

The thing is, the pyo files.  Pyo files are normally not import unless you run the main in a special mode, which I know nothing about (something about -o).

In game folder on my android, I have both main.py and main.pyo.  I don't know which one is being called to at startup.

[ZenCODE]

Inside some other folder (such an 'inc') will change the the import path as result in your breaking. Why not keep it simple.

1. Put these file alongside your main.py.
2. place an __init__.py here too.

That should work? Of course, there are more sophisticated ways to do this but you can research Python module importing for that and beyond the scope of kivy. It just uses Python.

Even if you choose another solution, just try this to get it working. Once it's working, you can experiment with alternative approaches?

[Horace Johnson]

I dumped the modules in the root before and it didn't work but I believe the spec file setting for source files and maybe extending the python path caused issues.  I knew it when I saw the main.py along side the main.pyo.  It should be all pyo.  I have my modules importing correctly now from root but know I'm worried.

My app closed again...this time complaining that a wav file didn't exits but it did, inside my cab folder.  I'm curious now...  Can you even have sub folders?  How else can you maintain a databased for you app if you have to dump everything in root...that's absurd.  I'm attaching the log file this time...maybe it's something else (hopefully).

[ZenCODE]

Yes, you can have as many subfolders as you want. And kivy packs and unpacks these normally without issue. How are you specifying the path? Try also printing a directory listing here so you can see what files and in the place you are looking.

[ZenCODE]

ps. Mobile OS's place your code is a read-only location. You want to use this path for a writable folder.

https://kivy.org/docs/api-kivy.app.html?highlight=user%20folder#kivy.app.App.user_data_dir

[Horace Johnson]

The spec file cause the paths to not be found...resource paths that is.  Modules have to be in root.  I'm now facing another issue.  My graphics have scaling code and what I mean by that is, position of graphics and size of graphics (wxh) is all done by % based on resolution.  That's what my a_sync mod does...it resizes the graphics if the screen changes size.

It's perfect on desktop and was perfect with KivyLauncher, until I updated the Launcher.  When building a debug apk pack, the same issue is there...  My graphics are all deformed looking!

I used kivy 1.9.0 on desktop an for the apk, I specified k==master in the spec file.  "Jeezus" "Kryst"...  Just one problem after the next.  This graphic issue just happened with the Kivy Launcher...graphics were scaled perfectly but as I said, I updated the Kivy Launcher.

My images are all pretty much png.  Any idea of what could be causing distorted graphics?

[Horace Johnson]

I uploaded a screenshot of the issue.  This issue just happened lately, since I've been building apks and updated Kivy Launcher.  I've changed nothing about my project so I'm assuming something is broken in buildozer's dependencies.

[ZenCODE]

I would suggest making another post for this. The title of this post is no longer appropriate.