Python-for-android: compile python only

[Peter]

I'm trying to 'hijack' Python for Android to get a set of 32 and 64 Python 3.7 libraries and header files to use in my existing application.

What is the best way to do this?

I tried 'p4a create' using the latest release, which gave this error:

[ERROR]:   Build failed: Couldn't find executable for CC. This indicates a problem locating the arm-linux-androideabi-gcc executable in the Android NDK, not that you don't have a normal compiler installed. Exiting.

I guess this is because I installed nkd-r20b, which no longer uses gcc.

I then installed the latest master version, which can proceed a bit further, but gave me this error:

libtool: compile:  /usr/bin/ccache /home/pbienst/android/android-ndk-r20b/toolchains/llvm/prebuilt/linux-x86_64/bin/clang -target armv7a-linux-androideabi17 -fomit-frame-pointer -march=armv7-a -mfloat-abi=softfp -mfpu=vfp -mthumb -fPIC -DHAVE_CONFIG_H -I. -I. -I./include -Iinclude -I./src -DANDROID -D__ANDROID_API__=17 -I/home/pbienst/android/android-ndk-r20b/sysroot/usr/include/arm-linux-androideabi -I/home/pbienst/.local/share/python-for-android/build/python-installs/unnamed_dist_3/include/python3.7 -target armv7a-linux-androideabi17 -fomit-frame-pointer -march=armv7-a -mfloat-abi=softfp -mfpu=vfp -mthumb -fPIC -Wall -fexceptions -MT src/closures.lo -MD -MP -MF src/.deps/closures.Tpo -c src/closures.c  -fPIC -DPIC -o src/.libs/closures.o

src/arm/sysv.S:363:2: error: invalid instruction, did you mean: fldmiax?

 fldmiadgt ip, {d0-d7}

 ^

src/arm/sysv.S:399:2: error: invalid instruction

 stmeqia r2, {r0, r1}

 ^

src/arm/sysv.S:412:2: error: invalid instruction, did you mean: fstmiax?

 fstmiadeq r2, {d0-d3}

 ^

/tmp/sysv-a335f9.s:263:2: error: invalid instruction

 fstmfdd sp!, {d0-d7}

 ^

/tmp/sysv-a335f9.s:313:2: error: invalid instruction, did you mean: fldmiax?

 fldmiad sp, {d0-d1}

 ^

/tmp/sysv-a335f9.s:316:2: error: invalid instruction, did you mean: fldmiax?

 fldmiad sp, {d0-d3}

 ^

Makefile:1346: recipe for target 'src/arm/sysv.lo' failed

make: *** [src/arm/sysv.lo] Error 1

[Robert Flatt]

use NDK 17c and SDK 29

First do

p4a clean_all

No idea what it takes to pull off the hijack, but here are some old instructions for p4a

https://groups.google.com/d/msg/kivy-users/Lf4zlYmLVPo/4UqxOozQAwAJ

And some updates

https://groups.google.com/d/msg/kivy-users/Lf4zlYmLVPo/I-ofobkbBgAJ

[Alexander Taylor]

If you use an NDK version that the release actually supports (17c, as Robert has said), that should resolve your immediate problem.

In general, p4a's python build is pretty straightforward. There isn't a prebuilt method to export the python build alone, but it should be easy to just pull it out of the build.

[Peter]

OK, downgrading the NDK seems to have worked, but 'p4a create' only created the armeabi-v7a libraries.

I can't get the others to build:

$ p4a create --arch=arm64-v8a

[WARNING]: $ANDROIDNDKVER is deprecated and no longer necessary, the value you set is ignored

[INFO]:    Will compile for the following archs: arm64-v8a

[INFO]:    Found Android API target in $ANDROIDAPI: 28

[INFO]:    Available Android APIs are (28)

[INFO]:    Requested API target 28 is available, continuing.

[INFO]:    Found NDK dir in $ANDROIDNDK: /home/pbienst/android/android-ndk-r17c

[INFO]:    Found NDK version 17c

[INFO]:    Found Android API target in $NDKAPI

[INFO]:    Found virtualenv at /usr/bin/virtualenv

[INFO]:    Found the following toolchain versions: ['4.9']

[INFO]:    Picking the latest gcc toolchain, here 4.9

[INFO]:    Of the existing distributions, the following meet the given requirements:

[INFO]:         unnamed_dist_1: min API 21, includes recipes (hostpython3, libffi, openssl, sqlite3, python3, genericndkbuild, setuptools, six, pyjnius, android), built for archs (armeabi-v7a)

[INFO]:    unnamed_dist_1 has compatible recipes, using this one

[Robert Flatt]

arm64-v8a works fine for me

This should have deleted the armeabi-v7a binaries (and a lot of other stuff)

p4a clean_all

[Peter]

Ok, I managed to get all the libraries and headers which I need, but I ran into another problem. I'm porting existing software which previously successfully used the Crystax Python libraries.

My code looks like this:

LOG("About to start Py_Initialize");

Py_Initialize();

LOG("Done Py_Initialise");

It never displays the second LOG statement, but crashes with:

2019-12-19 19:00:44.955 7787-7787/org.mnemosyne I/pybridge: About to start Py_Initialize

2019-12-19 19:00:44.973 7787-7839/org.mnemosyne A/libc: FORTIFY: pthread_mutex_lock called on a destroyed mutex (0x726a14158850)

2019-12-19 19:00:44.974 7787-7839/org.mnemosyne A/libc: Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 7839 (RenderThread), pid 7787 (org.mnemosyne)

Any idea what could cause this, and/or how to debug this better?

Full code here: https://github.com/mnemosyne-proj/mnemosyne/blob/android64/mnemosyne/android/app/src/main/cpp/pybridge.c

Thanks!

[Alexander Taylor]

I don't have any deep insight, but in my experience crashes during Py_Initialize are usually because the python environment isn't set up right so python doesn't find something it needs (I think usually the stdlib.zip). However, I'm sure there could be many other causes. It may be necessary to get gdb working to really look into it.

I haven't checked exactly what you're doing, but is there any possibility you could start with something essentially equivalent to the p4a start.c, and try to adjust it to your needs in steps so that you can see exactly when it eventually stops working?

--
You received this message because you are subscribed to a topic in the Google Groups "Kivy users support" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kivy-users/z_V5hLHQJ1w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kivy-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kivy-users/e90fc581-5c8f-44c7-b98e-5fc5e6a37cc3%40googlegroups.com.

[Peter]

OK, after a lot of blood, sweat and tears, I found out that the module libraries like zlib.cpython-37m.so are supposed to go in assets, and not in jniLibs.

I think this makes it impossible to use a single Android App Bundle for multiple architectures. I tried working around this by putting the libraries in jniLibs anyway, and preloading them before PyInitialise with

System.load(UIActivity.getApplicationInfo().nativeLibraryDir + "/libsqlite3.so");

System.load(UIActivity.getApplicationInfo().nativeLibraryDir + "/libpython3.7m.so");

System.load(UIActivity.getApplicationInfo().nativeLibraryDir + "/zlib.cpython-37m.so");

However, this doesn't work.

Is this a fundamental limitation of the architecture, or is there a way around it? 

Can we set Py_SetPath to include nativeLibrary dir in a portable way?

Thanks!

Peter