Python3.5 does not run on Android device

515 views
Skip to first unread message

João Ventura

unread,
Mar 30, 2016, 5:54:18 PM3/30/16
to crystax-ndk
Hi there,

so I've heard about crystax-ndk through kivy's python-for-android. I've tried to test python3.5 on android armeabi-v7a device, but I always get this error:


03-30 22:47:06.333 20166-20166/com.example.pytest I/pytest: Initializing the Python interpreter
03-30 22:47:06.348 20166-20166/com.example.pytest A/libc: Fatal signal 11 (SIGSEGV), code 1, fault addr 0x20 in tid 20166 (.example.pytest)

This is the function that get's called and throws the error (square.c):

JNIEXPORT jint JNICALL Java_com_example_pytest_Square_square
(JNIEnv *env, jclass jc, jint value)
{
LOG("Initializing the Python interpreter");
Py_Initialize();

LOG("Finalizing the Python interpreter");
Py_Finalize();

return value * value;
}

Note that this works with python 2.7.2 build with Kivy's python-for-android toolchain (with some patches to run with bionic), and it is a very barebones example..

My Application.mk file is:

APP_PLATFORM := android-19
APP_ABI := armeabi-v7a

and my Android.mk file is something like:

LOCAL_PATH := $(call my-dir)
CRYSTAX_PATH := /path/to/crystax-ndk-10.3.1

# Build libsquare.so
include $(CLEAR_VARS)
LOCAL_MODULE := square
LOCAL_SRC_FILES := square.c
LOCAL_LDLIBS := -llog
LOCAL_SHARED_LIBRARIES := python3.5m
include $(BUILD_SHARED_LIBRARY)

# Include libpython3.5m.so
include $(CLEAR_VARS)
LOCAL_MODULE := python3.5m
LOCAL_SRC_FILES := $(CRYSTAX_PATH)/sources/python/3.5/libs/$(TARGET_ARCH_ABI)/libpython3.5m.so
LOCAL_EXPORT_CFLAGS := -I $(CRYSTAX_PATH)/sources/python/3.5/include/python/
include $(PREBUILT_SHARED_LIBRARY)

# Include libcrystax.so
include $(CLEAR_VARS)
LOCAL_MODULE := crystax
LOCAL_SRC_FILES := $(CRYSTAX_PATH)/sources/crystax/libs/$(TARGET_ARCH_ABI)/libcrystax.so
include $(PREBUILT_SHARED_LIBRARY)
 
I compile my libsquare.so running: ~/Library/Android/android-ndk-r11b/ndk-build

Anyone has any idea why I get that "Fatal signal 11"?


Thanks,
João Ventura

Dmitry Moskalchuk

unread,
Mar 30, 2016, 5:57:58 PM3/30/16
to cryst...@googlegroups.com
Hello,

This is definitely the simplest possible example and it should work. The fact it crashes means there is something wrong. Give me some time, I'll try to reproduce the problem and will back with more information.

-- 
Dmitry Moskalchuk
--
You received this message because you are subscribed to the Google Groups "crystax-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to crystax-ndk...@googlegroups.com.
To post to this group, send email to cryst...@googlegroups.com.
Visit this group at https://groups.google.com/group/crystax-ndk.
To view this discussion on the web visit https://groups.google.com/d/msgid/crystax-ndk/a397d7e1-d09d-4d39-9768-baf3d72e6962%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

João Ventura

unread,
Mar 30, 2016, 6:06:37 PM3/30/16
to crystax-ndk
Hi Dmitry,

thanks!
Btw, sorry for the double post, but could not see my message published and I thought I pressed the wrong button. Can you please delete the other message?


João Ventura

Dmitry Moskalchuk

unread,
Mar 30, 2016, 6:18:02 PM3/30/16
to cryst...@googlegroups.com
Sure, no problem. I've removed duplicate post.

This happens because this is pre-moderated group so first message require manual approval. From now, all your messages will go without delay.

-- 
Dmitry Moskalchuk
--
You received this message because you are subscribed to the Google Groups "crystax-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to crystax-ndk...@googlegroups.com.
To post to this group, send email to cryst...@googlegroups.com.
Visit this group at https://groups.google.com/group/crystax-ndk.

Dmitry Moskalchuk

unread,
Mar 30, 2016, 6:37:13 PM3/30/16
to cryst...@googlegroups.com
OK, this was simple. Here is stderr from Py_Initialize(), when I ran your code from exe, not from so:

  Initializing the Python interpreter
  Could not find platform independent libraries <prefix>
  Could not find platform dependent libraries <exec_prefix>
  Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
  Fatal Python error: Py_Initialize: Unable to get the locale encoding
  ImportError: No module named 'encodings'

You didn't see that because stderr is not duplicated to logcat, and Python itself knows nothing about logcat and other Android specifics (in fact, it was our main goal - build Python to Android "as is", not touching even single byte in Python sources). However, it makes everything harder to debug, when stderr just goes into /dev/null instead of reporting problem. I'll try to add re-routing stdout/stderr to logcat to the libcrystax, but this is very basic thing and require high accuracy to make it properly, not breaking anything else.

Back to your issue - yes, it crashes inside Py_Initialize just because it can't find required Python module, implemented as a native extension. You should package all native modules and stdlib.zip within your application if you want embed Python into your application. If you'd look into $NDK/sources/python/3.5/libs/$ABI/ folder, you'll see 'modules' there, as well as stdlib.zip. Actually, before calling Py_Initialize, you should prepare environment for that, loading all native modules first and setting path to stdlib.zip. You can see example how we do that with our 'python' executable here: https://github.com/crystax/android-platform-ndk/blob/master/build/tools/build-target-python/interpreter.c.3.5

I know, Python embedding doesn't work yet as good as it can be (just because you should remember about all that things I've described above), but we're working on it, so embedding will work better soon. CrystaX NDK 10.3.1 was the first release where we've included Python, and we were concentrated mainly on getting it working "as is" - i.e. running it as an interpreter. Easy Python embedding is our next step, even though it already works, just not so easy as we would like to.

-- 
Dmitry Moskalchuk

João Ventura

unread,
Mar 31, 2016, 5:50:32 AM3/31/16
to crystax-ndk
Hi Dmitry,

thanks for catching that error!
It seems that contrary to what was done in Python2.7, Py_Initialize() in Python3 requires the encodings package in the stdlib (http://stackoverflow.com/a/9965641/4555926). So one has to copy the stdlib to the device, which makes Python3 a little bit less "embeddable" than Python2.

I'm going to play a little bit with this, and will write back later if I can make it work!


João Ventura
Message has been deleted
Message has been deleted

João Ventura

unread,
Mar 31, 2016, 11:05:33 AM3/31/16
to crystax-ndk
Just to confirm that I got it to work!


Here's some guidelines for those who may wish to try it as well:

1. You must find a way to copy the stdlib.zip file to the apk and extract it on the device.

Read what I've wrote in http://joaoventura.net/blog/2014/python-android-4/ regarding the AssetExtractor.java class. Basically you should copy stdlib.zip file to main/assets, and extract it on the device.

2. Before initialize, you should call Py_SetPath with the location of the "extracted" stdlib.zip file.

You must pass the location of the stdlib.zip file to the JNI layer. Since the AssetExtractor class above extracts the files in /assets to "android-data-dir"/files/, you must send that string to the JNI layer. I don't know if it is possible to get it directly on the JNI layer.

3. Then you can call Py_Initialize.

And you're good to go!


I'm thinking about implementing a generic library to pass JSON information from Java to Python via this JNI C layer using crystax-ndk. My idea is to implement the business logic in Python and use the data in the Java layer, so that I can do the same for iOS and desktop devices..

Thanks for the help!
João Ventura

Dmitry Moskalchuk

unread,
Mar 31, 2016, 11:12:37 AM3/31/16
to cryst...@googlegroups.com
Great, I'm glad you got it working!

-- 
Dmitry Moskalchuk

João Ventura


--
You received this message because you are subscribed to the Google Groups "crystax-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to crystax-ndk...@googlegroups.com.
To post to this group, send email to cryst...@googlegroups.com.
Visit this group at https://groups.google.com/group/crystax-ndk.

João Ventura

unread,
Jul 22, 2016, 5:20:58 PM7/22/16
to crystax-ndk
Hi again,

so I've been working on my project which uses crystax's libpython, and everything has been working fine while I've been working in "debug". However, now I'm ready to send the app to Google Play but it always crashes in "release" mode!

The crash happens when I call Py_Initialize() after I correctly set the location of the stdlib.zip file with Py_SetPath(). As I said, it works perfectly while in "debug"..

Here's the logcat for the event:
07-22 22:18:50.248 24633-24633/? I/pybridge: Initializing the Python interpreter
07-22 22:18:50.275 24633-24633/? A/libc: Fatal signal 6 (SIGABRT), code -6 in tid 24633 (latangle.charts)
07-22 22:18:50.378 2793-2793/? A/DEBUG: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
07-22 22:18:50.378 2793-2793/? A/DEBUG: Build fingerprint: 'motorola/titan_retaildsds/titan_umtsds:6.0/MPB24.65-34/31:user/release-keys'
07-22 22:18:50.378 2793-2793/? A/DEBUG: Revision: 'p400'
07-22 22:18:50.378 2793-2793/? A/DEBUG: ABI: 'arm'
07-22 22:18:50.378 2793-2793/? A/DEBUG: pid: 24633, tid: 24633, name: latangle.charts  >>> com.flatangle.charts <<<
07-22 22:18:50.378 2793-2793/? A/DEBUG: signal 6 (SIGABRT), code -6 (SI_TKILL), fault addr --------
07-22 22:18:50.401 2793-2793/? A/DEBUG:     r0 00000000  r1 00006039  r2 00000006  r3 b6feeb7c
07-22 22:18:50.401 2793-2793/? A/DEBUG:     r4 b6feeb84  r5 b6feeb34  r6 0000000d  r7 0000010c
07-22 22:18:50.401 2793-2793/? A/DEBUG:     r8 b6d96e50  r9 ffffffff  sl be92a4ac  fp a4026b3c
07-22 22:18:50.401 2793-2793/? A/DEBUG:     ip 00000006  sp be92a458  lr b6d74d89  pc b6d76cc0  cpsr 40070010
07-22 22:18:50.408 2793-2793/? A/DEBUG: backtrace:
07-22 22:18:50.408 2793-2793/? A/DEBUG:     #00 pc 00043cc0  /system/lib/libc.so (tgkill+12)
07-22 22:18:50.408 2793-2793/? A/DEBUG:     #01 pc 00041d85  /system/lib/libc.so (pthread_kill+32)
07-22 22:18:50.408 2793-2793/? A/DEBUG:     #02 pc 0001b8e7  /system/lib/libc.so (raise+10)
07-22 22:18:50.408 2793-2793/? A/DEBUG:     #03 pc 0001c683  /data/app/com.flatangle.charts-2/lib/arm/libcrystax.so (abort+26)

I've read that there's another crystax user having a similar problem regarding the "release" mode. Has anybody tried to make a release build already with libcrystax? Or does anybody have any idea of what could be making this?


Thanks,
João Ventura

Dmitry Moskalchuk

unread,
Jul 23, 2016, 4:13:08 AM7/23/16
to cryst...@googlegroups.com
Hello,

Yes, I've heard about similar problem in case of release build, but were not yet able to figure out what's wrong. Looking on your call stack, I see that top of the stack is "abort" function, but this doesn't explain real reason - who called "abort" and why?

I'm pretty sure it would be easy to fix, but first I need to reproduce the issue. I would be very appreciated if you could provide some minimal example demonstrating how to get the problem appearing. Feel free to file a ticket on our bug tracker (https://tracker.crystax.net/projects/ndk/issues) if needed.

-- 
Dmitry Moskalchuk
--
You received this message because you are subscribed to the Google Groups "crystax-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to crystax-ndk...@googlegroups.com.
To post to this group, send email to cryst...@googlegroups.com.
Visit this group at https://groups.google.com/group/crystax-ndk.

João Ventura

unread,
Jul 23, 2016, 9:23:37 AM7/23/16
to crystax-ndk
Hi Dmitry,

thanks for the fast response!

I've made a small example and published it on my github account: https://github.com/joaoventura/pyapp.
Since I have to copy the python stdlib.zip from the assets to the device, I had to use Java, and therefore this is the minimal example I could make for you to test.

I've added some comments in the readme file, but in MainActivity.java I extract the python files from the assets folder to the device, and then I use JNI to set the paths and try to initialize Python. It works on debug, but when I build the signed release APK, it always crashes.

I have a ARMv7 device, so I only compile for armeabi-v7a, but change in jni/Application.mk for the architectures you have. Let me know if you need me to make some changes to this code or test other things! I'm really enjoying the solution that I have for reusing my Python code in Android and really grateful for your project crystax to allow me to do it!


João Ventura

João Ventura

unread,
Jul 24, 2016, 6:21:56 AM7/24/16
to crystax-ndk
I've been messing a little bit with the build.grade file, and this is how I have it now:

buildTypes {
    release
{
       
//debuggable true
        minifyEnabled false
        proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        signingConfig signingConfigs.release
   
}
    debug
{
        minifyEnabled
false
        proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
        signingConfig signingConfigs.release
   
}
}

So, if I sign the debug build and run proguard, the debug build still works fine! The release build still fails.

However, if I uncomment the "debuggable true" line in release, the release build starts to work. So it is definitely related to something about the debugging vs release process in Android (I tested like this because I was suspecting that the signing process may be causing this error..)! Unfortunately I still do not know much about how Android (or crystax) manages debug vs release builds internally, so I'll keep looking for the low-hanging fruits..

Dmitry Moskalchuk

unread,
Jul 27, 2016, 2:07:29 AM7/27/16
to cryst...@googlegroups.com
Hello,

Just thought I haven't reacted to your message, so here it is.

I was able to reproduce the issue, but I still don't know why this happens. I'm debugging it and will let you know as soon as will know more about root cause.
-- 
Dmitry Moskalchuk
--
You received this message because you are subscribed to the Google Groups "crystax-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to crystax-ndk...@googlegroups.com.
To post to this group, send email to cryst...@googlegroups.com.
Visit this group at https://groups.google.com/group/crystax-ndk.

João Ventura

unread,
Jul 27, 2016, 5:27:45 AM7/27/16
to crystax-ndk
Hi Dmitry,

thanks for the feedback.
I'll keep waiting, let me know if I can help you with anything (such as testing or other code snippets).

By the way, last Saturday I've tried to register myself in crystax's issue tracker, but I've still not received the confirmation email (and it is not on my spam list).


João Ventura

Dmitry Moskalchuk

unread,
Jul 27, 2016, 7:52:17 AM7/27/16
to cryst...@googlegroups.com
Hi João,

I've activated your account in our bug tracker. However, this is strange you haven't received automatic confirmation e-mail - I've checked logs and I clearly see that message was sent to your gmail address and Google's SMTP server response was "250 2.0.0 OK" - i.e. mail was received and there was no errors. Please double check your mail box - you either miss that e-mail or (unlikely) it was Google's fail.

-- 
Dmitry Moskalchuk
--
You received this message because you are subscribed to the Google Groups "crystax-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to crystax-ndk...@googlegroups.com.
To post to this group, send email to cryst...@googlegroups.com.
Visit this group at https://groups.google.com/group/crystax-ndk.

João Ventura

unread,
Jul 27, 2016, 11:33:43 AM7/27/16
to crystax-ndk
Hi again Dmitry,

my fault, the email was indeed in my spam folder, I don't know how I missed it the first time!

I've submitted a new bug report at https://tracker.crystax.net/issues/1455.


João Ventura

Dmitry Moskalchuk

unread,
Jul 27, 2016, 12:56:44 PM7/27/16
to cryst...@googlegroups.com

I see. I'll let you know as soon as get it figured out. Thank you!

-- 
Dmitry Moskalchuk
--
You received this message because you are subscribed to the Google Groups "crystax-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to crystax-ndk...@googlegroups.com.
To post to this group, send email to cryst...@googlegroups.com.
Visit this group at https://groups.google.com/group/crystax-ndk.

João Ventura

unread,
Aug 2, 2016, 3:21:15 PM8/2/16
to crystax-ndk
I can confirm that the issue is now fixed (https://tracker.crystax.net/issues/1455). Thank you Dmitry for the fast response and how swiftly the issue was fixed!

Dmitry Moskalchuk

unread,
Aug 2, 2016, 3:50:06 PM8/2/16
to cryst...@googlegroups.com

Sure, no problem.

And thank you too! Your report was actually an indicator that we have missing testing of release applications (those having android:debuggable=false in AndroidManifest.xml) in our automated testing procedure. It was non-trivial to do that though, since typical Android applications, consisting from Activities, Services and Broadcast Receivers, are not friendly to any kind of automated testing. Nevertheless, I did it, so I'm going to add such test case to our testing suite and we'll never miss it again - all non-debuggable applications would be verified on a regular basis after every new NDK build. Thank you for jogging me in a right direction!


-- 
Dmitry Moskalchuk
On 02/08/16 22:21, João Ventura wrote:
I can confirm that the issue is now fixed (https://tracker.crystax.net/issues/1455). Thank you Dmitry for the fast response and how swiftly the issue was fixed!
--
You received this message because you are subscribed to the Google Groups "crystax-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to crystax-ndk...@googlegroups.com.
To post to this group, send email to cryst...@googlegroups.com.
Visit this group at https://groups.google.com/group/crystax-ndk.

João Ventura

unread,
Aug 11, 2016, 12:39:38 PM8/11/16
to crystax-ndk
Hi everyone,

just to let you know that I've create a repo at https://github.com/joaoventura/pybridge with an example of how to implement a Python backend for a native Android app. I'm using PyBridge successfully on an application published on the Play Store: https://play.google.com/store/apps/details?id=com.flatangle.charts, and the performance is quite good..


João Ventura




Dmitry Moskalchuk

unread,
Aug 12, 2016, 8:13:37 AM8/12/16
to cryst...@googlegroups.com

Hi João,

Thank you very much for sharing your work with community!

However, I've looked on your example on github, and found some things, which definitely could be improved. I'm going to fix that and send you pull request in a day or two.


-- 
Dmitry Moskalchuk
--
You received this message because you are subscribed to the Google Groups "crystax-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to crystax-ndk...@googlegroups.com.
To post to this group, send email to cryst...@googlegroups.com.
Visit this group at https://groups.google.com/group/crystax-ndk.
Reply all
Reply to author
Forward
0 new messages