[p4a] CythonRecipe, NDKRecipe, or something else? For ctypes binding with C++ code

[Eason Liu]

Hi there,

I have a Python app on Ubuntu that uses `ctypes` to load a shared library originally written in C++. The shared library (libdds.so) was compiled on Ubuntu using a Makefile (g++) and worked fine.

To make it usable on Android, seems that some form of cross compilation is needed. I have tried replacing the g++ in the Makefile with an llvm compiler in NDK (e.g. aarch64-linux-android30-clang++); this successfully compiled a new .so file and Buildozer was able to package the entire app.

Further, the .so file can be loaded by Python's ctypes.cdll.LoadLibrary on Android:

07-20 13:58:44.508 10271 15663 15805 I python  : Loaded lib <CDLL '/data/data/io.github.lyiqian.bidly/files/app/solver/pythondds_min/libdds.so', handle 84e9a95dc9535acf at 0x7935589a60>

However, my Android app would exit without any errors as soon as it called a function defined in the lib.

Now I am looking into more properly compling the C++ code and packaging it with p4a/buildozer.

Should I go with the Cython binding/Recipe instead of the current ctypes approach? or

Should I investigate the NDKRecipe? or something else?

My environments:

- Ubuntu 22.04

- Python 3.8

- Android 11 (have another phone with 13 but didn't test with it)

- Kivy 2.2.1

Thank you for taking a look!

===

Attached are bulidozer.spec and the Makefile compiling C++ code for Linux.

Logcat logs capturing the app exit (after Calculating DDTable..) is also pasted here but it does not show much for me:

07-20 13:58:48.825 10271 15663 15805 I python  : [DEBUG  ] Initiating deal..
07-20 13:58:48.825 10271 15663 15805 I python  : [DEBUG  ] Initiating result..
07-20 13:58:48.825 10271 15663 15805 I python  : [DEBUG  ] Calculating DDTable..
07-20 13:58:48.894  1000  1505  1769 W InputDispatcher: channel 'dc4c829 io.github.lyiqian.bidly/org.kivy.android.PythonActivity (server)' ~ Consumer closed input ch
annel or an error occurred.  events=0x9
07-20 13:58:48.894  1000  1505  1769 E InputDispatcher: channel 'dc4c829 io.github.lyiqian.bidly/org.kivy.android.PythonActivity (server)' ~ Channel is unrecoverably
 broken and will be disposed!
07-20 13:58:48.895  1000  1505  2871 I WindowManager: WIN DEATH: Window{dc4c829 u0 io.github.lyiqian.bidly/org.kivy.android.PythonActivity}
07-20 13:58:48.895  1000   957   957 I sensors-hal: activate_physical_sensor:220, android.sensor.accelerometer/11 en=0
07-20 13:58:48.895  root   919   919 I Zygote  : Process 15663 exited cleanly (1)
07-20 13:58:48.896  1000  1505  2871 W InputDispatcher: Attempted to unregister already unregistered input channel 'dc4c829 io.github.lyiqian.bidly/org.kivy.android.
PythonActivity (server)'
07-20 13:58:48.897  1000  1505  2833 I ActivityManager: Process io.github.lyiqian.bidly (pid 15663) has died: fg  TOP
07-20 13:58:48.899  1000   957   957 I sensors-hal: activate_physical_sensor:233, android.sensor.accelerometer/11 en=0 completed
07-20 13:58:48.900  1000  1505  2833 W ActivityTaskManager: Force removing ActivityRecord{3098a7c u0 io.github.lyiqian.bidly/org.kivy.android.PythonActivity t91}: ap
p died, no saved state
07-20 13:58:48.901  1000  1505  1546 I libprocessgroup: Successfully killed process cgroup uid 10271 pid 15663 in 0ms
07-20 13:58:48.912  1000  1505  1528 W ActivityManager: setHasOverlayUi called on unknown pid: 15663

[Robert]

Most sys libs are simply the Recipe class

ndkrecipe is for an Android ndk-build

cythonrecipe is for cython

A grep for sh.make shows lots of examples, I'd look at the lib* ones.

>grep -R "sh.make"
recipes/ffmpeg/__init__.py:            shprint(sh.make, '-j4', _env=env)
recipes/ffmpeg/__init__.py:            shprint(sh.make, 'install', _env=env)
recipes/freetype/__init__.py:            shprint(sh.make, '-j', str(cpu_count()), _env=env)
recipes/freetype/__init__.py:                shprint(sh.make, 'install', _env=env)
recipes/freetype/__init__.py:                shprint(sh.make, 'distclean', _env=env)
recipes/harfbuzz/__init__.py:            shprint(sh.make, '-j', str(cpu_count()), _env=env)
recipes/hostpython3/__init__.py:            shprint(sh.make, '-j', str(cpu_count()), '-C', build_dir, _env=env)
recipes/icu/__init__.py:                shprint(sh.make, "-j", str(cpu_count()), _env=host_env)
recipes/icu/__init__.py:                shprint(sh.make, "install", _env=host_env)
recipes/icu/__init__.py:                shprint(sh.make, "-j", str(cpu_count()), _env=env)
recipes/icu/__init__.py:                shprint(sh.make, "install", _env=env)
recipes/jpeg/__init__.py:            shprint(sh.make, _env=env)
recipes/lapack/__init__.py:            shprint(sh.make, '-j' + str(cpu_count()), _env=env)
recipes/lapack/__init__.py:            shprint(sh.make, 'install', _env=env)
recipes/leveldb/__init__.py:            shprint(sh.make, '-j' + str(cpu_count()), _env=env)
recipes/libbz2/__init__.py:                sh.make,
recipes/libcurl/__init__.py:            shprint(sh.make, '-j', str(cpu_count()), _env=env)
recipes/libcurl/__init__.py:            shprint(sh.make, 'install', _env=env)
recipes/libexpat/__init__.py:            shprint(sh.make, '-j', str(cpu_count()), _env=env)
recipes/libexpat/__init__.py:            shprint(sh.make, 'install', _env=env)
recipes/libffi/__init__.py:            shprint(sh.make, '-j', str(cpu_count()), 'libffi.la', _env=env)
recipes/libgeos/__init__.py:            shprint(sh.make, '-j' + str(cpu_count()), _env=env)
recipes/libgeos/__init__.py:            shprint(sh.make, 'install', _env=env)
recipes/libiconv/__init__.py:            shprint(sh.make, '-j' + str(cpu_count()), _env=env)
recipes/liblzma/__init__.py:                sh.make, '-j', str(cpu_count()),
recipes/liblzma/__init__.py:            shprint(sh.make, 'install', _env=env)
recipes/libmysqlclient/__init__.py:            shprint(sh.make, _env=env)
recipes/libmysqlclient/__init__.py:    #       # shprint(sh.make, _env=env)
recipes/libmysqlclient/__init__.py:    #       shprint(sh.make, _env=env)
recipes/libogg/__init__.py:            shprint(sh.make, _env=env)
recipes/libpcre/__init__.py:            shprint(sh.make, '-j', str(cpu_count()), _env=env)
recipes/libpq/__init__.py:            shprint(sh.make, 'submake-libpq', _env=env)
recipes/libsecp256k1/__init__.py:            shprint(sh.make, '-j' + str(cpu_count()), _env=env)
recipes/libshine/__init__.py:            shprint(sh.make, '-j', str(cpu_count()), _env=env)
recipes/libshine/__init__.py:            shprint(sh.make, 'install', _env=env)
recipes/libsodium/__init__.py:            shprint(sh.make, '-j', str(cpu_count()), _env=env)
recipes/libvorbis/__init__.py:            shprint(sh.make, _env=env)
recipes/libvpx/__init__.py:            shprint(sh.make, '-j', str(cpu_count()), _env=env)
recipes/libvpx/__init__.py:            shprint(sh.make, 'install', _env=env)
recipes/libwebp/__init__.py:            shprint(sh.make, '-j' + str(cpu_count()), _env=env)
recipes/libwebp/__init__.py:            shprint(sh.make, 'install', _env=env)
recipes/libx264/__init__.py:            shprint(sh.make, '-j', str(cpu_count()), _env=env)
recipes/libx264/__init__.py:            shprint(sh.make, 'install', _env=env)
recipes/libxml2/__init__.py:            shprint(sh.make, "libxml2.la", _env=env)
recipes/libxslt/__init__.py:            shprint(sh.make, "V=1", _env=env)
recipes/libzbar/__init__.py:            shprint(sh.make, '-j' + str(cpu_count()), _env=env)
recipes/libzmq/__init__.py:            shprint(sh.make, _env=env)
recipes/libzmq/__init__.py:            shprint(sh.make, 'install', _env=env)
recipes/openal/__init__.py:            shprint(sh.make, _env=env)
recipes/opencv/__init__.py:            shprint(sh.make, '-j' + str(cpu_count()), 'opencv_python' + python_major)
recipes/openssl/__init__.py:            shprint(sh.make, 'build_libs', _env=env)
recipes/png/__init__.py:            shprint(sh.make, '-j', str(cpu_count()), _env=env)
recipes/protobuf_cpp/__init__.py:                shprint(sh.make, 'libprotobuf.la', '-j'+str(cpu_count()), _env=env)
recipes/python3/__init__.py:                sh.make, 'all', '-j', str(cpu_count()),
recipes/snappy/__init__.py:            shprint(sh.make, _env=env)

[Robert]

If you print  (well, use info('string')) the contents of the dict returned by get_recipe_env() you'll see how the Clang related tools are specified.

[Eason Liu]

That helped; my app is working now.

• use ${CXX) to replace hard-coded path to compiler
• another special handling: had to install OpenMP (libomp.so) in postbuild_arch with

libomp = os.path.join(self.ctx.ndk_dir, 'toolchains/llvm/prebuilt/linux-x86_64/lib64/clang/14.0.6/lib/linux/aarch64/libomp.so')

info("Installing lib manually for OpenMP: %s", libomp)

self.install_libs(arch, libomp)

Thank you!

[Robert]

If you would like to contribute your recipe to the project please do.

Fork https://github.com/kivy/python-for-android

In the local copy add your changes,

Submit a Pull Request