ModuleNotFoundError when building pysqlcipher3 with recipe

[Jimminer]

Hello, I am trying to create a recipe that builds pysqlcipher3 (my first time making a recipe)

The module needs an external library to build against called sqlcipher which I have installed on my Linux machine. When building "by hand" using python3 setup.py build, the module gets built successfully including the .so library and everything works.

The app compiles into an .apk but when installed on an Android phone, it crashes saying ModuleNotFoundError: No module named 'pysqlcipher3'

My recipe is the following:

from pythonforandroid.recipe import CompiledComponentsPythonRecipe

class pysqlcipher3Recipe(CompiledComponentsPythonRecipe):
    name = "pysqlcipher3"
    version = "1.0.4"
    url = "https://pypi.python.org/packages/source/p/pysqlcipher3/pysqlcipher3-{version}.tar.gz"
    call_hostpython_via_targetpython = False

    build_cmd = "build"

    depends = ["setuptools"]
    patches = []

recipe = pysqlcipher3Recipe()

Is there anything I'm doing (or perhaps not doing) that is causing this issue?

Thanks!

[Robert]

Recipes are custom, help is unavailable.

> t crashes saying ModuleNotFoundError: No module named 'pysqlcipher3'

This probably means the directory was not installed in site-packages.

Check the build log for error messages.

[Jimminer]

Thanks for the suggestion!

I checked the build log more carefully and it seems that the recipe does not have access to the library I mentioned.

The library is installed in /usr/include/sqlcipher/

Is there an option for the recipe to include that directory?

This is the error:

[DEBUG]: src/python3/connection.h:33:10: fatal error: 'sqlcipher/sqlite3.h' file not

[DEBUG]: found

[DEBUG]: #include "sqlcipher/sqlite3.h"

[DEBUG]: ^~~~~~~~~~~~~~~~~~~~~

Also, I completely understand that recipes don't receive support. I'm just asking and maybe someone that knows what I'm doing wrong can throw me a bone.

[Robert]

p4a site-packages do not use the locally installed package, it is always downloaded.

So it is not finding the downloaded ".h"  which is odd because the include path would be expected to be under the cwd

So something about cwd ? 

[Jimminer]

Sorry, I did not explain it correctly.

In order for the module to be built, it needs an external library called sqlcipher.

The library does not come with the module. It is my job to have it installed (which I do).

Once installed, the library goes to /usr/include/sqlcipher and the module tries to include sqlite3.h (located at /usr/include/sqlcipher/sqlite3.h)

Normally, the C compiler is aware of anything placed in /usr/include and can include it.

From what I understand, when building a recipe, it runs in an isolated environment and that obviously would be the cause of this problem.

Is there any way to make it aware of the path /usr/include/sqlcipher ?

[Robert]

Yes, it is possible to modify an include path. I expect instructions don't exist - look at the existing recipes. Yes I know that is only marginally helpful.

Hmm I'm confused about the big picture as I understand it:

pysqlcipher is a Python (or maybe Cython?) wrapper around sqlcypher which is written in C.

I assume that means the app will need sqlcypher compiled with Clang for ARM, compiled for and installed on an Android device - but I don't infer this from the post above.

Then pysqlcipher build will look for header files from the sqlcipher build directory.

Perhaps I just don't understand, recipe support is close to impossible (without actually doing for you - which I don't have time for).

[Jimminer]

Since my last message I have been trying to build the sqlcipher library using a recipe.

Unfortunately upon running the make command, the process stops after some warnings and errors that I cannot understand.

My best guess is that it is not made to be compiled for Android architectures.

This SO answer seems to indicate something similar to what I'm thinking and points to a different library (maybe I should try building that one)

I will keep trying and maybe I will have a desirable outcome. Otherwise I will have to settle with normal sqlite3 which offers no encryption :(

[Robert]

Building sqlcipher with Clang/ARM should be doable.

But for perspective, the only released recipe I wrote (it was non-trivial) took 3 tries spaced over many months.

For each try I had to unlearn my assumptions about one or more things.

[Jimminer]

To build with Clang the app needs a Cmake file no? (I'm not experienced with anything but gcc and make)

[Robert]

Clang can be called by CMake, but Clang does not require it is called by CMake.

Use whatever build system the package setup uses, its hard enough to port somebody else's build instructions to another environment, rewriting them is close to unthinkable.

Clang options are mostly (but not always) backwards compatible with gcc, p4a will find the Android Clang instance and basic options.

Find an existing recipe that does a similar build, use that as a template.

[Jimminer]

Seems interesting. I will probably try that tomorrow.

Thank you very much for your assistance so far!!

[Jimminer]

Well, thanks for the suggestion about Clang. You really saved me!

Adding env["CC"] = "clang" seems to have fixed all previous errors!

sqlcipher compiles normally into a shared object!

Now I just have to compile pysqlcipher3 and I'll be done.

Thank you Robert for your time!!

[Moqing Yu]

You should build two recipes, one for sqlcipher, and other for pysqlcipher,  recipe for sqlciper refer to a pure c module like libffi, and in recipe for pysqlcipher, you need add CFLAGS and LDFLAGS to refer to the include and lib folder under the sqlcipher’s build folder

Please refer to my recipes(portaudio and pyautdio) at  https://github.com/yumoqing/recipes/andorid

Hope it will help

[Jimminer]

I was trying to do that but I was having difficulty building sqlcipher for ARM and as a result, pysqlcipher would not compile with an error "Skipping incompatible library" (referring to sqlcipher)

I had a look at your recipes and I think that they will help me a lot.

Thank you very much!