Buildozer APK crashes when used with mysql_connector

[Shoumik Das]

Hi, I have developed a simple app with a single button which, when clicked, updates some data in a MySQL database a remote server. The app is working fine when run from my Ubuntu 18.0.4 desktop but it doesn't work on Android. The apk is getting compiled and installed and the app launches without an issue. When I click on the button to start the database connection, the whole app crashes and the operation is not performed.

Following is my code:

import kivy

from kivy.app import App

from kivy.uix.button import Button

import mysql.connector

from android.permissions import request_permissions, Permission

def upload_files(*args):

mydb = mysql.connector.connect(host="XXX.XXX.XXX.XXX",user="XXXXX",password="XXXXXXXXXXXX",database="XXXX", ssl_ca='ssl-client/ca.pem', ssl_cert='ssl-client/client-cert.pem', ssl_key='ssl-client/client-key.pem')

mycursor = mydb.cursor(buffered=True)

sql1="SELECT * from images"

val1=()

mycursor.execute(sql1,val1)

sl_no=mycursor.rowcount

sl_no=sl_no+1

with open('icons8-address-book-96.png', 'rb') as f:

binaryData = f.read()

sql2="INSERT INTO images(serial_no, file_name, media) VALUES(%s, %s, %s)"

val2=(sl_no,'/var/sftp/sivamediassh/icons8-address-book-96.png',binaryData)

mycursor.execute(sql2,val2)

mydb.commit()

mycursor.close()

mydb.close()

class MyApp(App):

def build(self):

request_permissions([Permission.READ_EXTERNAL_STORAGE,Permission.WRITE_EXTERNAL_STORAGE])

btn1=Button(text='Upload',size_hint_x=0.2,size_hint_y=0.2, pos=(100,100))

btn1.bind(on_release=upload_files)

return btn1

if __name__=='__main__':

MyApp().run()

The Buildozer requirements are as follows:

requirements = python3,kivy,mysql_connector

The Buildozer permissions are as follows:

android.permissions = INTERNET,READ_EXTERNAL_STORAGE,WRITE_EXTERNAL_STORAGE

The logcat error is as follows:

12-19 23:57:02.050 26563 26563 W System.err: Caused by: android.system.ErrnoException: open failed: ENOENT (No such file or directory)

12-19 23:57:02.051 26563 26563 W System.err: at libcore.io.Linux.open(Native Method)

12-19 23:57:02.051 26563 26563 W System.err: at libcore.io.ForwardingOs.open(ForwardingOs.java:167)

12-19 23:57:02.051 26563 26563 W System.err: at libcore.io.BlockGuardOs.open(BlockGuardOs.java:252)

Could you please advise what I am doing wrong?

Thanks

[Darpan Verma]

Try reducing your app just to use is_connected() method of mysql to know if its connecting or not

[Shoumik Das]

Hi. I tried with just a simple is_connected function call. It is not connecting from the Android app. Works fine on desktop, though. On Android, the app crashes when the button is clicked. Any suggestion?

Code

import kivy

from kivy.app import App

from kivy.uix.button import Button

import mysql.connector

from android.permissions import request_permissions, Permission

class MyApp(App):

def upload_files(self, *args):

mydb=mysql.connector.connect(host="XXX.XXX.XXX.XXX", user="XXXXX", password="XXXXXXXXXXXX", database="XXXXX", ssl_ca='ssl-client/ca.pem', ssl_cert='ssl-client/client-cert.pem', ssl_key='ssl-client/client-key.pem')

mycursor = mydb.cursor(buffered=True)

if mydb.is_connected():

print("Upload function:",self)

mycursor.close()

mydb.close()

def build(self):

request_permissions([Permission.READ_EXTERNAL_STORAGE,Permission.WRITE_EXTERNAL_STORAGE])

btn1=Button(text='Upload',size_hint_x=0.2,size_hint_y=0.2, pos=(100,100))

btn1.bind(on_release=self.upload_files)

return btn1

if __name__=='__main__':

MyApp().run()

[Dan The Man]

No expert but wouldn't you have to request INTERNET permission in build(self) like you have in buildozer line? Not sure what android permission required for creating sockets out to internet.

[planckp...@gmail.com]

Little known detail: Run time permissions are only required if the Manifest permission is "Protection level: dangerous"

Check your favorite permission here https://developer.android.com/reference/android/Manifest.permission

And another:  WRITE_EXTERNAL_STORAGE implies READ_EXTERNAL_STORAGE

With regards to a crash ALWAYS look at the the logcat, anything else is guessing.

If the error is not shown by the word Traceback (because Python found the error),

look for the word backtrace or messages about app ending (because Android found the error).

[Dan The Man]

Oww interesting, so according to:

https://android.jlelse.eu/permission-protection-levels-969c9d0a7ebc

he is automatically considered a dangerous app because he is requesting READ/WRITE to external storage, I'm curious if requesting INTERNET INTENT solves his problem.

[Dan The Man]

I guess my suggestion would be do what  planckp said and remove request to external storage. Then you no longer would get blocked as a dangerous app. I think I read somewhere in android you can get a special internal and external storage directory for your app with no permissions needed, I can't remember where i saw that,  but maybe plankp knows how to make the calls to find out what they are.

[Shoumik Das]

Merry Christmas folks!! Sorry for the delayed response. I played around with the permissions a bit but I have managed to get other apps working in the past with similar permissions. I cannot remove the request to external storage because my app needs to read an image from the gallery/file-system and write it to a remote MySQL database as a blob object. In the runtime permissions, I have requested the following:

request_permissions([Permission.WRITE_EXTERNAL_STORAGE,Permission.READ_PHONE_STATE])

In the buildozer permissions, I have used the following:

android.permissions = INTERNET,ACCESS_NETWORK_STATE,CHANGE_NETWORK_STATE,ACCESS_WIFI_STATE,READ_EXTERNAL_STORAGE,WRITE_EXTERNAL_STORAGE,CHANGE_WIFI_STATE,CHANGE_WIFI_MULTICAST_STATE,MANAGE_DOCUMENTS,MANAGE_EXTERNAL_STORAGE

However, I get the following errors in the logcat output:

12-25 13:29:36.444 23373 23826 I python  :  Traceback (most recent call last):

12-25 13:29:36.445 23373 23826 I python  :    File "/home/neo/code/siva-android/blob/.buildozer/android/platform/build-armeabi-v7a/build/python-installs/myblob/mysql/connector/network.py", line 417, in switch_to_ssl

12-25 13:29:36.446 23373 23826 I python  :    File "/home/neo/code/siva-android/blob/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/python3/armeabi-v7a__ndk_target_21/python3/Lib/ssl.py", line 1400, in wrap_socket

12-25 13:29:36.481 23373 23826 I python  :    File "/home/neo/code/siva-android/blob/.buildozer/android/platform/build-armeabi-v7a/build/python-installs/myblob/mysql/connector/network.py", line 426, in switch_to_ssl

12-25 13:29:36.482 23373 23826 I python  :  mysql.connector.errors.InterfaceError: 2055: Lost connection to MySQL server at 'XXX.XXX.XXX.XXX:3306', system error: 2 No such file or directory

12-25 11:57:27.645  9741  9899 W System.err: Caused by: android.system.GaiException: android_getaddrinfo failed: EAI_NODATA (No address associated with hostname)

12-25 13:21:12.990 14150 14228 E fb4a.FbTrafficStats: Caused by: java.io.FileNotFoundException: /proc/net/xt_qtaguid/stats: open failed: EACCES (Permission denied)

I have attached my buildozer.spec file for reference. I am using ssl certificates stored in the sslclient folder in the same location as main.py, to connect to the MySQL server. Can reading those files be a problem? I specified the folder in my spec file:

source.include_patterns = sslclient/*

requirements = python3,kivy,openssl,cffi,ecdsa,cryptography,bcrypt,pynacl,mysql_connector

I have already shared my small Python code earlier in the thread. It just tests for MySQL DB connectivity at the click of a button. No fancy stuff, no global variable, no custom module. Please advise.

Thanks

[Darpan Verma]

Having your files in the same folder as main.py is not accessible. You will need to move your ssl certificate in android app data folder to access it. Complete code was available on this forum itself long back ago, but i can't find it right now.

--
You received this message because you are subscribed to a topic in the Google Groups "Kivy users support" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kivy-users/jQ6bKAw8HUk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kivy-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kivy-users/69574ebb-f716-4e1f-9501-aa90747309ben%40googlegroups.com.

[Shoumik Das]

I think, by default, the code files such as main.py and associated files (modules) are loaded into the app directory. I have tested a different app with an image button and the images were successfully loaded from a directory which was in the same location as main.py.

Moreover, the app directory would vary with each Android device. I can instantiate the paths only after the app has run. In my case, I would need to bundle the SSL certificates within the apk.

@planckp...@gmail.com 

Thanks

You received this message because you are subscribed to the Google Groups "Kivy users support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kivy-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kivy-users/CADjOLJevbvbdgnKPqxX9Df-KSh74RyGQe%2BCTQ8erisbgKC455Q%40mail.gmail.com.

[Darpan Verma]

Leave image.

Try editing a text file in your root directory. My assumption is that you will be unable to edit it. Report, what happens.

To view this discussion on the web visit https://groups.google.com/d/msgid/kivy-users/CAH1Ve%3DRgM2bV%2B2J6%2Bqz4DoGxT8yUhJHBRwfkSLL1MMv8m6qc1Q%40mail.gmail.com.

[Shoumik Das]

In my root directory, I have managed to create a file using "write" in the same location as main.py and also read from it. The app compiled and worked fine.

To view this discussion on the web visit https://groups.google.com/d/msgid/kivy-users/CADjOLJfAKzT3U3cfUSb2WKrqzSO6NsQNP7KLwMsuaw9zGsyusQ%40mail.gmail.com.

[planckp...@gmail.com]

Merry .......

The app directory, which is also the cwd, is writable and readable.

/proc/net/xt_qtaguid/stats

This is probably not writable, doesn't matter what permissions are set. And definitely not writable if android.api =28 or greater.

https://source.android.com/devices/tech/datausage/ebpf-traffic-monitor#legacy-xt_qtaguid-deprecation-process

I have not looked at this so I have no suggestion for a resolution.

In a similar application in the past (without knowing of this issue) I wrote an app that communicates with the my own server

which in turn communicates locally with mysql. The communication between machines is not ssl but that doesn't matter in my case.

I did this because I'm a control freak, and the less magic supplied by others the better ;)

[Shoumik Das]

Hi Rob, thanks for sharing the article but I am am unable to figure out from my logcat output where the issue lies. I am not accessing any file from any restricted storage area. The SSL certificates are stored in a folder in the same location as main.py.

Just trying to connect to a remote MySQL server using a Python connector. This error indicates a connection problem:

12-25 13:29:36.482 23373 23826 I python  :  mysql.connector.errors.InterfaceError: 2055: Lost connection to MySQL server at 'XXX.XXX.XXX.XXX:3306', system error: 2 No such file or directory

Can you suggest if I should try with a lower Android api? This error has been persistent in my other apps also. I am using the defaults mentioned in the buildozer.spec file.

Thanks

To view this discussion on the web visit https://groups.google.com/d/msgid/kivy-users/05cf20f7-f2d4-47c3-b785-3e9de3973702n%40googlegroups.com.

[patil.r...@gmail.com]

can you guys help to with this issue.. here is the link to my post    click here

[planckp...@gmail.com]

How do you know this version of mysql connector works on Android?

Poking around with permissions, or api version does not address that question.

If you don't know, recognize that you don't know, and try something else.

Mysqldb would be worth trying (because it has a recipe) - don't bet the farm, just try a small test case.

I have not used it so I can't answer any questions about it.

[Shoumik Das]

Yes, that is true. I do not know if mysql connector works on Android. I tried to use MySQLDB but it didn't get compiled due to a dependency for ConfigParser. It didn't work even though I installed the dependency. I used a different connector called PyMySQL but that didn't work as well. Same error (searched with Caused by):

12-26 17:32:32.460 20510 20561 E fb4a.FbTrafficStats: Caused by: java.io.FileNotFoundException: /proc/net/xt_qtaguid/stats: open failed: EACCES (Permission denied)

12-26 17:32:32.460 20510 20561 E fb4a.FbTrafficStats: at libcore.io.IoBridge.open(IoBridge.java:496)

12-26 17:32:32.460 20510 20561 E fb4a.FbTrafficStats: at java.io.FileInputStream.<init>(FileInputStream.java:159)

12-26 17:32:32.460 20510 20561 E fb4a.FbTrafficStats: at X.1Pf.A00(:15)

12-26 17:32:32.460 20510 20561 E fb4a.FbTrafficStats: ... 8 more

12-26 17:32:32.460 20510 20561 E fb4a.FbTrafficStats: Caused by: android.system.ErrnoException: open failed: EACCES (Permission denied)

12-26 17:32:32.460 20510 20561 E fb4a.FbTrafficStats: at libcore.io.Linux.open(Native Method)

If I search with with the keyword, Traceback, this is what I get:

12-26 21:22:15.034 22174 22621 I python  :    File "/home/neo/code/siva-android/blob/v2/.buildozer/android/platform/build-armeabi-v7a/build/other_builds/python3/armeabi-v7a__ndk_target_21/python3/Lib/ssl.py", line 745, in create_default_context

12-26 21:22:15.035 22174 22621 I python  :  FileNotFoundError: [Errno 2] No such file or directory

12-26 21:22:15.035 22174 22621 I python  : Python for android ended.

I don't know what file the system is looking for. I hate  to say this as a programmer, but, almost giving up hope. :-(

Thanks

[Shoumik Das]

I were to add this to the manifest, where and what should I update?

<manifest ... >

    <application android:requestLegacyExternalStorage="true" ... > 

     ...

    </application>

</manifest>

Thanks

[planckp...@gmail.com]

As I tried to say before, the log tells us this file is the issue : /proc/net/xt_qtaguid/stats

And it is ssh that is trying to access.

No messing with Android permissions is going to address an issue in /proc

I do not know anything about ssh on Android.

I still expect that mysqldb is the way to go, though I have no specific usage instructions.

[Dan The Man]

I would think aiomysql would be the way to go, so connection to mysql is asyncronous along with waiting for results from query.

I am just unsure how you could bind that to a button, kivy doesn't seem to allow us to provide "await dothis()" abilities even though its clearly using asyncio backend.

I tried today with:

import asyncio
import aiomysql

if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(MainApp().async_run(async_lib='asyncio'))
loop.close()

then tried to await custom callback, on button with:

on_release: app.rr.mysql_query('test')

all I get is 

RuntimeWarning: coroutine 'Mysql.mysql_query' was never awaited

   on_release: app.rr.mysql_query('test')

so if i changed that to:

on_release: app.test()

and do a :

async def test(self):
    rr=await Mysql.mysql_query() #custom aiomysql class returning results from query
    print("DO I make it here?")

RuntimeWarning: coroutine 'MainApp.test' was never awaited

So seems kivy hogs the eventloop, and only way around it I can see is to toss kivy itself in an eventloop with your own function. This poses a big problem because then you'd need to sit there

like a tcp server and wait for some app.root.ids.blah.text to change in order to execute events, instead of immediately interacting with event bind button. Not sure if anyone has experienced this.

[Dan The Man]

Here is the custom mysql class I used for testing aiomysql
I think this would be alot better to get working, as it would be truely asyncronous.

[planckp...@gmail.com]

Yes, I/O will block the Kivy event loop.

On a desktop this is not significant because the OS is multitasking and the block is transparent.

(Though it is perhaps poor style ?)

Android is not multitasking in the desktop sense.

Android will complain about a non responsive app (in the logcat) and exit the app.

So I/O activity must be off the UI Thread.

Some I/O packages inherit from Thread, so this is not something we have to think about, a good clue is a callback.

But you never really know how much is in a Thread, so just put the I/O in a Python Thread.

Remember to be Thread safe with the callback result.

Using Clock.schedule_once() is a safe way to synchronize with the Kivy event loop if you need to be safe.

[Dan The Man]

It's a good point, I mean if something was taking a lot of cpu time, I believe python would suffer because of the GIL. Only way around it I have seen is to start a subprocess so a new python process gets a new core and doesn't block the parent.

In this mysql case though, that wouldn't be an issue executing in main thread. TCP connections are slow, so I would return control back to eventloop connecting, and also while its reading results from the mysql server.

I'm surprised I cannot execute an async function as a callback from a button here.

[Dan The Man]

I mean way I have done it, not the creator of this thread, he'll probably crash his app way he is doing it blocking like that :)

[Shoumik Das]

Thank you for the suggestions. I shall try with asyncio and aiomysql.

[Darpan Verma]

Sqlite comes already bundled with kivy app for android if you built using buildozer. This may mean that it is perfectly supported. Give a try to it.

--
You received this message because you are subscribed to a topic in the Google Groups "Kivy users support" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kivy-users/jQ6bKAw8HUk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kivy-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kivy-users/93817c9c-d26e-4655-b7d4-231eae0750cfn%40googlegroups.com.

[Dan The Man]

For playstore I would think connecting to a remote mysql database with a fastapi or using a local sqlite instance would probably work the best, but there are reasons someone would want to use a pure mysql connection I'm sure, I could think of a couple, like a private app accessing your mysql server over your vpn for your own purposes. But yeah I don't think I'd be wanting to hand out my database username and password in an apk to anyone with root access to their phone, they could decompile and get it easily lol. 

I'm interested if you can get that aiomysql function working with kivy, I'm having problems binding async functions to buttons.

[Dan The Man]

OK I have a working example with aiomysql, I had help on this to make it work. Please remember mysql function I quickly coded should NOT be used in production. At least toss in proper error handling in that mysql function for connects, getting results etc if your gonna use it, it was just a quick hack for proof of concept nothing more, I would add error handling to it before using it for real.

Anyways here it is, let me know if it compiles on buildozer or not :)

[Dan The Man]

Ok i just tested building it on my android phone with buildozer:

requirements = python3==3.8.6,hostpython3==3.8.6,kivy==2.0.0,https://github.com/kivymd/KivyMD/archive/master.zip,aiomysql,pymysql

works prefectly.....enjoy...

[Shoumik Das]

Thanks for sharing the details. However, in my case, I am using ssl certificates to connect to a remote mysql server. I have not tried with a non-ssl user yet.

[Darpan Verma]

You need to use relative path even if you included in your app.

For ssl key, use './<folder name>/<file name>'

If problem is for accessing files, then this should solve the problem

To view this discussion on the web visit https://groups.google.com/d/msgid/kivy-users/237a8700-40cc-4c2c-b07f-0a1681224f94n%40googlegroups.com.

[Darpan Verma]

Another suggestion is that

Beware of fake package names, there are two packages namely mysql-connector-python and another named mysql-connector.

The former one is purely pythonic thus recommended for kivy apps. If you installed latter one, then uninstall to avoid name conflict.