List contents of Download, Books (etc.) directories

[Robert Flatt]

SDK 29,  How do I list the contents of the shared Download, Books (etc.) directories?

The question is about API, probably not about permissions.

Android Studio tells me the enclosing folder is called /storage/self/primary

My friend Google points me to these, but they are depreciated:

Environment.getExternalStorageDirectory()
Environment.getExternalStoragePublicDirectory()

This returns True , which I understand means scoped storage is unavailable (even if I understood how to use it)

Environment.isExternalStorageLegacy()

This gives me an app specific shared directory, which is not what I'm looking for

Context.getExternalFilesDir()

And of course Kivy gives me an app specific private directory, which is not what I'm looking for

App.user_data_dir

Android Studio tells me the directory is called (for example) '/storage/self/primary/Download'

But listdir() retuns a permission error even though READ_EXTERNAL_STORAGE is specified

Anyway specifing a (fake) hard path like this is a recipe for disaster in Android.

The answer might be MediaStore, lots of documentation, but I can't find an example that is simple enough for me to understand.

Anybody have a small example?

[ZenCODE]

With later versions of the Android SDK, you have to request permission dynamically: setting READ_EXTERNAL_STORAGE in your manifest is not enough.

https://python-for-android.readthedocs.io/en/latest/apis/

Once you have permission, the normal Python file operation all work e.g. listdir.

[Robert Flatt]

Thank you. The docs are accurate if a little terse, documenting the callback and the reason for it would be an excellent enhancement.

Can you help me see what I am doing wrong, with primary_external_storage_path() I always get:

[Errno 13] Permission denied:'/storage/emulated/0'

I have

android.permissions = INTERNET, READ_EXTERNAL_STORAGE

The behavior seems correct, the dialog box opens the first time the app runs. I tap Allow.

Permission is granted but listdir fails [Errno 13] on the first and subsiquent (no dialog) tries.

from kivy.app import App
from kivy.uix.label import Label
from os import listdir
from android.permissions import request_permissions,Permission,check_permission
from android.storage import app_storage_path, primary_external_storage_path

# Build for Android with: orientation = landscape

class Demo(App):
    def list_dir(self):
        self.msg += 'Permission Granted is '
        self.msg += str(check_permission(Permission.READ_EXTERNAL_STORAGE))+'\n'
        location = str(primary_external_storage_path())
        self.msg += "List Dir " + location + '\n'
        try:
            for d in listdir(location):
                self.msg+= d + '\n'
        except Exception as e:
            self.msg += str(e)
   
    def callback(self,permissions,grants):
        self.msg += 'Response from Dialog Box is:\n'
        for p,g in zip(permissions,grants):
            self.msg += str(p) + ' is ' + str(g) + '\n'
        self.list_dir()
        self.label.text = self.msg
       
    def build(self):
        self.msg = ""
        self.label = Label()
        if check_permission(Permission.READ_EXTERNAL_STORAGE):
            self.msg += 'Pre Approved\n'
            self.list_dir()
        else:
            self.msg += 'First Approval\n'
            request_permissions([Permission.READ_EXTERNAL_STORAGE],
                                self.callback)
        self.label.text = self.msg
        return self.label

if __name__ == '__main__':
    Demo().run()

This is what I see:

[ZenCODE]

`listdir` sometimes requires executable permissions. As test, try requesting write permissions?

Permission.WRITE_EXTERNAL_STORAGE

[Robert Flatt]

WRITE_EXTERNAL_STORAGE  (in .spec and .py) did not change behavior (except dialog text).

I tried moving the logic to on_start() - no change

I tried listing the directory using Java File, in place of os.listdir(), get a File, confirmed the path, that it is a Directory, and is not Hidden

But list() which should return a list of strings, returns <class 'None Type'>

Which is unexpected!

            File = autoclass('java.io.File')
            directory = File(location)
            self.msg += str(type(directory)) + '\n'
            self.msg += str(directory.getAbsolutePath()) + '\n'
            self.msg += str(directory.isDirectory()) + '\n'
            self.msg += str(directory.isHidden()) + '\n'
            self.msg += str(type(directory.list())) + '\n'

[ZenCODE]

That's a good test, and just confirms that it's a permission issue, not a language one. Strange, I've used the request permissions as above and it worked for me no problems. So, if you manually add the permissions, does it work? Also, what if you use Clock.schedule and do the listdir after the callback has returned? Just to ensure it's not a timing issue...

[Robert Flatt]

I suspect the issue is API 29 is preventing direct access.

See this page https://developer.android.com/training/data-storage

Which means for Download, the answer is here

https://developer.android.com/training/data-storage/shared/documents-files#java

But at this point there are too many things that I don't understand to be certain.

[ZenCODE]

Wow. They really do keep moving the goalposts. But it does look like the "Scoped storage" is the problem. Also, files are accessible through the MediaStore API? Try using a SDK < 29 to confirm. 27 is currently working for me.

[Robert Flatt]

Same failure for me on SDK 27 as SDK 29.

In both cases I am testing on Android 10, and as I read the docs that might be the issue.

[ZenCODE]

Must be... I think you will need to look at the docs you pointed to, and use the MediaStore API...;-(

[Robert Flatt]

Yep, though today my interest is Download directory so I tried Storage Access Framework first.

The architecture for Media Store will be similar.

It works but I get a system file Picker for the first of two user permission confirmations :(
I don't really want the Picker..... I probably need a different Intent... so still some work to do.
If I don't select a folder in the Picker then the intent gives the root contents, which is (automagically) the Download directory

(note in the code below, nowhere is external storage or Download specified)

No READ_EXTERNAL_STORAGE permission is required, check_permission() always returns True. (only tried on Android 10)

Proof of concept, which sort of answers my original question:

from kivy.app import App
from kivy.uix.label import Label

import os.path
from android import activity, mActivity
from jnius import autoclass

Intent = autoclass('android.content.Intent')
DocumentsContract = autoclass('android.provider.DocumentsContract')
Document = autoclass('android.provider.DocumentsContract$Document')

class Demo(App):
    REQUEST_CODE = 42 # unique request ID
   
    def set_intent(self):
        intent = Intent(Intent.ACTION_OPEN_DOCUMENT_TREE)
        intent.addFlags(Intent.FLAG_GRANT_READ_URI_PERMISSION)
        mActivity.startActivityForResult(intent, self.REQUEST_CODE)        

    def intent_callback(self, requestCode, resultCode, intent):
        if requestCode == self.REQUEST_CODE:
            msg = ""
            try:
                root_uri = intent.getData()
                root_id = DocumentsContract.getTreeDocumentId(root_uri)
                children = DocumentsContract.buildChildDocumentsUriUsingTree(root_uri,root_id)
                contentResolver = mActivity.getContentResolver();
                info = [Document.COLUMN_DISPLAY_NAME]
                c = contentResolver.query(children, info, None, None, None);

                while c.moveToNext():
                    name = str(c.getString(0))
                    if 'rce_plugin' not in name:  # junk from Kindle App
                        msg += name + '\n'
                c.close()
            except Exception as e:
                msg += str(e) + '\n'
            self.label.text+=msg

    def on_start(self):
        self.set_intent()

    def build(self):
        activity.bind(on_activity_result=self.intent_callback)
        self.label = Label()

        return self.label

if __name__ == '__main__':
    Demo().run()