deleteFile and deleteFolder issues

[Pascal Georges]

Hello,

I have an issue with deleteFile performance when working in a directory with about 800 files : a file deletion takes between 2 to 10 seconds (imagine deleting 8 files takes one minute !). Using any Java app it is instantly performed. So I had to write a service to manage those asynchronously. The same thing for ListFolder, which takes quite a long time. So I am considering writing a plugin for those 2 syscalls.

Isn't there a performance issue in file management code in DS ?

Note also that app.DeleteFolder does not behave like the doc says at https://symdstools.github.io/Docs/docs/app/DeleteFolder.htm : when deleting a non empty folder, it will simply do nothing when the doc states 'It recursively removes all files and folders of the given folder and finally deletes it.'

Pascal G.

 

[Steve Garman]

Yes, it's all the calls across the bridge

Every so often someone complains and every couple of years it is suggested that someone should write a java plugin

I'm not aware that anything has happened though

[Pascal Georges]

Hi,

I started to write a plugin to get better performance for I/O calls and here is my result for app.ListFolder (which I found really slow, as other calls). For a directory with 689 files :

10 538 ms for app.ListFolder

204 ms for my plugin ListFolder

 => 50x faster !

So I don't understand why DS is so slow when there are some faster solutions available. I suspect some other code is running in DS that may be a bit underoptimized. 

Pascal G.

[Dave]

Hi Pascal,

The DS folder listing method is quite complex as it has many options and supports virtual file paths, SAF, sorting and listing of internal assets etc.

Having said that.... I just noticed that it is checking for SAF folders every time it recurses down into a sub-folder, so that could be partly why it is so slow.  I think we can just check the top level folder for being outside of scoped area rather than checking each sub-folder.

If you can send me a snippet of test js code that lists a folder recursively and shows the time taken in the logs, that would help me greatly in improving the performance (my time is very limited)

Thanks

[Pascal Georges]

Hi Dave,

Thank you for looking at this.

Here is the code I use ('plug' == 'plugin') :

// ================= DS code for the bench ===================

function bench(dir) {
       console.time("bench plug");
    let result = JSON.parse(plug.ListFolder(dir));
    console.log("plug got " + result.length);
    console.timeEnd("bench plug");

    console.time("bench app");
    let la = app.ListFolder(dir);
    console.log("app got " + la.length);
    console.timeEnd("bench app");
}

// ================= Java code for ListFolder ===================

    private String ListFolder(Bundle b) throws Exception {
        Log.d(TAG, "Got ListFolder");

        String dir = b.getString("p1");
        File directory = new File(dir);
        File[] files = directory.listFiles();
        ArrayList<String> paths = new ArrayList();

        for (File f : files) {
            paths.add(f.getPath());
        }

        JSONArray jsonArray = new JSONArray(paths);
        String s = jsonArray.toString();
        return s;
    }

// ================ Bench output ===============================

plug got 673
bench plug: 115.82275390625 ms
<cut irrelevant data>
app got 673
bench app: 10430.629150390625 ms

=> 90x faster despite the JSON.parse call.

Greetings

Pascal G.