why the new API for writing to external SD card is a pain for developers
440 views
Skip to first unread message
stere...@yahoo.com
Apr 9, 2016, 5:17:45 PM4/9/16
to Tasker
In defence of the developer of Tasker, I should point out adding support for external SD cards in apps is a non-trivial task.
Google took away SD card support in Kit Kat - there was a backlash by users and developers (because apps were blamed for this - notably file manager apps).
Eventually some type of external SD card support was reinstated in Lolipop.
This was enough to cause fragmentation in how apps were dealing with external SD cards. It adds to the complexity for the developer (and the support issues) - if the landscape of devices and android versions change (adding to the already thousands of android device variants).
Adding to injury, the Lollippop support that was added required use of an alternative API and a different way of doing things. From the user end it includes a screen which is shown (which itself requires handholding for naive user to navigate).
All in all, it has created a situation where handling of the external SD card has become toxic for an app developer (some will say that was the whole purpose of meddling in the SD card issue by Google, and reinstating has absolved Google, but retained the disruption).
In Marshmallow, the situation has been complicated further - this time for the user - as they are now offered a "Portable" format (which is same as Lollipop), and an "Internal" format - which is not what naive users think it is - and it is not what expert users think it should do. It takes a good two paragraphs and some back and for each user to understand exactly what it does.
Most of the early media reports on this feature got it completely wrong (i.e. the complexity and side-effects it entailed).
This reddit post was one of the first to explain the pitfalls of Internal mode in Marshmallow:
https://www.reddit.com/r/Android/comments/3oz7eu/guidelines_for_marshmallow_users_formatting/
Guidelines for Marshmallow users - formatting options for external SD cards (Portable vs. Internal modes)
Coming back to the alternative API that was released for Lollipop - this is an API which few apps have bothered to implement. Some of the better file manager apps like ES File Explorer and Total Commander have included it - because it falls in their area of operation (also users penalized them heavily for something that was in effect Google's fault in the early days after the changes).
This is what Pent was referring to in his post (the link he refers to was started me me in another google thread):
https://groups.google.com/d/msg/tasker/zuY21R_bvyU/ruN2HYPiAgAJ
Quote:
Here's a nice link explaining the situation:
https://groups.google.com/forum/#!topic/eyes-free/rspqjlmu_ss
As I understand it, if on lollipop or 'portable' mode on marshmallow are used, access is required via a special
API. I've looked at this API (DocumentFile) and it's going to require rewriting huge chunks of code
used for the file functions, plus some awkwardness with the user pre-accepting file operations
on external SD (and I don't know how long that preacceptance stays valid, potentially making it
useless for later automation)
Summary, I'm not going to be doing this in the near future, if more mainstream devices start to get
SD I may reconsider.
End of Quote.
The thread where Pent posted this is now closed, so I am posting observations here.
The user-level screen that is used to get user's permission (once only) is certainly quite non-professional (it is an Android screen). That could certainly be improved by Google.
However, it seems once permission is granted by the user, it will persist (until one does Settings - Apps - Clear Data for the app that is using the alternative API).
Using the alternative API (for the developer) requires changing the whole way you access files. This is a major pain which Google could have done something about alleviating.
There are some libraries which have appeared for this which may make it easier, but most app developers have even now not bothered.
We make this audio recorder/field recorder/call recorder app which also has Tasker support:
https://www.reddit.com/r/tasker/comments/3z1rgh/dev_android_mp3_voice_memo_field_recorder_call/
[DEV] [ANDROID] MP3 Voice memo + field recorder + call recorder with Tasker support (Record, Pause, Stop, Play, Folder, Settings)
however, we have not bothered to add external SD card support (although it would be nice). We may add it eventually if we are comfortable doing it. The reason is it will require changes over a wide range of code, and developers are really wary about such wide ranging changes, as there is great risk with large changes if you have millions of users. Making a stable app is like making a commercial aeroplane. You are very careful about making big changes. This is one of the disadvantages for mature apps - less of a problem for new apps (who have no established base which needs to be supported).
I should mention that there IS a way to write to the SD card from all app - and this was mentioned by a posted in that thread:
https://groups.google.com/d/msg/tasker/zuY21R_bvyU/4lma8GvMAwAJ
Quote:
Tested and working. I found a solution by creating a similar folder to the one described here.
http://www.titaniumtrack.com/kb/titanium-backup-kb/titanium-backup-troubleshooting.html#tb-hangs-while-making-backups
The very last entry describes how to create a folder that has write permissions for a specific app and should persist after a factory reset. Create the following folder in your external sd card.
android/data/net.dinglisch.
Google took away SD card support in Kit Kat - there was a backlash by users and developers (because apps were blamed for this - notably file manager apps).
Eventually some type of external SD card support was reinstated in Lolipop.
This was enough to cause fragmentation in how apps were dealing with external SD cards. It adds to the complexity for the developer (and the support issues) - if the landscape of devices and android versions change (adding to the already thousands of android device variants).
Adding to injury, the Lollippop support that was added required use of an alternative API and a different way of doing things. From the user end it includes a screen which is shown (which itself requires handholding for naive user to navigate).
All in all, it has created a situation where handling of the external SD card has become toxic for an app developer (some will say that was the whole purpose of meddling in the SD card issue by Google, and reinstating has absolved Google, but retained the disruption).
In Marshmallow, the situation has been complicated further - this time for the user - as they are now offered a "Portable" format (which is same as Lollipop), and an "Internal" format - which is not what naive users think it is - and it is not what expert users think it should do. It takes a good two paragraphs and some back and for each user to understand exactly what it does.
Most of the early media reports on this feature got it completely wrong (i.e. the complexity and side-effects it entailed).
This reddit post was one of the first to explain the pitfalls of Internal mode in Marshmallow:
https://www.reddit.com/r/Android/comments/3oz7eu/guidelines_for_marshmallow_users_formatting/
Guidelines for Marshmallow users - formatting options for external SD cards (Portable vs. Internal modes)
Coming back to the alternative API that was released for Lollipop - this is an API which few apps have bothered to implement. Some of the better file manager apps like ES File Explorer and Total Commander have included it - because it falls in their area of operation (also users penalized them heavily for something that was in effect Google's fault in the early days after the changes).
This is what Pent was referring to in his post (the link he refers to was started me me in another google thread):
https://groups.google.com/d/msg/tasker/zuY21R_bvyU/ruN2HYPiAgAJ
Quote:
Here's a nice link explaining the situation:
https://groups.google.com/forum/#!topic/eyes-free/rspqjlmu_ss
As I understand it, if on lollipop or 'portable' mode on marshmallow are used, access is required via a special
API. I've looked at this API (DocumentFile) and it's going to require rewriting huge chunks of code
used for the file functions, plus some awkwardness with the user pre-accepting file operations
on external SD (and I don't know how long that preacceptance stays valid, potentially making it
useless for later automation)
Summary, I'm not going to be doing this in the near future, if more mainstream devices start to get
SD I may reconsider.
End of Quote.
The thread where Pent posted this is now closed, so I am posting observations here.
The user-level screen that is used to get user's permission (once only) is certainly quite non-professional (it is an Android screen). That could certainly be improved by Google.
However, it seems once permission is granted by the user, it will persist (until one does Settings - Apps - Clear Data for the app that is using the alternative API).
Using the alternative API (for the developer) requires changing the whole way you access files. This is a major pain which Google could have done something about alleviating.
There are some libraries which have appeared for this which may make it easier, but most app developers have even now not bothered.
We make this audio recorder/field recorder/call recorder app which also has Tasker support:
https://www.reddit.com/r/tasker/comments/3z1rgh/dev_android_mp3_voice_memo_field_recorder_call/
[DEV] [ANDROID] MP3 Voice memo + field recorder + call recorder with Tasker support (Record, Pause, Stop, Play, Folder, Settings)
however, we have not bothered to add external SD card support (although it would be nice). We may add it eventually if we are comfortable doing it. The reason is it will require changes over a wide range of code, and developers are really wary about such wide ranging changes, as there is great risk with large changes if you have millions of users. Making a stable app is like making a commercial aeroplane. You are very careful about making big changes. This is one of the disadvantages for mature apps - less of a problem for new apps (who have no established base which needs to be supported).
I should mention that there IS a way to write to the SD card from all app - and this was mentioned by a posted in that thread:
https://groups.google.com/d/msg/tasker/zuY21R_bvyU/4lma8GvMAwAJ
Quote:
Tested and working. I found a solution by creating a similar folder to the one described here.
http://www.titaniumtrack.com/kb/titanium-backup-kb/titanium-backup-troubleshooting.html#tb-hangs-while-making-backups
The very last entry describes how to create a folder that has write permissions for a specific app and should persist after a factory reset. Create the following folder in your external sd card.
android/data/net.dinglisch.
android.taskerm
Copy any files from your old Tasker file on your external card to this file. When you attempt to do a backup in Tasker direct the backup to the following on your sd card and you should have a proper backup.
android/data/net.dinglisch.android.taskerm/configs/user
End of Quote.
This is a folder that Android does allow all apps to write - on the external SD card - it is of the form:
Android/data/package-name-here/files
HOWEVER, there is a major caveat for this - this is basically considered a TEMPORARY location by Android. So while you CAN choose sub-folders within this app-specific folder - an all apps (even our app above) will be able to write to it (you can create that folder using Total Commander etc.).
But Android will remove this app-specific folder if your app happens to get uninstalled.
So only users who are aware of the risks should bother using this app-specific folder on the external SD card.
But if you only need it for some temporary space etc. - then it should a good place to use - and should work with Tasker also. You just need to make sure that location has been created (use Total Commander, ES File Explorer - once these have been granted permission by you, you should be able to create that folder in Android/data). Or Tasker etc. could create that folder in advance to save the user the trouble of having to create it.
This should cover most of the issues concerning external SD card use on Android - at least as an introduction for naive users.
Copy any files from your old Tasker file on your external card to this file. When you attempt to do a backup in Tasker direct the backup to the following on your sd card and you should have a proper backup.
android/data/net.dinglisch.android.taskerm/configs/user
End of Quote.
This is a folder that Android does allow all apps to write - on the external SD card - it is of the form:
Android/data/package-name-here/files
HOWEVER, there is a major caveat for this - this is basically considered a TEMPORARY location by Android. So while you CAN choose sub-folders within this app-specific folder - an all apps (even our app above) will be able to write to it (you can create that folder using Total Commander etc.).
But Android will remove this app-specific folder if your app happens to get uninstalled.
So only users who are aware of the risks should bother using this app-specific folder on the external SD card.
But if you only need it for some temporary space etc. - then it should a good place to use - and should work with Tasker also. You just need to make sure that location has been created (use Total Commander, ES File Explorer - once these have been granted permission by you, you should be able to create that folder in Android/data). Or Tasker etc. could create that folder in advance to save the user the trouble of having to create it.
This should cover most of the issues concerning external SD card use on Android - at least as an introduction for naive users.
stere...@yahoo.com
Apr 9, 2016, 5:42:40 PM4/9/16
to Tasker
I should add that creating and writing to that "temporary" (since it will be removed by Android if app is uninstalled) can be done by a legacy app without issue i.e. using the old ways of writing to file etc. - so no change needed.
-stereomatch
-stereomatch
Pent
Apr 10, 2016, 3:13:03 AM4/10/16
to Tasker
Thanks, I was expecting another my-file-manager-can-do-it-so-why-cant-tasker complaint thread :-)
Pent
Pent
stere...@yahoo.com
Apr 10, 2016, 6:42:05 AM4/10/16
to Tasker
With our app we have not bothered either - although we would like to - perhaps will get to it when have time.
But surprisingly users are not clamoring for it as much - and I think it is the general climate which has been created - very few apps (except for the file manager apps whose mandate it is to provide access to storage in the first place). As a result I think even concerned users now have gotten jaded and don't bother asking for this support. Or if they do, they do it weakly.
So in that sense, while "support" for external SD cards has been reinstated, it has provided sufficient discontinuities in the android timeline for it to be unpalatable to support it (from developer side).
The damaging effect of fragmentation is that a developer who has gone to the effort of implementing the feature cannot really claim it works everywhere (since it will not work for Kit Kat or may not with the user interface being so confusing for end user - the permission screen which Android shows makes no sense - you click on top right - then something appears - and you then click on external storage etc.).
As a result a developer cannot really ADVERTISE that feature as something the app does.
I feel this aspect alone is sufficient for developers to see implementing of this feature as a "no fun" endeavor. They will be implementing a feature which will not work well, and will perhaps give them MORE support calls because expert users even will have issues using it.
However, I suspect as a greater percentage of users shift to Marshmallow and Lollipop (and Kit Kat user base becomes 5%) - at that point we might start to see more support for it (however, will by that time the majority of users have forgotten that external SD cards could be used this way).
-stereomatch
But surprisingly users are not clamoring for it as much - and I think it is the general climate which has been created - very few apps (except for the file manager apps whose mandate it is to provide access to storage in the first place). As a result I think even concerned users now have gotten jaded and don't bother asking for this support. Or if they do, they do it weakly.
So in that sense, while "support" for external SD cards has been reinstated, it has provided sufficient discontinuities in the android timeline for it to be unpalatable to support it (from developer side).
The damaging effect of fragmentation is that a developer who has gone to the effort of implementing the feature cannot really claim it works everywhere (since it will not work for Kit Kat or may not with the user interface being so confusing for end user - the permission screen which Android shows makes no sense - you click on top right - then something appears - and you then click on external storage etc.).
As a result a developer cannot really ADVERTISE that feature as something the app does.
I feel this aspect alone is sufficient for developers to see implementing of this feature as a "no fun" endeavor. They will be implementing a feature which will not work well, and will perhaps give them MORE support calls because expert users even will have issues using it.
However, I suspect as a greater percentage of users shift to Marshmallow and Lollipop (and Kit Kat user base becomes 5%) - at that point we might start to see more support for it (however, will by that time the majority of users have forgotten that external SD cards could be used this way).
-stereomatch
Alex Alex-ma
Apr 17, 2016, 12:02:49 AM4/17/16
to Tasker
Hey!
found this and the other thread (refund) about external SD-Card yesterday while i try copy files with tasker! There are many people with the same problem outthere! Tasker is a great app .. so i decide to find anonther solution for the external sd-card and dont waste my time :D ----> Here is it: http://forum.xda-developers.com/android/apps-games/apps2sd-partition-sd-card-link-apps-to-t3122919
Splitt external SD-Card in 2 Partitons, if u want you can format the second Partition as an ext4-device and link and/or bind/mount folders from external card into the filesystem of the internal storage! Doesnt test it with taskers actions yet,,, but an ordanary oem-stock-filemanger with just access to the internal storage havnt got any Problems. Will test it tommorow or monday with tasker.. but now it is 6 a clock in the morning and my 2 cute little doughters want to go swimming in 4-6 hours :D :D
Doesnt matter what google dig, does or will doing for what ever reason (sic! :D ) i think.... the solution will worwith every app without an app-developer has to care about vold ;-)
Hope i could help a little bit.... and many thanks for this great app: Tasker :D
Alex
PS. Sorry for my awefull english :( ;-p
found this and the other thread (refund) about external SD-Card yesterday while i try copy files with tasker! There are many people with the same problem outthere! Tasker is a great app .. so i decide to find anonther solution for the external sd-card and dont waste my time :D ----> Here is it: http://forum.xda-developers.com/android/apps-games/apps2sd-partition-sd-card-link-apps-to-t3122919
Splitt external SD-Card in 2 Partitons, if u want you can format the second Partition as an ext4-device and link and/or bind/mount folders from external card into the filesystem of the internal storage! Doesnt test it with taskers actions yet,,, but an ordanary oem-stock-filemanger with just access to the internal storage havnt got any Problems. Will test it tommorow or monday with tasker.. but now it is 6 a clock in the morning and my 2 cute little doughters want to go swimming in 4-6 hours :D :D
Doesnt matter what google dig, does or will doing for what ever reason (sic! :D ) i think.... the solution will worwith every app without an app-developer has to care about vold ;-)
Hope i could help a little bit.... and many thanks for this great app: Tasker :D
Alex
PS. Sorry for my awefull english :( ;-p
Alex Alex-ma
Apr 17, 2016, 12:06:04 AM4/17/16
to Tasker
ooooops cant edit!? And no.... i dont want a refund <evil-smile> ;-p
stere...@yahoo.com
Apr 17, 2016, 4:33:32 AM4/17/16
to Tasker
Thanks for posting this, but the solution you suggested requires root (App2SD app requires root). So for most non-rooted users it will not help them.
-stereomatch
-stereomatch
N Mok
May 25, 2016, 1:26:35 PM5/25/16
to Tasker
this issue hit me recently since I upgraded to Marshmallow and because this affected several apps I use I had to look for something more permanent. Because I'm rooted, it wasn't a problem for me to implement the "fix" described here http://forum.xda-developers.com/showpost.php?p=66554730&postcount=27. only posting this here for those that have root and feel the pain of this whole mess Google created. unfortunately this "fix" has to be implemented by the user if I'm not mistaken...
Reply all
Reply to author
Forward
0 new messages