Will be FUSE removed?

[Michał Kowalczyk]

Hello.

Please see my research about the way Android uses FUSE: http://fixbugfix.blogspot.com/2015/11/why-does-fuse-on-android-suck.html . Are there any chances that Google will replace it with some kind of in-kernel implementation?

/ Michal

[Dave Smith]

I would be quite surprised if there were any plans to remove FUSE, as it well serves its purpose for external storage—which is dynamic permissions. The performance implications would likely be considered more critical if they existed system-wide, but they only affect external storage volumes (as you noted in your article).

I don't believe that pure DAC is fine-grained enough for the "unique" permissions that apply to external storage directories. One could potentially argue that SE policy could pick up the slack and create something similar…but on Android that policy is static. FUSE offers them the ability to reacts to requests dynamically and enforce permissions based on changing criteria (that also vary between primary and secondary volumes), which I'm guessing outweighs the performance penalties.

Note that I'm just a lowly passer-by, and there's just as much chance that Google is in fact working to improve this. But, as I said earlier, that would surprise me.

Cheers,

--

Dave Smith, PE

@devunwired

[Michał Kowalczyk]

I didn't write that it affects only an external storage. I wrote exactly the opposite: it affects the internal storage as well. Of course the external storage is also mounted via FUSE, but my main concerns were about performance issues while using internal flash by applications.

BR,

Michal

[Dave Smith]

I think you're confusing what I mean by internal/external. I'm referring to Androids use of those terms: http://developer.android.com/guide/topics/data/data-storage.html

Your article states (correctly) that RW on /data is faster than /sdcard. This is because /sdcard is the mounted volume for primary external storage (and, thus, uses FUSE). The /data partition is where all the internal storage is mounted, which does not use FUSE.

Cheers,
--
Dave Smith, PE
@devunwired

--
You received this message because you are subscribed to the Google Groups "android-platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-platfo...@googlegroups.com.
To post to this group, send email to android-...@googlegroups.com.
Visit this group at http://groups.google.com/group/android-platform.
For more options, visit https://groups.google.com/d/optout.

[Jeff Sharkey]

We're definitely feeling the overhead that you described, and we'd like to improve it over time.  There are a couple different things coming into play with how this could evolve:

Because FUSE is a nice stable API, there is essentially zero maintenance work required when moving between kernel versions.  If we migrated to an in-kernel solution, we'd be signing up for maintaining a set of patches for each stable kernel version.

I've heard that some OEMs have moved to an in-kernel solution, but I haven't looked at any of them.  The one tricky problem we'd need to solve is communicating the package-name-to-appId mapping into the kernel.  I'd *strongly* oppose using the current FUSE inotify-based approach inside the kernel, and we should instead look at the framework pushing those mappings through an ioctl() somewhere.  (That also has the benefit of being able to do lightweight delta updates, instead of a full packages.list reparse.)

A potential middle-ground would be writing and upstreaming a new in-kernel FUSE feature to "off-load" the I/O for an opened file so the kernel can shortcut normal read/write() traffic to an underlying file handle.  We'd still have all the directory traversal overhead, but this would at least help with single-file I/O performance.

So in summary, we'd like to improve things, and there are a few different options, but we can't promise anything.

j

--

You received this message because you are subscribed to the Google Groups "android-platform" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-platfo...@googlegroups.com.
To post to this group, send email to android-...@googlegroups.com.
Visit this group at http://groups.google.com/group/android-platform.
For more options, visit https://groups.google.com/d/optout.

--

Jeff Sharkey
jsha...@android.com

[Michał Kowalczyk]

Yep, in that context you are correct. 

So, to clarify: the "/sdcard" folder is physically internal flash (eMMC) mounted via FUSE. There can be also other location, like for instance "/sdcard1", that can be physically external uSD card and is also mounted via FUSE.  For the application developer, the "external" storage will be those mounted via FUSE ("/sdcard" or "/sdcard1") and "internal" will be some folder inside /data partition which is EXT4 partition on eMMC flash. Is that correct? :) 

Although not all data files from apks are stored via FUSE, still there are use cases when FUSE limits performance: offline maps or photos can be stored that way. 

BR,

Michal 

[Dave Smith]

So, to clarify: the "/sdcard" folder is physically internal flash (eMMC) mounted via FUSE. There can be also other location, like for instance "/sdcard1", that can be physically external uSD card and is also mounted via FUSE.  For the application developer, the "external" storage will be those mounted via FUSE ("/sdcard" or "/sdcard1") and "internal" will be some folder inside /data partition which is EXT4 partition on eMMC flash. Is that correct?

Mostly, yes. Google doesn't require OEMs to put either primary or secondary external storage on eMMC, removable SD Card, or any other medium. However, the convention these days to almost always to use on-board eMMC for primary external storage. If devices contain a removable SD Card, it is usually secondary…but again, that is not a requirement.

…still there are use cases when FUSE limits performance: offline maps or photos can be stored that way.

Correct, FUSE always limits performance. My original point was that the dynamic permissions it allows in a zero-maintenance package (as Jeff pointed out) currently seems to outweigh that in the priority list.

Cheers,

--

Dave Smith, PE

@devunwired

[Michał Kowalczyk]

W dniu środa, 9 grudnia 2015 20:49:00 UTC+1 użytkownik Jeff Sharkey napisał:

We're definitely feeling the overhead that you described, and we'd like to improve it over time.  There are a couple different things coming into play with how this could evolve:

Because FUSE is a nice stable API, there is essentially zero maintenance work required when moving between kernel versions.  If we migrated to an in-kernel solution, we'd be signing up for maintaining a set of patches for each stable kernel version.

I've heard that some OEMs have moved to an in-kernel solution, but I haven't looked at any of them.  The one tricky problem we'd need to solve is communicating the package-name-to-appId mapping into the kernel.  I'd *strongly* oppose using the current FUSE inotify-based approach inside the kernel, and we should instead look at the framework pushing those mappings through an ioctl() somewhere.  (That also has the benefit of being able to do lightweight delta updates, instead of a full packages.list reparse.)

At least Samsung did it and the solution can be seen at http://opensource.samsung.com/ (Kernel/fs/sdcardfs). Last version I checked was for Lollipop.

A potential middle-ground would be writing and upstreaming a new in-kernel FUSE feature to "off-load" the I/O for an opened file so the kernel can shortcut normal read/write() traffic to an underlying file handle.  We'd still have all the directory traversal overhead, but this would at least help with single-file I/O performance.

I could be missing something, but from my research about "middle-ground" solution (modifying kernel part of FUSE) it turned out it's not worth of implementing. The WrapFS just suits better for this application.

So in summary, we'd like to improve things, and there are a few different options, but we can't promise anything.

Thanks for the feedback.