mount remote storage without FUSE

[ChrisLu]

Hi, Gophers (who are also good in C),

Currently I am using FUSE to mount and write to remote storage. 

The library I am using is Basil Fuse. The performance is good as far as FUSE can go.

User -> file.write() -> Linux Kernel -> Virtual File System -> libfuse -> FUSE -> DFS Client(in Go) -> Remote Storage

However, I am researching how to write directly through linux virtual file system, skipping FUSE layer. Seems all similar libraries are in C. Is this even possible in Go?

User -> file.write() -> Linux Kernel -> Virtual File System -> DFS Client(in Go) -> Remote Storage

Thanks! 

Chris

https://github.com/chrislusf/seaweedfs

[Axel Wagner]

Can you link to an example C project? Theoretically, everything that's possible in C should also be possible in Go. However, I'm unaware of a method to mount a userspace filesystem except FUSE. So as far as I'm aware, you'd need a kernel module for that (and, yeah, doing that with Go would probably be inadvisable).

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/6f4c54fb-d5a5-4a1f-9840-0fbab596f785n%40googlegroups.com.

[Chris Lu]

Hi, Axel,

Thanks for answering! I was checking CEPH, Lustre, etc. Here is one example:

https://github.com/ceph/ceph/blob/3931b21ae8ae508f2cd826a1e6cc89c9b2f64f97/src/libcephfs.cc

Chris

https://github.com/chrislusf/seaweedfs

[Axel Wagner]

Yes, both of those seem to be using kernel modules :) Implementing kernel modules in Go would be ambitious, to say the least (at least in my opinion).

[Bakul Shah]

This may help: https://github.com/zargony/fuse-rs

The kernel driver is provided by the FUSE project, the userspace implementation needs to be provided by the developer. fuse-rs provides a replacement for the libfuse userspace library between these two. This way, a developer can fully take advantage of the Rust type interface and runtime features when building a FUSE filesystem in Rust.

Go implementation should be possible.

[Chris Lu]

This Rust fuse-rs project is equivalent to the Go basizl/fuse project that I am using.

Chris

[Didier Spezia]

One possibility is to use the 9P client of the kernel and implements a 9P server in Go - there are a few 9P libraries for Go around.

I doubt it will be more efficient than FUSE though, and btw FUSE is largely based on the ideas promoted by 9P.

Another possibility is to use the ndb client of the kernel and implements a ndb server in Go. See Axel's ndb package at https://github.com/Merovius/nbd

Now ndb exposes a block device, not a filesystem. I think I would be quite difficult to emulate the format of one the filesystems supported by the kernel and implements it in this Go ndb server.

Regards,

Didier.

[Axel Wagner]

Yeah, NFS would work as well (similar tradeoffs as 9P, though the linux kernel driver might perform better because it's more widely used).

nbd is not suitable for the use case, I think :) I'd also recommend against using my nbd package if performance is the goal - it is not at all optimized, I wrote it mostly as a tool to test filesystem implementations and things that depend on certain filesystem semantics.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/835bbe35-6845-4d44-98b9-83a07c86b79bn%40googlegroups.com.