async_read on init?

[JT Olds]

Hey all,

I have a low-level interface FUSE filesystem working on Linux that I am porting to OS X. In the init method, a struct fuse_conn_info* is passed in, detailing what got set up.

The docs for fuse4x say that, like mainline fuse, async_read is the default setting. However, my assert in the init method on the async_read flag on the fuse_conn_info struct is failing. Is there a reason why the async_read flag isn't set? I'm not mounting the filesystem with a sync_read option or anything.

Thanks,

JT

[Anatol Pomozov]

Hi

On Fri, Jun 1, 2012 at 6:07 PM, JT Olds <he...@jtolds.com> wrote:

Hey all,

I have a low-level interface FUSE filesystem working on Linux that I am porting to OS X. In the init method, a struct fuse_conn_info* is passed in, detailing what got set up.

The docs for fuse4x say that, like mainline fuse, async_read is the default setting.

This field is no-op, currently it does not affect fuse4x. The doc should be fixed.

However, my assert in the init method on the async_read flag on the fuse_conn_info struct is failing. Is there a reason why the async_read flag isn't set? I'm not mounting the filesystem with a sync_read option or anything.

It should be easy to set the field before calling init(), but implementing async_read in the kernel is harder.

Here is the story - currently fuse4x extension asks kernel read buffers in async mode. But the problem is that VNOP_STRATEGY (that performs page read, equivalent of linux read/writepages) acquires a per-vnode lock (aka Biglock). This means that for a given vnode only one vnode operation can be run at a moment. Thus it effectively makes VNOP_STRATEGY and fuse reads serialized.

When biglock code will be eliminated (not an easy task :( ) then it will be possible to implement async_read correctly.

What are you trying to achieve? Is there any problem you are trying to solve by async_read?

[JT Olds]

Oh, no, not specifically. My codebase is running its own event loop
(calling fuse_chan_recv), and I was planning to do disk io in
different threads. I might not understand what the async_read option
is (but seeing as how it's the Linux-side default), I assumed what is
happening is that I might read a fuse_chan_recv request for a read
operation off the channel before I had sent a fuse_reply_buf for a
previous one. If my assumption is correct, this is nice so I can
handle requests simultaneously.

I suppose it's no real loss if they get serialized. Or perhaps I'm
just confused about what the async_read option is.

-JT

[Anatol Pomozov]

Hi

No, it is not related to read() on fuse device file (/dev/fuse). It is related to read() on files in fuse filesystem. Imagine following situation - some user opens a file on your fuse filesystem and wants to read it to a large buffer. When kernel (the fuse kernel module) receives this request it can read the data from the userspace it in several ways:

- sync mode. It means it sends FUSE_READ for the first page (4K) to your filesystem, then waits for a response, then sends FUSE_READ for the second page waits for a response, and so on.

- async mode or read-ahead. It sends FUSE_READ for the first page and then it sends another one, once response for first page received it sends request for 3rd page, .... Thus in case if you use multithreaded filesystem it might happen that multiple threads process read() for the same file at the same time. And these requests might be processed out-of-order.

PS the information I wrote is how I understand the linux kernel code, so I can be wrong. I suggest to clarify async_read in upstream maillist as it is more authoritative source of information.

[JT Olds]

Oh, it appears I've poorly described my assumptions. Your description
is exactly what I thought it meant.

And I assume your first email was saying that getting parallel
readahead requests working with Fuse4X is challenging, possibly
without much I can do, besides submit patches? I suppose not doing
something that can't currently be done anyway isn't that big of a
loss. :)

I've taken out the assert. And thanks.

-JT

[Anatol Pomozov]

Hi

I do not think you can do anything to enable the async read.

A good news is that the read-ahead is not that important in fuse4x. In linux READ uses a small buffer (4K, page) if i remember correctly. So read-ahead is important to hide the latency. In fuse4x READ uses large buffers and can read up to 32M at a time. Async read will not improve the performance as much as in linux.