Possibility of asynchronous file I/O

[Marc Rosen]

Hi,

I was looking at the current implementation of File.ReadAt(), and it seems that it currently uses pread() to perform reads synchronously. Go, currently, does not perform blocking reads on reads on sockets, using methods such as kqueue and epoll to increase performance. On Linux, there is are special asynchronous I/O syscalls (https://code.google.com/p/kernel/wiki/AIOUserGuide) which allows epoll() to be extended to disk I/O operations. According to http://www.lighttpd.net/2007/2/3/raw-io-performance/, the use of asynchronous I/O can produce significant speed increases in disk I/O. I was wondering whether anybody had looked into replacing calls to pread() and pwrite(), on Linux with calls to asynchronous I/O and registration with epoll(). Replacing the pread() and pwrite() calls, should, in theory, enable speed increases on existing code, without causing side effects. If this seems like something worth doing, I'd be happy to look into it.

~ Marc

[Dave Cheney]

The net package uses epoll (and friends) to reduce the number of thread blocked on system calls. This is tangential to performance. 

There is little value in using async io rather that pread as that just shifts the ownership of the blocked thread from userspace to kernel space. 

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Gustavo Niemeyer]

There's some data and more in-depth coverage to back Dave's point here:

Why Events Are A Bad Idea (for high-concurrency servers)
http://static.usenix.org/events/hotos03/tech/full_papers/vonbehren/vonbehren_html/

--

gustavo @ http://niemeyer.net

[Dmitry Vyukov]

On Sat, Aug 24, 2013 at 6:26 AM, Gustavo Niemeyer <gus...@niemeyer.net> wrote:
> There's some data and more in-depth coverage to back Dave's point here:
>
> Why Events Are A Bad Idea (for high-concurrency servers)
> http://static.usenix.org/events/hotos03/tech/full_papers/vonbehren/vonbehren_html/

This seems to be saying in favor of async file IO, because their
"threading" library is a user-space cooperative scheduler.

[Dmitry Vyukov]

Hi,

Is that lighttpd thing multithreaded? Because if you issues enough
(e.g. 50) blocking reads concurrently, it should give a very good
performance (probably even better than any of aio/epoll). But there is
still a problem with thread consumption. Currently in Go if one
carelessly issue file reads from lots of goroutines, it can cause
creation of basically infinite amount of threads.

I think it's worth doing.

What OSes are you interested in? I would expect windows to be the
easiest to implement.
Do you know how to implement it on linux?

The following program fails with:
epoll_ctl: Operation not permitted
EPERM The target file fd does not support epoll.

#include <sys/epoll.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
int main(void) {
struct epoll_event evt = { .events = EPOLLIN };
int ep, fd;
ep = epoll_create1(0);
if (ep == -1) { perror("epoll_create"); return 1; }
fd = open("/bin/ls", O_RDONLY | O_NONBLOCK);
if (fd == -1) { perror("open"); return 1; }
if (epoll_ctl(ep, EPOLL_CTL_ADD, fd, &evt) == -1) {
perror("epoll_ctl"); return 1; }
return 0;
}

As far as I know, aio used a background pool of threads on linux,
which is absolutely unacceptable for Go. And I would like to not have
the second IO dispatcher for files.

In any case, here is the issue to track this:
https://code.google.com/p/go/issues/detail?id=6222

A good start may be creating a benchmark and estimating size of the problem.

[Gustavo Niemeyer]

On Sat, Aug 24, 2013 at 6:25 AM, Dmitry Vyukov <dvy...@google.com> wrote:
> This seems to be saying in favor of async file IO, because their
> "threading" library is a user-space cooperative scheduler.

It's saying that you can use user-space scheduling to offer a blocking
API on top of an asynchronous system call.

"Our thread package uses the coro coroutine library [15] for
minimalist context switching, and it translates blocking I/O requests
to asynchronous requests internally."

Sounds familiar?


gustavo @ http://niemeyer.net

[Gustavo Niemeyer]

On Sat, Aug 24, 2013 at 6:41 AM, Dmitry Vyukov <dvy...@google.com> wrote:
> In any case, here is the issue to track this:
> https://code.google.com/p/go/issues/detail?id=6222

Sorry, I actually see what you meant now. We're basically all in
agreement, and agree with the article as well.


gustavo @ http://niemeyer.net