about async I/O support in Go

[Manlio Perillo]

Hi.

I would like to share some idea about async I/O support in Go.

Historically, UNIX systems have implemented support for multiplexed I/O with polling.

Polling is the more natural (IMHO) method in network programming, but it is no good for

disk I/O.

Windows only supports async I/O, for everything including sockets.

In UNIX, async I/O is supported by a new POSIX aio API, and some operating systems like

FreeBSD have async I/O support in the poller.

Since Go offers a portable API for network programming and a simple portable API for files,

a possible solution to implement async I/O in Go is to implement a separate, portable, API

in io/async and/or os/async package.

This API should support async I/O for every files like objects, as it is done on Windows,

probably excluding sockets.

As an unrelated example, ACE provides both a Reactor and a Proactor:

http://www.artima.com/articles/io_design_patternsP.html

Go will have both the net package, with a reactor pattern implementation,

and an async package with a proactor pattern implementation.

Internally, both packages will use an internal. different, poller, when supported by

the operating system.

Comments?

Thanks   Manlio Perillo

[Jan Mercl]

On Fri Dec 12 2014 at 14:04:07 Manlio Perillo <manlio....@gmail.com> wrote:

> Comments?

Any I/O done in a goroutine is async I/O from the point of view of any other goroutine.

-j

[Manlio Perillo]

I was assuming async I/O supported by the kernel.

Of course you can do async I/O in Go, but I assume there is a valid reason why the

net package uses operating system support for polling .

The idea is to do the same with a new async package.

Regards   Manlio Perillo

[Luna Duclos]

Which use-case will you satisfy with this that is currently not possible or hard with Go ?

I struggle to find any.

Kr,

Luna

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Konstantin Khomoutov]

On Fri, 12 Dec 2014 05:04:01 -0800 (PST)
Manlio Perillo <manlio....@gmail.com> wrote:

> I would like to share some idea about async I/O support in Go.

[...]

> Go will have both the net package, with a reactor pattern
> implementation, and an async package with a proactor pattern
> implementation. Internally, both packages will use an internal.
> different, poller, when supported by
> the operating system.

Are you aware of [1], recently mentioned in another thread?

1. https://github.com/golang/go/issues/6817

[Konstantin Khomoutov]

On Fri, 12 Dec 2014 14:55:14 +0100
Luna Duclos <luna....@palmstonegames.com> wrote:

> Which use-case will you satisfy with this that is currently not
> possible or hard with Go ?
> I struggle to find any.

Currently, once the Go runtime detects a goroutine gets blocked in a
syscall (it waits on it for longer than such and such milliseconds),
the runtime would fork another OS thread to let other goroutines
continue to run. Hence if you for some reason have lots of goroutines
making syscalls which tend to block for longer than a certain
threshold, the runtime could start generate lots of OS threads kind of
defeating the whole idea of having multiple goroutines multiplexed onto
a single or few OS threads. See [1] for more info.

1. https://github.com/golang/go/issues/6817

[Ian Lance Taylor]

On Fri, Dec 12, 2014 at 5:04 AM, Manlio Perillo
<manlio....@gmail.com> wrote:
>
> Since Go offers a portable API for network programming and a simple portable
> API for files,
> a possible solution to implement async I/O in Go is to implement a separate,
> portable, API
> in io/async and/or os/async package.
> This API should support async I/O for every files like objects, as it is
> done on Windows,
> probably excluding sockets.

Why not simply change the os package to use async I/O internally? I
can't think of any useful purpose to introducing a new package that
would essentially duplicate the functionality of os.File.

Ian

[David Crawshaw]

One reason for a new package is context plumbing for cancellation. It
would break the current os package interface. Here's an experiment
along those lines:

http://github.com/crawshaw/fs

I'm not proposing adding it to the standard library any time soon, in
fact it has a long way to go before I would even propose it for
golang.org/x/net.

Note that if you are interested in using underlying async APIs to
extract different semantics from your OS (for example, cancellation),
you should map them onto synchronous interfaces in Go. There is no
need complex, error-prone, unreadable async-style programming when you
have lightweight goroutines: simply start another goroutine and block.

d.