                                                                                                                                                                                                                                                                
MIME-Version: 1.0
Date: Sun, 9 Jan 2011 18:11:56 -0800 (PST)
In-Reply-To: <AANLkTint18Mg2+tMKWR=9VAvEAzZ1yjSZxqG+2f-VOEa@mail.gmail.com>
References: <AANLkTi=Yq42rw_mo5Q6=+=RabGZ8sgm_0tYGuseOOriO@mail.gmail.com>
 <AANLkTi=FAeCWeSc8vsyw+HkXQ03V88Ftsz8r2yL893wP@mail.gmail.com>
 <AANLkTi=cVxZ0-V4oL6tYY79dq2_sTLv2BVjBCxV3LyQ0@mail.gmail.com>
 <a52bc509-15a3-453d-877e-f23548c32c18@k14g2000pre.googlegroups.com>
 <AANLkTi=gPRw8yiLc29KbO4f5McUygA1Z-4N=s8_2pAWX@mail.gmail.com>
 <ca8b0e92-c21e-4efd-a027-4758071dfb12@m11g2000vbs.googlegroups.com> <AANLkTint18Mg2+tMKWR=9VAvEAzZ1yjSZxqG+2f-VOEa@mail.gmail.com>
User-Agent: G2/1.0
Message-ID: <5d7e752e-c2f8-4888-b09e-e77d778d17fb@e20g2000vbn.googlegroups.com>
Subject: Re: Asynchronous file I/O with AIO
From: billywhizz <apjohn...@gmail.com>
To: nodejs-dev <nodejs-dev@googlegroups.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

i agree that the libeio solution seems hacky but don't know enough
about the alternatives (namely Posix AIO) to offer an educated
opinion. i was recently reading this though:

http://davmac.org/davpage/linux/async-io.html

and am wondering if this criticism of AIO on Linux is still valid:

"aio_suspend(), while it might seem to solve the issue of
notification, requires scanning the list of the aiocb structures by
the kernel (to determine whether any of them have completed) and the
userspace process (to find which one completed). That is to say, it
has exactly the same problems as poll()"

i can sympathise with the desire to have a generic/cross-platform io
library like libeio as the default in node but would be very
interested in a compatible alternative library based on something that
performs better for specific platforms, i.e. AIO on linux/osx,
Overlapped IO/Completion ports on Windows...

On Jan 9, 10:18=A0pm, Rasmus Andersson <ras...@notion.se> wrote:
> On Sun, Jan 9, 2011 at 19:00, Marco Rogers <marco.rog...@gmail.com> wrote=
:
> > I'm always interested when this thread comes back up. There has been
> > discussion on it before. I keep these bookmarked for reference. =A0Some
> > key points. The author of libeio doesn't think AIO is always better.
> > Also Ry would much prefer any AIO support to be implemented as part of
> > libeio. =A0Of course these sentiments may have changed.
>
> >http://groups.google.com/group/nodejs/browse_thread/thread/aff97d25c5...
> >http://groups.google.com/group/nodejs/browse_thread/thread/de115072e2...
>
> Thanks for the update!
>
> In my opinion:
>
> - libeio -- a "hack" to make blocking system calls non-blocking
> - libev -- event-based runloop framework for working with asynchronous
> I/O, timers and signals
>
> Putting AIO support in libeio feels a little bit like flying an
> airplane inside another airplane. Libev on the other hand, has the
> official shortcoming of not supporting file descriptors pointing to
> files. Simply adding AIO support to the internals of libev and --
> voil=E1 -- you can pass a filesystem file descriptor and use all the
> existing infrastructure. Such a solution would mean less code for node
> and numerous other projects using libev.
>
>
>
>
>
>
>
>
>
>
>
> > Hope this background is helpful. =A0I always learn a lot from these
> > discussions.
>
> > :Marco
>
> > On Jan 9, 1:39=A0am, Rasmus Andersson <ras...@notion.se> wrote:
> >> On Sun, Jan 9, 2011 at 06:31, Liam <networkimp...@gmail.com> wrote:
> >> > I think the question is: should we consider implementing a new fs i/=
o
> >> > lib that leverages the native OS better, assuming there's sufficient
> >> > AIO support in the mostly widely used OSes...
>
> >> Yeah, that's the question. Personally, I'm using node.js instead of
> >> Python, Perl or Ruby nowadays and see it as a replacement for all
> >> tasks those environments otherwise serve. Kind of like my
> >> universal/default programming environment. Disk I/O is a pretty
> >> important feature in these use-cases (crunching logs, etc), so I think
> >> AIO support would be a nice idea. It's not that much work given it's
> >> possible to use aio_suspend with the relatively simplistic libev
> >> runloop (I think we would need to query libev for any timers and set a
> >> slightly pessimistic timeout based on that on aio_suspend though).
>
> >> Putting AIO support into libeio doesn't make sense (adding AIO support
> >> to libev make a little more sense though).
>
> >> > On Jan 8, 8:21=A0pm, Isaac Schlueter <i...@izs.me> wrote:
> >> >> It seems to me like this is something that should live in the EIO
> >> >> implementation, rather than in node.
>
> >> --
> >> Rasmus Andersson
>
> --