Received: by 10.204.6.19 with SMTP id 19mr544208bkx.8.1350174391037;
Sat, 13 Oct 2012 17:26:31 -0700 (PDT)
X-BeenThere: nodejs@googlegroups.com
Received: by 10.205.124.16 with SMTP id gm16ls4066870bkc.7.gmail; Sat, 13 Oct
2012 17:26:11 -0700 (PDT)
Received: by 10.204.127.19 with SMTP id e19mr543147bks.4.1350174371205;
Sat, 13 Oct 2012 17:26:11 -0700 (PDT)
Received: by 10.204.127.19 with SMTP id e19mr543146bks.4.1350174371172;
Sat, 13 Oct 2012 17:26:11 -0700 (PDT)
Return-Path: <m...@hahnca.com>
Received: from mail-lb0-f173.google.com (mail-lb0-f173.google.com [209.85.217.173])
by gmr-mx.google.com with ESMTPS id 27si1206798bks.3.2012.10.13.17.26.11
(version=TLSv1/SSLv3 cipher=OTHER);
Sat, 13 Oct 2012 17:26:11 -0700 (PDT)
Received-SPF: neutral (google.com: 209.85.217.173 is neither permitted nor denied by best guess record for domain of m...@hahnca.com) client-ip=209.85.217.173;
Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 209.85.217.173 is neither permitted nor denied by best guess record for domain of m...@hahnca.com) smtp.mail=m...@hahnca.com
Received: by mail-lb0-f173.google.com with SMTP id gj3so3440009lbb.18
for <nodejs@googlegroups.com>; Sat, 13 Oct 2012 17:26:10 -0700 (PDT)
d=google.com; s=20120113;
h=mime-version:x-originating-ip:in-reply-to:references:from:date
:message-id:subject:to:content-type:x-gm-message-state;
bh=o0pkCl55PHLgbtefn65IoisD95iW/22O150996Ez1pI=;
b=CAld+8D+uSGtLx5zoag2REKU8cPbDnGYZkXPMKxPBVBOsZNIR7P2ZKwHHq90AMY3YD
+vvR6J5f3zbdAo6MjKU9nfhraZz6uAmzhN7BziOvOU9fTwrm8u8mC7gGd73LpxOdpeFh
kL1n7ppJll0tNF7QHGfuggkxJBEZqG0PsRYhqwC1fMuaVC7+nHImbRDE7suCv6TT4Kg1
nuPGWA46fPIxJpFeqp0RmYYfdooQQwFP+5gW5SHqgOeNojdtF29Gk+gGoGvO/QnlKpgq
rDy728J+vGM28PlsOoADAR1dcd547tz5mniqk5UGtd0UDgY7PnD+i2Y+Hwz5wjho6/VI
t2fw==
Received: by 10.112.84.135 with SMTP id z7mr2901024lby.119.1350174370738; Sat,
13 Oct 2012 17:26:10 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.114.48.34 with HTTP; Sat, 13 Oct 2012 17:25:50 -0700 (PDT)
X-Originating-IP: [68.5.117.177]
In-Reply-To: <CAN5MXodCGiasWyGyt+rogt2-1g-HEd-KU7Gqn4Cwz9Up+Ow...@mail.gmail.com>
References: <42eeb107-5160-4b41-9c77-3e3dc781585e@googlegroups.com>
<c09f1ba7-b9b3-4dd6-bcdc-f44a1b9148ac@googlegroups.com> <D9FACECB-1EAE-4C09-A864-AC223F3A0...@gmail.com>
<1d8a2240-6830-4b59-9722-3d7f1755dbfe@googlegroups.com> <A4CD268E-BACF-49F1-9565-2D059B1EF...@gmail.com>
<CADcwD-EiEPDD6zyMPZ-kUgbJvyhzu8wGaMGrEiNu-6GWMwi...@mail.gmail.com>
<415ec109-fe0d-4892-bfde-37171218c5dd@googlegroups.com> <CACrj35Ev1-N6sfrngo8B60z+rtrqe_+SU=tMabSqDiKgZEL...@mail.gmail.com>
<CAN5MXodCGiasWyGyt+rogt2-1g-HEd-KU7Gqn4Cwz9Up+Ow...@mail.gmail.com>
From: Mark Hahn <m...@hahnca.com>
Date: Sat, 13 Oct 2012 17:25:50 -0700
Message-ID: <CACrj35FoyXreSeusfD5VhABnuq1-sPKdO4dXS-xLYzjCimS...@mail.gmail.com>
Subject: Re: [nodejs] Re: How to write stream chunk by chunk with callbacks
To: nodejs@googlegroups.com
Content-Type: multipart/alternative; boundary=f46d0401687bc0b1d704cbf9f73f
X-Gm-Message-State: ALoCoQls+2dXSozdn+nrUM9JWWT46GktDmdCh6geK0TiYblq1cNdd8QEUU3dSig3bmfItRv4LeUh
--f46d0401687bc0b1d704cbf9f73f
Content-Type: text/plain; charset=ISO-8859-1
But pipe only works if the writes are to another stream. If they are to a
db driver or something without pipe support then I have to do my own reads.
Or am I missing something here?
On Sat, Oct 13, 2012 at 5:19 PM, Nathan Rajlich <nat...@tootallnate.net>wrote:
> Mark, to pump at max rate you'd use .pipe().
>
> On Sat, Oct 13, 2012 at 5:16 PM, Mark Hahn <m...@hahnca.com> wrote:
> >> There is no 'data' event. There is a read() method, and a 'readable'
> >> event. You call read() until it returns null, and then wait for a
> readable
> >> event to tell you it's time to read() more.
> >
> > So, if we want to pump it at max rate we would run a tight loop to read
> and
> > write in the beginning and then on every readable event? It seems like
> > more work and a lot messier compared to the old data event scheme.
> >
> >
> > On Sat, Oct 13, 2012 at 5:08 PM, Alexey Petrushin
> > <alexey.petrus...@gmail.com> wrote:
> >>
> >> Thanks for help, especially You Isaac for such a detailed answer.
> >>
> >> As far as I understand it's possible to wrap existing evented stream API
> >> into callback interface (with in-memory data buffers to handle mismatch
> >> between explicit/implicit control flow).
> >> But probably it won't worth it, it will be more easy to just use it as
> >> it's supposed to be used (with pipes) and wait untill those changes in
> 0.10.
> >> The new API seems to be very similar to what I asked for.
> >>
> >> P.S.
> >>
> >> As for the question and why do I need it - I'm working on application
> that
> >> uses custom streams and though that maybe I can cheat and simplify my
> work a
> >> little by not implementing complex evented interface :).
> >>
> >> I once used such abstraction for working with streams in ruby:
> >>
> >> to.write do |writer|
> >> from.read{|buff| writer.write buff}
> >> end
> >>
> >> Files are open and closed properly, buffer also have some default size,
> so
> >> the code is very simple to use (more details
> >> http://alexeypetrushin.github.com/vfs ).
> >> Basically by implementing just those two methods You get ability to
> stream
> >> from any stream into any stream (fs, s3, sftp, ...).
> >>
> >> I tried to do something similar with asynchronous streams.
> >>
> >> --
> >> Job Board: http://jobs.nodejs.org/
> >> Posting guidelines:
> >> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> >> You received this message because you are subscribed to the Google
> >> Groups "nodejs" group.
> >> To post to this group, send email to nodejs@googlegroups.com
> >> To unsubscribe from this group, send email to
> >> nodejs+unsubscribe@googlegroups.com
> >> For more options, visit this group at
> >> http://groups.google.com/group/nodejs?hl=en?hl=en
> >
> >
> > --
> > Job Board: http://jobs.nodejs.org/
> > Posting guidelines:
> > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> > You received this message because you are subscribed to the Google
> > Groups "nodejs" group.
> > To post to this group, send email to nodejs@googlegroups.com
> > To unsubscribe from this group, send email to
> > nodejs+unsubscribe@googlegroups.com
> > For more options, visit this group at
> > http://groups.google.com/group/nodejs?hl=en?hl=en
>
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nodejs@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+unsubscribe@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>
--f46d0401687bc0b1d704cbf9f73f
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
But pipe only works if the writes are to another stream. =A0If they are to =
a db driver or something without pipe support then I have to do my own read=
s. =A0Or am I missing something here?<br><br><div class=3D"gmail_quote">On =
Sat, Oct 13, 2012 at 5:19 PM, Nathan Rajlich <span dir=3D"ltr"><<a href=
=3D"mailto:nat...@tootallnate.net" target=3D"_blank">nat...@tootallnate.net=
</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">Mark, to pump at max rate you'd use .pip=
e().<br>
<div class=3D"HOEnZb"><div class=3D"h5"><br>
On Sat, Oct 13, 2012 at 5:16 PM, Mark Hahn <<a href=3D"mailto:mark@hahnc=
a.com">m...@hahnca.com</a>> wrote:<br>
>> There is no 'data' event. =A0There is a read() method, and=
a 'readable'<br>
>> event. =A0You call read() until it returns null, and then wait for=
a readable<br>
>> event to tell you it's time to read() more.<br>
><br>
> So, if we want to pump it at max rate we would run a tight loop to rea=
d and<br>
> write in the beginning and then on every readable event? =A0 It seems =
like<br>
> more work and a lot messier compared to the old data event scheme.<br>
><br>
><br>
> On Sat, Oct 13, 2012 at 5:08 PM, Alexey Petrushin<br>
> <<a href=3D"mailto:alexey.petrus...@gmail.com">alexey.petrushin@gma=
il.com</a>> wrote:<br>
>><br>
>> Thanks for help, especially You Isaac for such a detailed answer.<=
br>
>><br>
>> As far as I understand it's possible to wrap existing evented =
stream API<br>
>> into callback interface (with in-memory data buffers to handle mis=
match<br>
>> between explicit/implicit control flow).<br>
>> But probably it won't worth it, it will be more easy to just u=
se it as<br>
>> it's supposed to be used (with pipes) and wait untill those ch=
anges in 0.10.<br>
>> The new API seems to be very similar to what I asked for.<br>
>><br>
>> P.S.<br>
>><br>
>> As for the question and why do I need it - I'm working on appl=
ication that<br>
>> uses custom streams and though that maybe I can cheat and simplify=
my work a<br>
>> little by not implementing complex evented interface :).<br>
>><br>
>> I once used such abstraction for working with streams in ruby:<br>
>><br>
>> =A0 =A0 to.write do |writer|<br>
>> =A0 =A0 =A0 from.read{|buff| writer.write buff}<br>
>> =A0 =A0 end<br>
>><br>
>> Files are open and closed properly, buffer also have some default =
size, so<br>
>> the code is very simple to use (more details<br>
>> <a href=3D"http://alexeypetrushin.github.com/vfs" target=3D"_blank=
">http://alexeypetrushin.github.com/vfs</a> ).<br>
>> Basically by implementing just those two methods You get ability t=
o stream<br>
>> from any stream into any stream (fs, s3, sftp, ...).<br>
>><br>
>> I tried to do something similar with asynchronous streams.<br>
>><br>
>> --<br>
>> Job Board: <a href=3D"http://jobs.nodejs.org/" target=3D"_blank">h=
ttp://jobs.nodejs.org/</a><br>
>> Posting guidelines:<br>
>> <a href=3D"https://github.com/joyent/node/wiki/Mailing-List-Postin=
g-Guidelines" target=3D"_blank">https://github.com/joyent/node/wiki/Mailing=
-List-Posting-Guidelines</a><br>
>> You received this message because you are subscribed to the Google=
<br>
>> Groups "nodejs" group.<br>
>> To post to this group, send email to <a href=3D"mailto:nodejs@goog=
legroups.com">nodejs@googlegroups.com</a><br>
>> To unsubscribe from this group, send email to<br>
>> <a href=3D"mailto:nodejs%2Bunsubscribe@googlegroups.com">nodejs+un=
subscribe@googlegroups.com</a><br>
>> For more options, visit this group at<br>
>> <a href=3D"http://groups.google.com/group/nodejs?hl=3Den?hl=3Den" =
target=3D"_blank">http://groups.google.com/group/nodejs?hl=3Den?hl=3Den</a>=
<br>
><br>
><br>
> --<br>
> Job Board: <a href=3D"http://jobs.nodejs.org/" target=3D"_blank">http:=
//jobs.nodejs.org/</a><br>
> Posting guidelines:<br>
> <a href=3D"https://github.com/joyent/node/wiki/Mailing-List-Posting-Gu=
idelines" target=3D"_blank">https://github.com/joyent/node/wiki/Mailing-Lis=
t-Posting-Guidelines</a><br>
> You received this message because you are subscribed to the Google<br>
> Groups "nodejs" group.<br>
> To post to this group, send email to <a href=3D"mailto:nodejs@googlegr=
oups.com">nodejs@googlegroups.com</a><br>
> To unsubscribe from this group, send email to<br>
> <a href=3D"mailto:nodejs%2Bunsubscribe@googlegroups.com">nodejs+unsubs=
cribe@googlegroups.com</a><br>
> For more options, visit this group at<br>
> <a href=3D"http://groups.google.com/group/nodejs?hl=3Den?hl=3Den" targ=
et=3D"_blank">http://groups.google.com/group/nodejs?hl=3Den?hl=3Den</a><br>
<br>
--<br>
Job Board: <a href=3D"http://jobs.nodejs.org/" target=3D"_blank">http://job=
s.nodejs.org/</a><br>
Posting guidelines: <a href=3D"https://github.com/joyent/node/wiki/Mailing-=
List-Posting-Guidelines" target=3D"_blank">https://github.com/joyent/node/w=
iki/Mailing-List-Posting-Guidelines</a><br>
You received this message because you are subscribed to the Google<br>
Groups "nodejs" group.<br>
To post to this group, send email to <a href=3D"mailto:nodejs@googlegroups.=
com">nodejs@googlegroups.com</a><br>
To unsubscribe from this group, send email to<br>
<a href=3D"mailto:nodejs%2Bunsubscribe@googlegroups.com">nodejs+unsubscribe=
@googlegroups.com</a><br>
For more options, visit this group at<br>
<a href=3D"http://groups.google.com/group/nodejs?hl=3Den?hl=3Den" target=3D=
"_blank">http://groups.google.com/group/nodejs?hl=3Den?hl=3Den</a><br>
</div></div></blockquote></div><br>
--f46d0401687bc0b1d704cbf9f73f--
Message from discussion How to write stream chunk by chunk with callbacks
| Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy |
| ©2013 Google |