How to write stream chunk by chunk with callbacks - nodejs

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion How to write stream chunk by chunk with callbacks
Received-SPF: pass (google.com: domain of mikeal.rog...@gmail.com designates 74.125.82.52 as permitted sender) client-ip=74.125.82.52;
From: Mikeal Rogers <mikeal.rog...@gmail.com>
Content-Type: multipart/alternative; boundary="Apple-Mail=_87558070-80BD-4546-BEF5-BF372BBAD744"
Message-Id: <A4CD268E-BACF-49F1-9565-2D059B1EF...@gmail.com>
Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\))
Subject: Re: [nodejs] Re: How to write stream chunk by chunk with callbacks
Date: Sat, 13 Oct 2012 22:33:00 +0200
References: <42eeb107-5160-4b41-9c77-3e3dc781585e@googlegroups.com> <c09f1ba7-b9b3-4dd6-bcdc-f44a1b9148ac@googlegroups.com> <D9FACECB-1EAE-4C09-A864-AC223F3A0...@gmail.com> <1d8a2240-6830-4b59-9722-3d7f1755dbfe@googlegroups.com>
To: nodejs@googlegroups.com
In-Reply-To: <1d8a2240-6830-4b59-9722-3d7f1755dbfe@googlegroups.com>


--Apple-Mail=_87558070-80BD-4546-BEF5-BF372BBAD744
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=iso-8859-1

i'm not engaging with your strawman Bruno.

i showed how we *actually* move data in node. this is not a debate, =
that's how it works. if anyone wants to use node, or write a module that =
has a stream that moves data, that's how they do it.

this was a question, not an open invitation for bikeshedding. please let =
the list answer questions.

On Oct 13, 2012, at October 13, 20128:04 PM, Bruno Jouhier =
<bjouh...@gmail.com> wrote:

> What's wrong?=20
>=20
> You'll find links to gists at the end of my post. The code works!
> And Alexey's pumping function is equivalent to the pumping loop I gave =
in my post.
>=20
> On Saturday, October 13, 2012 7:31:54 PM UTC+2, Mikeal Rogers wrote:
> all of these are wrong.
>=20
> inputStream.pipe(outputStream)
> outputStream.on('close', callback)
>=20
> On Oct 13, 2012, at October 13, 20127:23 PM, Bruno Jouhier =
<bjou...@gmail.com> wrote:
>=20
>> I wrote a post about plain callback APIs for streams: =
http://bjouhier.wordpress.com/2012/07/04/node-js-stream-api-events-or-call=
backs/
>>=20
>>=20
>> On Saturday, October 13, 2012 11:17:13 AM UTC+2, Alexey Petrushin =
wrote:
>> I don't quite understand how steam pause/resume works, or more =
exactly - how to use it in simple manner. It's necessary to use it in =
situations when the read stream produces data faster than the write =
stream can consume.=20
>>=20
>> I need to write custom stream implementation and writing it with =
proper handling of `pause/resume` functionality seems not a very easy =
task.
>>=20
>> Plain callbacks seems simpler to me, can streams be somehow wrapped =
into a code like that ( code with highlighting =
https://gist.github.com/3883920 ) ?
>>=20
>>     var copy =3D function(inputStream, outputStream, callback){
>>       var copyNextChunk =3D function(){
>>         inputStream.read(fuction(err, chunk){   =20
>>           if(err) return callback(err)
>>           // When chunk =3D=3D null there's no data, copying is =
finished.
>>           if(!chunk) return callback()
>>           outputStream.write(chunk, function(err){
>>             // Callback called only when chunk of data=20
>>             // delivered to the recipient and
>>             // we can send another one.
>>             if(err) return callback(err)
>>             copyNextChunk()
>>           }) =20
>>         })
>>       }
>>     }
>>=20
>> --=20
>> Job Board: http://jobs.nodejs.org/
>> Posting guidelines: =
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>> You received this message because you are subscribed to the Google
>> Groups "nodejs" group.
>> To post to this group, send email to nod...@googlegroups.com
>> To unsubscribe from this group, send email to
>> nodejs+un...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/nodejs?hl=3Den?hl=3Den
>=20
>=20
> --=20
> Job Board: http://jobs.nodejs.org/
> Posting guidelines: =
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nodejs@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+unsubscribe@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=3Den?hl=3Den


--Apple-Mail=_87558070-80BD-4546-BEF5-BF372BBAD744
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=iso-8859-1

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Diso-8859-1"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space; =
"><div>i'm not engaging with your strawman =
Bruno.</div><div><br></div><div>i showed how we *actually* move data in =
node. this is not a debate, that's how it works. if anyone wants to use =
node, or write a module that has a stream that moves data, that's how =
they do it.</div><div><br></div><div>this was a question, not an open =
invitation for bikeshedding. please let the list answer =
questions.</div><br><div><div>On Oct 13, 2012, at October 13, 20128:04 =
PM, Bruno Jouhier &lt;<a =
href=3D"mailto:bjouh...@gmail.com">bjouh...@gmail.com</a>&gt; =
wrote:</div><br class=3D"Apple-interchange-newline"><blockquote =
type=3D"cite">What's wrong? <br><br>You'll find links to gists at the =
end of my post. The code works!<br>And Alexey's pumping function is =
equivalent to the pumping loop I gave in my post.<br><br>On Saturday, =
October 13, 2012 7:31:54 PM UTC+2, Mikeal Rogers wrote:<blockquote =
class=3D"gmail_quote" style=3D"margin: 0;margin-left: 0.8ex;border-left: =
1px #ccc solid;padding-left: 1ex;"><div =
style=3D"word-wrap:break-word"><div>all of these are =
wrong.</div><div><br></div><div>inputStream.pipe(outputStream)</div><div>o=
utputStream.on('close', callback)</div><br><div><div>On Oct 13, 2012, at =
October 13, 20127:23 PM, Bruno Jouhier &lt;<a href=3D"javascript:" =
target=3D"_blank" =
gdf-obfuscated-mailto=3D"QQvY7lyp9CUJ">bjou...@gmail.com</a>&gt; =
wrote:</div><br><blockquote type=3D"cite">I wrote a post about plain =
callback APIs for streams: <a =
href=3D"http://bjouhier.wordpress.com/2012/07/04/node-js-stream-api-events=
-or-callbacks/" =
target=3D"_blank">http://bjouhier.wordpress.com/<wbr>2012/07/04/node-js-st=
ream-api-<wbr>events-or-callbacks/</a><br><br><br>On Saturday, October =
13, 2012 11:17:13 AM UTC+2, Alexey Petrushin wrote:<blockquote =
class=3D"gmail_quote" style=3D"margin:0;margin-left:0.8ex;border-left:1px =
#ccc solid;padding-left:1ex"><div>I don't quite understand how steam =
pause/resume works, or more exactly - how to use it in&nbsp;simple =
manner. It's necessary to use it in situations when the read stream =
produces data faster than the write stream =
can&nbsp;consume.&nbsp;</div><div><br></div><div>I need to write custom =
stream implementation and writing it with proper handling of =
`pause/resume` functionality seems not a very easy =
task.</div><div><br></div><div>Plain callbacks seems simpler to me, can =
streams be somehow wrapped into a code like that ( code with =
highlighting <a href=3D"https://gist.github.com/3883920" =
target=3D"_blank">https://gist.github.com/<wbr>3883920</a> ) =
?</div><div><br></div><div>&nbsp; &nbsp; var copy =3D =
function(inputStream, outputStream, callback){</div><div>&nbsp; &nbsp; =
&nbsp; var copyNextChunk =3D function(){</div><div>&nbsp; &nbsp; &nbsp; =
&nbsp; inputStream.read(fuction(err, chunk){ &nbsp; =
&nbsp;</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; if(err) return =
callback(err)</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; // When chunk =
=3D=3D null there's no data, copying is finished.</div><div>&nbsp; =
&nbsp; &nbsp; &nbsp; &nbsp; if(!chunk) return =
callback()</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; =
outputStream.write(chunk, function(err){</div><div>&nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; // Callback called only when chunk of =
data&nbsp;</div><div>&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; // =
delivered to the recipient and</div><div>&nbsp; &nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; // we can send another one.</div><div>&nbsp; &nbsp; &nbsp; =
&nbsp; &nbsp; &nbsp; if(err) return callback(err)</div><div>&nbsp; =
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; copyNextChunk()</div><div>&nbsp; =
&nbsp; &nbsp; &nbsp; &nbsp; }) &nbsp;</div><div>&nbsp; &nbsp; &nbsp; =
&nbsp; })</div><div>&nbsp; &nbsp; &nbsp; }</div><div>&nbsp; &nbsp; =
}</div></blockquote><div><br></div>

-- <br>
Job Board: <a href=3D"http://jobs.nodejs.org/" =
target=3D"_blank">http://jobs.nodejs.org/</a><br>
Posting guidelines: <a =
href=3D"https://github.com/joyent/node/wiki/Mailing-List-Posting-Guideline=
s" =
target=3D"_blank">https://github.com/joyent/<wbr>node/wiki/Mailing-List-<w=
br>Posting-Guidelines</a><br>
You received this message because you are subscribed to the Google<br>
Groups "nodejs" group.<br>
To post to this group, send email to <a href=3D"javascript:" =
target=3D"_blank" =
gdf-obfuscated-mailto=3D"QQvY7lyp9CUJ">nod...@googlegroups.com</a><br>
To unsubscribe from this group, send email to<br>
<a href=3D"javascript:" target=3D"_blank" =
gdf-obfuscated-mailto=3D"QQvY7lyp9CUJ">nodejs+un...@<wbr>googlegroups.com<=
/a><br>
For more options, visit this group at<br>
<a href=3D"http://groups.google.com/group/nodejs?hl=3Den?hl=3Den" =
target=3D"_blank">http://groups.google.com/<wbr>group/nodejs?hl=3Den?hl=3D=
en</a><br>
</blockquote></div><br></div></blockquote><div><br =
class=3D"webkit-block-placeholder"></div>

-- <br>
Job Board: <a =
href=3D"http://jobs.nodejs.org/">http://jobs.nodejs.org/</a><br>
Posting guidelines: <a =
href=3D"https://github.com/joyent/node/wiki/Mailing-List-Posting-Guideline=
s">https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines</a>=
<br>
You received this message because you are subscribed to the Google<br>
Groups "nodejs" group.<br>
To post to this group, send email to <a =
href=3D"mailto:nodejs@googlegroups.com">nodejs@googlegroups.com</a><br>
To unsubscribe from this group, send email to<br>
<a =
href=3D"mailto:nodejs+unsubscribe@googlegroups.com">nodejs+unsubscribe@goo=
glegroups.com</a><br>
For more options, visit this group at<br>
<a =
href=3D"http://groups.google.com/group/nodejs?hl=3Den?hl=3Den">http://grou=
ps.google.com/group/nodejs?hl=3Den?hl=3Den</a><br>
</blockquote></div><br></body></html>=

--Apple-Mail=_87558070-80BD-4546-BEF5-BF372BBAD744--
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
