Google Groups

How to signal to a readable stream to stop reading (and close)?


Michael Hart Jun 28, 2013 12:01 AM
Posted in group: nodejs
Hi all,

I'm struggling with a fairly basic problem to do with limiting (ie, truncating) streams and signalling to the source readable to stop reading and close.

Here's a simple example that hopefully illustrates what I mean:

fs.createReadStream('myHugeFile')
  .pipe(someTransform)
  .pipe(someTruncatingStream)
  .pipe(process.stdout)

Where someTruncatingStream wants to only take the first n bytes of the transformed stream, and then signal "hey, I'm done (ie, no need to keep reading the massive file)".

Is there a way to do this from someTruncatingStream without it having direct access to the source read stream? ie, some way to signal up the pipe to stop reading and close everything down (ie, not just pause)?

In my case it's actually not a file stream that I want to do this for - I just figured that was the easiest illustration. Mine is an object stream I want to create in front of a DB which is being queried a page at a time - I want to expose a stream for this that can continue paging behind the scenes and spitting out data, but that knows to stop paging when the consumers have consumed all the data they want. ie:

db.createReadStream({some: query}).pipe(someOtherFilter).pipe(stringify).pipe(take100).pipe(process.stdout)

I could use another paradigm for this, a lazy collection for example - but it seems to me that streams should support this and are more of a lingua franca to expose from a module API (and indeed can be transformed into lazy collections themselves, a la lazy.js/node-lazy)

Cheers,

Michael