Difference stream.pipe and util.pump?

[Tom]

Hi folks,

What is the difference between using stream.pipe and util.pump?

I'm running on a very low memory VPS, and my goal is to not upgrade
it's memory for as long as possible by optimizing memory usage.

The only thing my server has to do is accept incoming streaming data
from one client and immediately (without any processing or storage)
pipe/pump (not sure what to use now) it to a target stream (which is
another client).

Should I use stream.pipe or util.pump? Finally, how do I make sure
that the broadcasting stream is actually pausing and if it refuses to
pause how can I detect this refusal so that I can close the
connection?

Any feedback is appreciated.

- Tom

[Marco Rogers]

stream.pipe is now the preferred method.  util.pump was the original and was very useful for testing proofing this method of connecting streams.  You can expect util.pump to go away in the future.  I also believe that any fixes or updates to this paradigm are being made in stream.pipe. I'm not sure if they're always back-ported to pump.

As for the pause() method, there is lots of discussion around it in recent months. See this thread

https://groups.google.com/forum/#!searchin/nodejs/morestreams/nodejs/yv6Dl-O-wYk/O4kXGB_3a1kJ

The short version is that a stream that is based on an underlying socket (e.g. net, http, etc) will always pause eventually. It's "eventually" because of the way node works with libev to watch for socket events, some data may be in the pipeline to be emitted before the pause takes effect. It's a little more complicated obviously, but that's a simple way to think about it. I hope others will cut me some slack :) Basically you can't count on pause to happen immediately. But it will happen.

The way pipe works, it manages the data flow between the writestream and readstream. Pause is only called when the writestream reports that it's buffer is full. As soon as the buffer drains, the pipe starts back up.  So in general, the amount of data you may need to buffer correlates with the write speed to your remote server.

If you have data about how much bandwidth you're transferring and how many pause and resume cycles you're seeing, that's helpful for the people who want to fine tune this process.

:Marco