Node buffers stdin when it's a pipe - turn this off?

[Adam Spragg]

Hi there,

I'm currently working on a system that passes javascript commands to node.js
through a pipe.

Node.js appears to buffer it's stdin under these circumstances, which is a
problem as I'm wanting to send one command, wait a few seconds, send another,
wait a bit more, send the next, etc... - but I want the first command to be
executed as soon as it's received.

To reproduce:

$ cat | node
setTimeout(function() { console.log('testing'); }, 10000);
console.log('start');
^D
start
testing
$

I can pause for as long as I like between the console.log(...) line and
hitting ^D, but the 'start' never comes up until I do, and 'testing' doesn't
appear until 10 seconds after that.

Is there a command line switch I can use to get it to turn this buffering off?
If not, where might I start looking through the code to patch my local copy
myself? I found what looks like the main event loop
"uv_run(uv_default_loop());" in node.cc:Node::Start(), but can't figure out
where to look next from there.

Thanks,

Adam Spragg

--
Adam Spragg <adam....@octaltelecom.co.uk>
Developer
Octal Telecom <http://www.octaltelecom.co.uk/>

It reverses the logical flow of conversation!
> Why?
> > No.
> > > Should I top post?
<http://www.google.com/search?q=%22top+posting%22>

[Ben Noordhuis]

On Mon, Apr 23, 2012 at 12:34, Adam Spragg
<adam....@octaltelecom.co.uk> wrote:
> I'm currently working on a system that passes javascript commands to node.js
> through a pipe.
>
> Node.js appears to buffer it's stdin under these circumstances, which is a
> problem as I'm wanting to send one command, wait a few seconds, send another,
> wait a bit more, send the next, etc... - but I want the first command to be
> executed as soon as it's received.
>
> To reproduce:
>
>  $ cat | node
>  setTimeout(function() { console.log('testing'); }, 10000);
>  console.log('start');
>  ^D
>  start
>  testing
>  $
>
> I can pause for as long as I like between the console.log(...) line and
> hitting ^D, but the 'start' never comes up until I do, and 'testing' doesn't
> appear until 10 seconds after that.
>
> Is there a command line switch I can use to get it to turn this buffering off?
> If not, where might I start looking through the code to patch my local copy
> myself? I found what looks like the main event loop
> "uv_run(uv_default_loop());" in node.cc:Node::Start(), but can't figure out
> where to look next from there.

Look for the _normalWrite() method in lib/readline.js

[Nathan Rajlich]

Basically what is happening is node is expecting you to be piping an entire file into stdin, so it does indeed buffer and wait for EOF before executing the script passed in from stdin.

But it sounds like maybe your're expecting to enter the REPL in this case, and enter commands one-at-a-time? I'm node v0.7.7 the "-i" flag was added to make this case work. So with node v0.7.7 or above:

  $ cat | node -i

But this "buffering" behavior in general doesn't happen, it's only because you were attempting to pipe a script into node with no script file to run itself, so it waits for a script from stdin and executes that. If you try the same thing again with a simple script (test.js):

  process.stdin.setEncoding('utf8')
  process.stdin.resume()

  process.stdin.on('data', function (command) {

    console.log('got command: %s', command.trim())

  })

and run it:

  $ cat | node test.js

Then you can enter commands one-line-at-a-time.

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[Adam]

Hi Nathan,

On Apr 23, 4:07 pm, Nathan Rajlich <nat...@tootallnate.net> wrote:
> But it sounds like maybe your're expecting to enter the REPL in this case,
> and enter commands one-at-a-time?

I have a program occasionally piping commands, one-at-a-time, into
node's stdin which I want to be executed as soon as possible. Like in
the REPL. The commands need to be executed in the same node instance,
as they share state, so I can't just close the pipe and start a new
node for subsequent commands. I don't need the REPL prompt, but it
won't do any harm.

> I'm node v0.7.7 the "-i" flag was added
> to make this case work. So with node v0.7.7 or above:
>
>   $ cat | node -i

I think that sounds like what I need. But... that's the unstable
branch. Do you have any experience with that? How unstable is it? Is
it likely to crash on me or cause other problems?

> But this "buffering" behavior in general doesn't happen, it's only because
> you were attempting to pipe a script into node with no script file to run
> itself, so it waits for a script from stdin and executes that. If you try
> the same thing again with a simple script (test.js):
>
>   process.stdin.setEncoding('utf8')
>   process.stdin.resume()
>   process.stdin.on('data', function (command) {
>     console.log('got command: %s', command.trim())
>   })
>
> and run it:
>
>   $ cat | node test.js

So I could just change the console.log() to an eval() in that test.js
and that should work.

OK, thanks.

Adam