exec node - spawn mode not working in Windows

[Mal Coltent]

Hi All,
Trying to run an executable that outputs results at intervals and trying to use "spawn mode" to get output while its running but does not seem to work as advertised.

Tried running in a function node with spawn call but still not able to get the output at intervals, only at command exit.

e.g.

const {spawn} = context.global.process; /* required module(s) already set up in setting.js*/

const exe = spawn('cmd', ['/s', '/c', '<insert command here>']);

exe.stdout.on('data', function(data) {
    console.log('stdout: ' + data);
});

Thanks.

[Julian Knight]

Have you tried the https://flows.nodered.org/node/node-red-node-daemon node?

[Mal Coltent]

Yes, tried that, same behavior.

even tried this ..

exe.stdout.on('readable', function() {
  while ((chunk = exe.stdout.read())!== null){
    console.log(`Received ${chunk.length} bytes of data.`);
  }
});

.. to see if data chunks are coming through but this snippet only fires when the process exits.
It should fire as soon as data is available at stdout of the process which is once a second.

But, if I replace the command with the system "ping" command, all works as expected.
Only thing I can think of is that ping returns <CR><LF> and the command I'm using probably only a <CR>.

This may be the default trigger for a "readable" event. Is there a way to force this to something else?

[Julian Knight]

You might try running PowerShell rather than cmd.exe to see if that helps?

[Mal Coltent]

OK. Tried powershell, Same behavior.

Also tried this

const cmd = "c:\\Users\\xxxxx\\Documents\\test.bat";

exe = spawn('cmd.exe', ['/c' ,cmd],
{
    stdio: 'pipe',
    shell: false,
    detached: false
});

exe.stdout.on('readable', (chunk) => {
  while (null !== (chunk = exe.stdout.read()) ){
    console.log('*******', chunk.toString(), '*******');
  }});

When test.bat runs the command I need to run, I get the command and its parameters ok but the output which is supposed to be at intervals only gets to stdout when the command exits. That is, rather than get 5 outputs per second and exit, I get an exit and one output after 5 seconds. However when I replace the command with "ping 127.0.0.1" I get the data at the intervals and then an exit.

I can set "detached" to true and see it on a console window at intervals just fine,
I set stdio to "inherit" and I can see it on the nodejs console at intervals just fine.

I just can't seem to consume the data inside nodejs itself at intervals which is what I need.

Any help appreciated.

[Julian Knight]

Clutching at straws now - in powershell, what happens when you try to pipe the output of your command to another powershell command? I'm wondering if there is a way to streamwise replace the cr with lf-cr (if that is the issue).

[Dave C-J]

I suspect it's more to do with the buffering of stdout... eg when running a python command you can specify -u to stop it buffering. 

[Julian Knight]

Yes, I was thinking it might buffer up to a crlf so manually adding the missing char might do the job? As I say, straws.

[Mal Coltent]

Surely I should be able to pipe the output of the child process directly to another node. Why does the detached console see that data immediately? How does that work?

[Mal Coltent]

I tried this to address the CR/LF issue but to no avail.

The 'line' event triggers on CR, LF or CR/LF

var child_process = require('child_process');

var readline = require('readline');

const test = "c:\\Users\\xxxxxx\\Documents\\test.bat";

var proc = child_process.spawn('cmd.exe', ['/c ', test],{

//stdio: 'pipe',

//shell: true,

//detached: true,

//windowsHide : true,

});

readline.createInterface({

input : proc.stdout,

terminal :false

}).on('line', function(line) {

console.log(line);

});

[Julian Knight]

I'm at a loss now I'm afraid. Without being able to see the program that is generating the output there isn't much more to say.

[Mal Coltent]

The program is iperf3 https://iperf.fr/iperf-download.php

Client command in a batch file "test.bat" is :

c:\\Users\\<your user id>\\iperf\\iperf3 -c <your server IP> -p 57444 -u -b 5M -t 5 -l 1024 -w 1M -i 0.1

Server command is :
iperf3 -s -p 57444 -V

Nodejs code is: 'process.js'

const spawn = require('child_process').spawn;

const iperf = "c:\\Users\\<your user id>\\Documents\\iperf\\test.bat";

const exe = spawn('cmd.exe', ['/c ', iperf],

{

//stdio: 'inherit',

//shell: true,

//detached: true,

//windowsHide : true,

});

exe.stdout.on('readable', () => {

let chunk;

while ( (chunk = exe.stdout.read()) !== null ) {

console.log(chunk.toString());

}

});

Objective here is to get the data in the callback as its being output per the -i switch, at intervals.

Un-comment the 'inherit' or the 'detached' line to see output in real time in a console,

but that data will not show up in in the callback routine i.e. 'readable' never triggers.

This needs to be in a function node in node-red that will send messages to the dashboard node(s).

Here is the output:

C:\Users\<user id>\Documents>c:\\Users\\<user id>\\Documents\\iperf\\iperf3 -c <your server> -p 57444 -u -b 5M -t 5 -l 1024 -w 1M -i 0.1
Connecting to host <your server>, port 57444
[  4] local <local IP> port 50022 connected to <remote IP> port 57444
[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-0.11   sec   643 KBytes  48.2 Mbits/sec  643
[  4]   0.11-0.20   sec  1.00 KBytes  87.5 Kbits/sec  1
[  4]   0.20-0.31   sec  1.00 KBytes  75.0 Kbits/sec  1
[  4]   0.31-0.41   sec  1.00 KBytes  87.5 Kbits/sec  1
[  4]   0.41-0.51   sec  1.00 KBytes  75.0 Kbits/sec  1
[  4]   0.51-0.61   sec  1.00 KBytes  87.5 Kbits/sec  1
[  4]   0.61-0.70   sec  1.00 KBytes  87.5 Kbits/sec  1
[  4]   0.70-0.81   sec  1.00 KBytes  75.0 Kbits/sec  1
[  4]   0.81-0.90   sec  1.00 KBytes  87.5 Kbits/sec  1
[  4]   0.90-1.01   sec  1.00 KBytes  75.0 Kbits/sec  1
[  4]   1.01-1.11   sec  1.00 KBytes  87.5 Kbits/sec  1
[  4]   1.11-1.20   sec  25.0 KBytes  2.19 Mbits/sec  25
[  4]   1.20-1.31   sec  57.0 KBytes  4.28 Mbits/sec  57
[  4]   1.31-1.40   sec  66.0 KBytes  5.78 Mbits/sec  66
[  4]   1.40-1.51   sec  57.0 KBytes  4.28 Mbits/sec  57
[  4]   1.51-1.61   sec  77.0 KBytes  6.74 Mbits/sec  77
[  4]   1.61-1.70   sec  47.0 KBytes  4.11 Mbits/sec  47
[  4]   1.70-1.81   sec  57.0 KBytes  4.28 Mbits/sec  57
[  4]   1.81-1.90   sec  67.0 KBytes  5.86 Mbits/sec  67
[  4]   1.90-2.01   sec  57.0 KBytes  4.28 Mbits/sec  57
[  4]   2.01-2.11   sec  76.0 KBytes  6.65 Mbits/sec  76
[  4]   2.11-2.22   sec   105 KBytes  7.88 Mbits/sec  105
[  4]   2.22-2.31   sec  10.0 KBytes   875 Kbits/sec  10
[  4]   2.31-2.40   sec  57.0 KBytes  4.99 Mbits/sec  57
[  4]   2.40-2.51   sec  57.0 KBytes  4.28 Mbits/sec  57
[  4]   2.51-2.61   sec  66.0 KBytes  5.78 Mbits/sec  66
[  4]   2.61-2.71   sec  58.0 KBytes  4.35 Mbits/sec  58
[  4]   2.71-2.81   sec  76.0 KBytes  6.65 Mbits/sec  76
[  4]   2.81-2.90   sec  47.0 KBytes  4.11 Mbits/sec  47
[  4]   2.90-3.01   sec  58.0 KBytes  4.35 Mbits/sec  58
[  4]   3.01-3.10   sec  66.0 KBytes  5.78 Mbits/sec  66
[  4]   3.10-3.21   sec  57.0 KBytes  4.28 Mbits/sec  57
[  4]   3.21-3.31   sec  67.0 KBytes  5.86 Mbits/sec  67
[  4]   3.31-3.40   sec  57.0 KBytes  4.99 Mbits/sec  57
[  4]   3.40-3.51   sec  57.0 KBytes  4.28 Mbits/sec  57
[  4]   3.51-3.60   sec  67.0 KBytes  5.86 Mbits/sec  67
[  4]   3.60-3.71   sec  57.0 KBytes  4.28 Mbits/sec  57
[  4]   3.71-3.81   sec  67.0 KBytes  5.86 Mbits/sec  67
[  4]   3.81-3.90   sec  57.0 KBytes  4.99 Mbits/sec  57
[  4]   3.90-4.01   sec  57.0 KBytes  4.28 Mbits/sec  57
[  4]   4.01-4.10   sec  67.0 KBytes  5.86 Mbits/sec  67
[  4]   4.10-4.21   sec  57.0 KBytes  4.28 Mbits/sec  57
[  4]   4.21-4.31   sec  66.0 KBytes  5.78 Mbits/sec  66
[  4]   4.31-4.41   sec  57.0 KBytes  4.28 Mbits/sec  57
[  4]   4.41-4.51   sec  67.0 KBytes  5.86 Mbits/sec  67
[  4]   4.51-4.60   sec  57.0 KBytes  4.99 Mbits/sec  57
[  4]   4.60-4.71   sec  57.0 KBytes  4.28 Mbits/sec  57
[  4]   4.71-4.80   sec  67.0 KBytes  5.86 Mbits/sec  67
[  4]   4.80-4.91   sec  57.0 KBytes  4.28 Mbits/sec  57
[  4]   4.91-5.01   sec  67.0 KBytes  5.86 Mbits/sec  67
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datag
rams
[  4]   0.00-5.01   sec  2.93 MBytes  4.91 Mbits/sec  0.000 ms  0/3000 (0%)
[  4] Sent 3000 datagrams

iperf Done.

Any help appreciated.

[Dave C-J]

may be worth trying the last suggestion (bottom of page) here - https://stackoverflow.com/questions/15339379/node-js-spawning-a-child-process-interactively-with-separate-stdout-and-stderr-s

[Mal Coltent]

Thanks for the suggestion but it requires a unix tool, 'stdbuf'. There is a windows version but it gets messy. If I have no other recourse I'll give it a try.

[Dave C-J]

from the iperf docs...

[Mal Coltent]

Hi Dave,
Which version of iperf?
Cant seem to find that switch in the version I'm using.

[Dave C-J]

The one on github

https://github.com/esnet/iperf/blob/master/docs/invoking.rst

[Julian Knight]

OK, so now we know what you are trying to do. Have you tried using an alternative client?

Not sure if this will do what you want but might be worth a try?

Unfortunately, forceflush doesn't help. I've just tried it.

I think that part of the problem is that this is actually a Linux app that has been cross-compiled to Windows using cygwin. If desparate, maybe you could try using the W10 Ubuntu Subsystem. That is real Linux on Windows so should be able to run the Debian/Ubuntu version of the client. You can also run Node-RED under it as well of course if you want to.