Node + Spawn a child process once and then provide stdin's and read stdout's from it

[S.Sai Prashath]

I'm spawning a child process using spawn-command npm package, i do this when the node server starts and then for every request i read the query value and hit the running child process with stdin. The stdout that comes out of the child process is an event stream and i add a listener to read the data and then will be sending the data to sse-channel instance.

Everything works the way it should for the first request, the issue arises for subsequent requests. When i console.log the data inside the stdout's on listener handler then i could see that the data is getting cloned. Meaning, same set of data is getting repeated twice for the second request and thrice for the third request etc... I was researching this issue and possible solution for this would be to create independent streams for each request so that this data cloning would be avoided in the child.stdout's on listener handler. But i'm not sure whether this could resolve the issue. Could some one please suggest a way to overcome this hurdle.

Code looks like below,

var spawnCommand = require('spawn-command');
var cmd = 'path to the binary file'; 
 module.exports = function (app) {
  var child     = spawnCommand(cmd);
  srvc = {
   get: function(req, res) {
     child.stdin.write('a json object in a format that is expected by binary' + '\n');
     child.stdout.on('data', function() {
       console.log(''+ data);
     });
   }
  }
}

As i mentioned earlier the child process would be spawned once when the node server starts and then the get() method would run for every request. So each time when a request runs, the stdin's would be sent to the binary and then the "on" listener would log the data in stdout. This works perfectly for the first request, but when a second request is made (with the first request still logging data) i could see that the data's for both first and second are getting cloned twice and for third request the data's for each request is getting cloned thrice and it increases exponentially.

On the other hand if i spawn a child everytime inside the get() method i can overcome this problem but unfortunately i cannot do that because there'll be 100's of requests and i cannot afford 100's of binary instance eating up the memory.

Hope i explained it better, any suggestions would be of great help.

[Ryan Schmidt]

I don't see how you can avoid spawning a new instance of the binary for each request. Otherwise, how are you going to keep straight which output from a single instance of the binary should be sent to which request?

If you cannot afford that many binaries running at once, use a queue or pool or other similar system that would limit the number of processes that can be spawned simultaneously. If too many binaries are already running, any additional requests would have to wait until those binaries quit and new ones can be started.