Limit and handling of spawning child_process of ffmpeg

[Ofir Attia]

Hello,

I facing with issue of nodejs and spawning child process.

I have noticed that I can spawn at max 5 instances of ffmpeg instances.

I have a server that listen for requests and each request parsed as ffmpeg instance that working for 5 minutes.

The problem is that the number of the instances is limited for 5 I cant get more instances.

 I would like to get some advice in this subject, if you faced with this issue ( Not necessarily with ffmpeg ) .

Thanks.

[Ryan Graham]

Are you using a client you wrote yourself in node? This sounds more like the maxSockets problem in the default http.Agent in node 0.10.x.

~Ryan

--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/8b4b24c8-20b9-4cc7-b5ce-f1286874c479%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ryan Schmidt]

The web server probably shouldn't spawn long-running processes. Instead, have a separate process to do long-running work, and find a way to communicate between the web server process and the long-running work process. For example, using redis is popular. npm has several modules for helping you manage a work queue.

However... I'm not sure why spawning more than 5 processes from a web server would be a problem. I'd need to see some code demonstrating the problem.

[Ofir Attia]

The server is waiting for request, for each request I am take a video input ( it can be RTSP input, video file .. ) and output it as 1 image.

for example - 

this.stream = child_process.spawn("ffmpeg", [ "-i", this.url, "-rtsp_transport", "tcp",'-f', 'mpeg1video', '-b:v', '800k', '-r', '30', '-'], {
      detached: false
    });
    
    this.inputStreamStarted = true;
    this.stream.stdout.on('data', function(data) {
      return self.emit('mpeg1data', data);
    });
    this.stream.stderr.on('data', function(data) {
      return self.emit('ffmpegError', data);
    });

this is part of the code in the object I wrote, just spawning this child process and wait for events in the wrapper.

Its not a problem that the solution is clustering or queue because each request execute at the moment its arrive. When I open the Task Manager

I can see only 5 process of ffmpeg.exe , I tried to create manually the process from the cmd line and still, the maximum is 4~5 processes.

Any Idea?

[Matt]

You've exhausted the threadpool. Try setting the UV_THREADPOOL_SIZE env var higher and see if you can spawn more.

--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/bd610254-3ddb-4fde-ab9b-9614f58284e0%40googlegroups.com.

[Sam Roberts]

Can you provide minimal code to reproduce this problem? I could try
and build it from your example, but don't have time, if you gave me a
snippet I could just run, I would, and could see what your issue is.

On Mon, Jun 29, 2015 at 10:37 AM, Matt <hel...@gmail.com> wrote:
> You've exhausted the threadpool. Try setting the UV_THREADPOOL_SIZE env var
> higher and see if you can spawn more.

Child processes don't go through the thread pool.

The spawn is synchronous, the IPC (if any) on stdio is async, uses
pipes, and goes through the epoll() event loop.

--
http://StrongLoop.com makes it easy to develop APIs in Node, plus get
DevOps capabilities like monitoring, debugging and clustering.

[Matt]

On Sun, Jul 5, 2015 at 6:12 PM, Sam Roberts <s...@strongloop.com> wrote:

On Mon, Jun 29, 2015 at 10:37 AM, Matt <hel...@gmail.com> wrote:
> You've exhausted the threadpool. Try setting the UV_THREADPOOL_SIZE env var
> higher and see if you can spawn more.

Child processes don't go through the thread pool.

The spawn is synchronous, the IPC (if any) on stdio is async, uses
pipes, and goes through the epoll() event loop.

Are you sure? I haven't gone through the Node source, but I would assume that the wait/waitpid calls can't be done without the thread pool. Of course I could be entirely wrong about that, and the calls could just wait on the closing of the fd instead. But it seemed like a logical conclusion.

[Sam Roberts]

I am sure.

That it isn't async (in a thread pool) can be observed:

child = cp.spawn(...)
console.log(child.pid)

prints a pid... but those two lines are sync, of course, there is no
callback, right? Well, that means that the fork() at least must have
happened synchronously, because until fork returns there is no pid.
libuv actually blocks a bit longer than just the fork, it waits for
the child to do exec, too, to guarantee the success of the exec, using
a cool trick with pipes and close-on-exec, and that part of the
process is nicely commented:

https://github.com/libuv/libuv/blob/v1.x/src/unix/process.c#L422-L441

And you mention waitpid going through a thread pool, but it does not,
waitpid() doesn't block (not as called by libuv, with WNOHANG), so it
doesn't need to go through a thread pool. Also, even if it did, it
would just mean that the maximum number of child processes that could
be **waited on simultaneously** would be limited to the thread pool
size, not that the maximum number of processes would be capped at
that.

Cheers,
Sam

[Matt]

Thanks for the clarification Sam.

--
Job board: http://jobs.nodejs.org/
New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
---
You received this message because you are subscribed to the Google Groups "nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nodejs+un...@googlegroups.com.
To post to this group, send email to nod...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/CACmrRmQ1rT1QLPNm9KN7xVZ9G98hEjTGGcyP3ijs5_PEF4%2B2Sg%40mail.gmail.com.

[Ofir Attia]

I am waiting for requests, you can parse it as for loop that create those objects -

new Mpeg1Muxer({
           url: this.streamUrl,
  timer: this.timer,
  name: this.name,
  sid : this.stream_id
    });

And the implementation of this object is :

(function() {
  var Mpeg1Muxer, child_process, events, util;


  child_process = require('child_process');


  util = require('util');


  events = require('events');


  Mpeg1Muxer = function(options) {
    var self;
    self = this;
    this.url = options.url;
 this.timer = options.timer;
 this.name = options.name;
 this.sid = options.sid;
 var fs = require('fs');
 var dir = 'LOCATION_DIR';


 if (!fs.existsSync(dir)){
 fs.mkdirSync(dir);
 }
    this.stream = child_process.spawn("ffmpeg",['-i',this.url,'-t',this.timer/1000,'-preset','ultrafast','-threads','2','-r',(1/3),'-an','-y','-update','1','SOME_DIR/1.jpg'] , {
      detached: false

    });
    this.inputStreamStarted = true;
    this.stream.stdout.on('data', function(data) {
 
      return self.emit('mpeg1data', data);
    });
    this.stream.stderr.on('data', function(data) {
 
      return self.emit('ffmpegError', data);
    });

    return this;
  };


  util.inherits(Mpeg1Muxer, events.EventEmitter);


  module.exports = Mpeg1Muxer;


}).call(this);

this is not the problem of the implementation in the code, its problem of the limitation of "FFMPEG" processes that I can create.

If you have other solutions, I would like to hear! 

thanks anyway.

[Sam Roberts]

That's not a standalone reproduction. If you want to debug this,
create a smaller example. In the process of doing so, you'll probably
figure out what you are doing wrong. If not, someone will be able to
help you by running your complete code.

Here I start 10 ffmpeg in parallel, at end, you can see I hit ctrl-c,
and they all exit.

% node ffmpeg.js
ffmpeg -i http://127.0.0.1:38203 -v warning -an -y -update 1 1.jpg
0 'pid:' 26260
1 'pid:' 26261
2 'pid:' 26262
3 'pid:' 26263
4 'pid:' 26264
5 'pid:' 26265
6 'pid:' 26266
7 'pid:' 26267
8 'pid:' 26268
9 'pid:' 26269
new request!
new request!
new request!
new request!
new request!
new request!
new request!
new request!
new request!
new request!
http://127.0.0.1:38203: Immediate exit requested
http://127.0.0.1:38203: Immediate exit requested
http://127.0.0.1:38203: Immediate exit requested
http://127.0.0.1:38203: Immediate exit requested
http://127.0.0.1:38203: Immediate exit requested
http://127.0.0.1:38203: Immediate exit requested
http://127.0.0.1:38203: Immediate exit requested
http://127.0.0.1:38203: Immediate exit requested
http://127.0.0.1:38203: Immediate exit requested
http://127.0.0.1:38203: Immediate exit requested


my code:

var spawn = require('child_process').spawn;
var server = require('http').createServer(connection).listen(0);

function connection(req) {
console.log('new request!');
}

server.on('listening', function() {
var url = 'http://127.0.0.1:' + this.address().port;

var args = [
'-i', url,
'-v', 'warning',
// '-preset', 'ultrafast',
// '-threads', '2',
// '-r', (1/3),

'-an', '-y',
'-update', '1',

'1.jpg'
];

var ff = [];

console.log('ffmpeg %s', args.join(' '));

for (var i = 0; i < 10; i++) {
var c = spawn('ffmpeg', args, {stdio: 'inherit'});
ff.push(c);
console.log(i, 'pid:', c.pid);
}
});


Cheers,
Sam