Error: spawn EMFILE
5,695 views
Skip to first unread message
Joshua Kehn
Feb 10, 2012, 10:58:27 AM2/10/12
to nod...@googlegroups.com
I'm getting some peculiar errors when using `child_process.exec` or `child_process.spawn`. See https://gist.github.com/1790433
Essentially creating several spawn/exec calls in a loop will error out with a EMFILE error (too many file descriptors open). And simple advice on avoiding this without say manually managing how many spawns I have running at any given time?
Ben Noordhuis
Feb 10, 2012, 11:02:43 AM2/10/12
to nod...@googlegroups.com
On Fri, Feb 10, 2012 at 16:58, Joshua Kehn <josh...@gmail.com> wrote:
> I'm getting some peculiar errors when using `child_process.exec` or
> `child_process.spawn`. See https://gist.github.com/1790433
>
> Essentially creating several spawn/exec calls in a loop will error out with
> a EMFILE error (too many file descriptors open). And simple advice on
> avoiding this without say manually managing how many spawns I have running
> at any given time?
> I'm getting some peculiar errors when using `child_process.exec` or
> `child_process.spawn`. See https://gist.github.com/1790433
>
> Essentially creating several spawn/exec calls in a loop will error out with
> a EMFILE error (too many file descriptors open). And simple advice on
> avoiding this without say manually managing how many spawns I have running
> at any given time?
Bump the file descriptor limit: ulimit -n 16384
Joshua Kehn
Feb 10, 2012, 11:58:13 AM2/10/12
to nod...@googlegroups.com
Ulimit already says unlimited. http://cl.ly/2n2O2g470d2g0Q1S3N20
--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+un...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
Matt
Feb 10, 2012, 12:12:21 PM2/10/12
to nod...@googlegroups.com
There are always limits (sadly).
Ben Noordhuis
Feb 10, 2012, 12:12:42 PM2/10/12
to nod...@googlegroups.com
On Fri, Feb 10, 2012 at 17:58, Joshua Kehn <josh...@gmail.com> wrote:
> Ulimit already says unlimited. http://cl.ly/2n2O2g470d2g0Q1S3N20
> Ulimit already says unlimited. http://cl.ly/2n2O2g470d2g0Q1S3N20
That's the file size limit, not the file descriptor limit.
Joshua Kehn
Feb 10, 2012, 12:14:16 PM2/10/12
to nod...@googlegroups.com
Either way, this is what I get when setting it:
limit: setrlimit failed: invalid argument
[7679]: ulimit -n 1200
[7680]: ulimit
unlimited
[7681]:
Joshua Kehn
Feb 10, 2012, 12:15:42 PM2/10/12
to nod...@googlegroups.com
And even if I assume it *is* setting something, ulimit -n 10000 doesn't change the error I'm seeing.
On Feb 10, 2012, at 12:12 PM, Matt wrote:
Ben Noordhuis
Feb 10, 2012, 12:21:42 PM2/10/12
to nod...@googlegroups.com
On Fri, Feb 10, 2012 at 18:15, Joshua Kehn <josh...@gmail.com> wrote:
> And even if I assume it *is* setting something, ulimit -n 10000 doesn't
> change the error I'm seeing.
> And even if I assume it *is* setting something, ulimit -n 10000 doesn't
> change the error I'm seeing.
Let me guess, you're working on a Mac? You need to log in again after
changing the ulimit on OS X, don't ask me why.
Matt
Feb 10, 2012, 12:24:05 PM2/10/12
to nod...@googlegroups.com
It also has stupidly low maximums for the OS set by default in sysctl.
Matt.
Joshua Kehn
Feb 10, 2012, 12:25:00 PM2/10/12
to nod...@googlegroups.com
Actually I found out that ulimit is only a modification for the shell session. If I run `ulimit -n 10000` in the same session as the node process it works (or appears to).
Martin Cooper
Nov 15, 2012, 11:30:29 AM11/15/12
to nod...@googlegroups.com
On Thu, Nov 15, 2012 at 8:24 AM, James Weston <jawo...@gmail.com> wrote:
I was running into the same problem and wrote a package to queue up the spawn requests. It can be found at https://github.com/jaw187/queueit
The 'async' package already has:
--
Martin Cooper
Reply all
Reply to author
Forward
0 new messages