Concurrent connections limit / memory usage

272 views
Skip to first unread message

Ricardo Tomasi

unread,
Nov 24, 2009, 12:31:17 PM11/24/09
to nodejs
I was doing some testing today and couldn't go above 1017 concurrent
connections. I used Richard Jones' Erlang flood test script for that
(http://www.metabrew.com/article/a-million-user-comet-application-with-
mochiweb-part-1/). Node kept using more memory but wouldn't accept new
connections.

Also, the requests were set to finish after 30 seconds using a
timeout, but even a few minutes after the test was halted memory use
wouldn't come down. Any insights?

thanks,
Ricardo

Daniel Bartlett

unread,
Nov 24, 2009, 2:29:42 PM11/24/09
to nod...@googlegroups.com
Hi Ricardo,

This is due to ulimit settings. On most Linux systems there are limits on how many files/sockets you can open. By default this is 1024.

Applications that require more than this are generally started as root, with a higher ulimit set then drop back to the user account.

This example script should explain:

[root@x ~]# ./ulimit_test.sh
Inside startup script:
I am running as root and I have 1024 sockets available.
Setting ulimit...
I am running as root and I have 10240 sockets available.
Switching to user...
I am running as dan and I have 10240 sockets available.
[root@x ~]#

[root@x ~]# cat ulimit_test.sh
#!/bin/bash

echo "Inside startup script:"
/tmp/daemon.sh
echo "Setting ulimit..."
ulimit -n 10240
/tmp/daemon.sh

echo "Switching to user..."
su - dan -c /tmp/daemon.sh

[root@x ~]# cat /tmp/daemon.sh
#!/bin/bash

echo "I am running as $( id -un ) and I have $( ulimit -n ) sockets available.";

Obviously you would only need a start up script with the ulimit -n 10240 and su - <user> -c <command> :)

I hope that makes sense!



Cheers,

DanB.
> --
>
> You received this message because you are subscribed to the Google Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com.
> To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.
>
>

Ricardo Tomasi

unread,
Nov 24, 2009, 3:31:20 PM11/24/09
to nodejs
My ulimit is set to "unlimited". I tried setting ulimit -n 999999 to
no effect. Also tried running both node and the erlang runtime as
root... using apache instead bench I get the "Too many open files"
error, but I already modified my config as per the article.

thanks a lot for your help,
ricardo

Daniel Bartlett

unread,
Nov 24, 2009, 3:39:25 PM11/24/09
to nod...@googlegroups.com
If you just run ulimit on it's own, it will will say unlimited, but that isn't the open files.

Even if you are root, the default ulimit on open files (network sockets) is 1024. You can see all your defaults||current settings with a ulimit -a

Can I suggest you open up to consoles, sudo to root in both, run ulimit -n 999999 in both and then in 1 the server and the 2nd the load test side. That should work.

Cheers,

DanB.
Reply all
Reply to author
Forward
0 new messages