Trying to scale concurrent TLS/SSL connections to > 100k

[Leif Jackson]

All,

I am working on a project where I would like to be able to use
nodejs as my platform, the event driven IO model of nodejs appears to
work really well for non-encrypted streams. When encryption is enabled
the nodejs process cannot handle very many concurrent connections, 2k
give or take, before CPU is pegged. Without encryption as a raw tcp
server it appears to be able to handle > 65k. To make nodejs viable for
this project I really need to be able to handle 100k+ per nodejs process
and thus 400k / box (4 core vm...etc) in the cloud with some headroom in
each process.

I am asking the community if anyone else has run into this issue
and if they have any ideas they have been toying with for a
architecture/design. My current theory, after testing as libev based tls
server communicating with nodejs via unix socket, would be to create an
implementation for the tls library or as an addition/bolt-on module in
c/c++. This module would handle all the TLS streams inside of a
eventloop (libev to start then maybe libuv, but libuv cannot manage fd's
directly due to abstraction and windows interplay as I seems?) and feed
the data/messages as nodejs Buffers to a callback. Ideally it would be a
drop in replacement for the tls library but I am not sure that is
possible currently with libuv due to the fd issue.

If it doesn't automatically enhance the tls library then the
Callback /API for this module would be something like
server.on("message", function(data, info) { }); where data is nodejs
Buffer object and info is object/hash with routing info e.g. src IP
address and port and maybe a handle to send a message back to that
client async? The info hash may have to have a session/guid to allow for
a api like server.send(sessionId, data) where data is also a nodejs
Buffer. However sessionId is probably a misnomer as it would be more
like connectionId which would/could relate directly to the fd under unix
that is is connected with.

My use case requires only TLS streams and inside that will be a
custom binary protocol, the main issue appears to be the amount of cpu
time consumed by shuffling the buffers to and from the google v8 engine?
And not as much with the cpu time required to perform the crypto.

Thoughts?

Thanks,
Leif Jackson

[Александр Лозовюк]

maybe, use nginx to ssl connect and proxy plain http to node.js backend. 

2011/12/5 Leif Jackson <ljac...@jjcons.com>

Leif Jackson

--
Job Board: http://jobs.nodejs.org/
Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--
C уважением, Александр Лозовюк
Alpha-Beta-Release Blog
http://abrdev.com

http://asyncionews.com

[Jaromír Müller]

Hi Leif,

had you any luck with this?

I'm not sure what is better solution whether to have node+ssl or nginx+ssl as frontend for node.

In our scenario there is possibility to have 100k+ websocket connections and having nginx in front of node would mean usage of another memory for connection between node and nginx (I guess).

Thank you,

Jaro.

[Matt]

On Mon, Feb 3, 2014 at 12:44 PM, Jaromír Müller <jaromir...@gmail.com> wrote:

In our scenario there is possibility to have 100k+ websocket connections and having nginx in front of node would mean usage of another memory for connection between node and nginx (I guess).

If you're doing this make sure you use a Unix Domain socket for the backend node process or you're limited to the 64k TCP connections on one port/ip.