Deploying SockJS server(s) behind AWS ELB or other proxies

[onie....@gmail.com]

Hi folks,

I'm going to be building an app which uses sockjs client and server on nodejs. We are using Amazon Web Services and ELB. I would like to know if SockJS will work happily in an ELB environment. If not, how does it work with Nginx? What about HAproxy? We will have many users which eventually might reach upto 10k users hopefully. Which technology would you recommend? I would like to implement a very stable solution.

Any help would be greatly appreciated!

Thanks in advance!

Neil

[Alec Embke]

Hey there, I'm not a member of the sockjs group but we've been using sockjs on node on AWS behind an ELB for a few months now without any issues. Just make sure the load balancer protocol is on the transport layer (tcp and/or secure tcp) and not the application layer (http or https).

[recta...@gmail.com]

On Monday, May 12, 2014 4:30:50 PM UTC-4, Alec Embke wrote:

Hey there, I'm not a member of the sockjs group but we've been using sockjs on node on AWS behind an ELB for a few months now without any issues. Just make sure the load balancer protocol is on the transport layer (tcp and/or secure tcp) and not the application layer (http or https).

ELB doesn't support session stickiness for TCP listeners, and sockjs needs that for websocket emulation.

https://github.com/sockjs/sockjs-client/wiki/%5BArticle%5D-SockJS:-WebSocket-emulation-done-right#use-almost-any-load-balancer

We are running sockjs behind a TCP ELB and having issues when clients fall back to websocket emulation since the session is not sticky. I think we will have to add an HTTP listener with stickiness to the ELB and configure the clients to use that port when they fallback to emulation.

[Bryce Kahle]

Stickiness is indeed important. Instead of switching ports (since many firewalls block ports other than 80 and 443) we are using HTTPS for Websockets, since that also helps avoid issues with proxies messing with the Websocket connection, and HTTP for the emulation transports. I currently use a fork to accomplish this with SockJS, but I hope to add this feature soon.

Another thing to be aware of with HTTPS terminated at a load balancer is making sure the server knows that it was originally HTTPS, via the X-Forwarded-Proto header or some other means. This is important for a correct Websocket upgrade. Modern browsers won’t complain about a mismatch, but older ones like Safari 5.1 will. Unfortunately an ELB in TCP mode cannot add these headers. One option is to use the PROXY protocol (http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/enable-proxy-protocol.html) and parse this data on your server. We ended up having our Varnish layer add the header since all the traffic was tunneling through it anyways.

--
You received this message because you are subscribed to the Google Groups "sockjs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sockjs+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Michael Laing]

If it is of interest, we have deployed a fully conforming, combined sockjs/websocket server in production in AWS that is now benchmarking over 100K clients on c1-mediums with excellent throughput.

We think it can go higher, but we also run a lot of other cruft on these machines: rabbitmq, subscription management, Cassandra interfaces, etc.

We run multi-region, multi-zone and autoscale this layer. We use least-latency routing and ELBs to initially connect, but clients then connect directly to their assigned endpoint and upgrade, bypassing the ELB.

It's python, and part of the NYTimes nyt⨍aбrik, though the heavy lifting is in a libev-driven C co-process. We'll be pushing it out in stages as open source, together with benchmarks.

ml

(actually I'll re-subscribe from my NYTimes account - this started out as 'fun' but now it is also 'work')

[Neil Camara]

Have you guys heard of primus? Not sure if I should go with this or sockjs

[Bryce Kahle]

Primus is a wrapper around other libraries, such as SockJS and Engine.IO that provides additional capabilities on top.

[onie....@gmail.com]

Bryce,

What do you think about my diagram? What I'm planning to do is to also make the nosql or sql server or any other database as a sockjs client similar to a browser. However, this server will only send updates to the sockjs server. There are no user cases of the browser sending any updates. What are your thoughts?

Thanks,

Neil

[Laing, Michael]

Well you stimulated me to update our own diagram.

We connect 100k to 1m client devices in this manner at any one time (currently). All connections are bidirectional. The gateway layer autoscales depending on load. We coordinate the sessions of devices being used by the same client even if they are coming through separate gateways, possibly in different datacenters.

Our gateway 'prefers' raw websockets (for performance) and 'falls back' to sockjs.

This supports our production website at the NYTimes, so I guess I would say that this type of architecture 'works'.

BTW I will be giving a talk on this at OSCON in Portland, OR in July.

ml

--

[onie....@gmail.com]

That is amazing!

Can you please explain more about "raw websockets" you mentioned below? Is it another application like sockjs?

Honestly, I want to visit Portland. I want to see the lighthouses. I'll try to fix my schedule so that I can make it happen :)

[onie....@gmail.com]

Hi Bryce,

Is it worth adding Primus or should I stick with just SockJS? I was told by folks in #primus in IRC freenode that it's awesome.

[sebastie...@geeked.ca]

Hi Michael,

Thanks for those insights.

I'm curious, what is your scale-down policy? Do you terminate instances when they still have open connections?

Regards,

Sebastien

[Laing, Michael]

+sockjs

---------- Forwarded message ----------
From: Laing, Michael <michae...@nytimes.com>
Date: Fri, Jun 27, 2014 at 5:22 PM
Subject: Re: [sockjs] Re: Deploying SockJS server(s) behind AWS ELB or other proxies
To: sebastie...@geeked.ca

The gateway can shed load several thousand at a time. Sometimes we do just let them die as well.

This creates a surge as clients reconnect to new boxes. Any message they might have missed is pulled from cache and they are resubscribed.

The user doesn't notice the delay, as far as we can tell, it's only a few seconds.

We may introduce a more orderly session migration/consolidation capability later.