Setup multiple Tornado servers handling different pages

[mrtn]

I want to setup two Tornado servers for the same app: one handles requests to a few URLs as well as WebSockets messagings, and the other handles requests to all other specified URLs.

I know I can put all the handlers inside a single Tornado server instance. However, I choose the above because I want the server doing WebSockets communications handles as few page requests as possible, and offload all other page requests to the other server.

Is this possible? If so, how should I do it? 

[Nick Jennings]

Well, it's definitely possible, though I don't know if it's necessary.
Maybe someone else could answer about that.

As for how you'd do it. Seems like it would make sense to run them
both on internal ports (127.0.0.1:8000 + 8001 for example), and use a
proxy daemon like HAProxy to run on port 80 and send traffic to either
port depending on the URIs. That's how i'd start at least.

Hope this helps
Nick

[mrtn]

Thank you for your advice.

Well, it's definitely possible, though I don't know if it's necessary. 
Maybe someone else could answer about that. 

Could you please elaborate on this? Why do you think it may not be necessary? 

As for how you'd do it. Seems like it would make sense to run them 
both on internal ports (127.0.0.1:8000 + 8001 for example), and use a 
proxy daemon like HAProxy to run on port 80 and send traffic to either 
port depending on the URIs. That's how i'd start at least. 

From what I've gathered, HAProxy does not server static files as it is only a proxy, where Nginx does that well but it does not work with WebSockets. So is there any way I can use HAProxy but also have static files served fast?

 

[Nick Jennings]

Hello

On Thu, Sep 20, 2012 at 5:25 PM, mrtn <mrtn...@gmail.com> wrote:
>
> Thank you for your advice.
>
>> Well, it's definitely possible, though I don't know if it's necessary.
>> Maybe someone else could answer about that.
>
>
> Could you please elaborate on this? Why do you think it may not be
> necessary?

Well, there are various options, like having independant worker
threads communicating via Redis. This could offload any of the
heavy-lifting and keep your app non-blocking. However, I'm still
learning about Tornado and don't pretend to be an expert on the pros
and cons to your desired setup. :)

>> As for how you'd do it. Seems like it would make sense to run them
>> both on internal ports (127.0.0.1:8000 + 8001 for example), and use a
>> proxy daemon like HAProxy to run on port 80 and send traffic to either
>> port depending on the URIs. That's how i'd start at least.
>
>
> From what I've gathered, HAProxy does not server static files as it is only
> a proxy, where Nginx does that well but it does not work with WebSockets. So
> is there any way I can use HAProxy but also have static files served fast?

Correct, HAProxy does not serve static files. Nginx does this great,
and right now I run both on my server. HAProxy is the "front-line" and
can route all websocket traffic directly to tornado, while any static
or other non-socket traffic gets directed to Nginx to handle.

I actually just finished (a few minutes ago) a blog post on some
common setups using HAProxy with WebSockets
http://blog.silverbucket.net/post/31927044856/3-ways-to-configure-haproxy-for-websockets

And a few months ago I wrote about using Nginx in your stack:
http://blog.silverbucket.net/post/25156269990/5-reasons-to-add-nginx-to-your-lamp-stack-now

I'm not intending to just promote my posts >:D, but I think it's
directly applicable to your issue. It's basically based on the setup I
eventually came to with my project, and I think combining the topics
covered from both posts will give you the setup you're looking for.

Hope that helps!
Nick

[mrtn]

Thanks a lot for the great posts! Very useful guides for me to get started.

Correct, HAProxy does not serve static files. Nginx does this great, 
and right now I run both on my server. HAProxy is the "front-line" and 
can route all websocket traffic directly to tornado, while any static 
or other non-socket traffic gets directed to Nginx to handle.

So my current design is this: have a HAProxy sit in front of a Nginx, tornado1 (server that mainly handles WebSockets), and tornado2 (server that mainly handles http requests).

1. If the incoming traffic is WebSockets (or any other transport listed here: https://github.com/sockjs/sockjs-client#supported-transports-by-browser-html-served-from-http-or-https), then HAProxy directs to tornado1.

2. If the traffic is for static file, then HAProxy directs to Nginx.

3. If the traffic is for HTTP requests, then based on the URLs, HAProxy directs to either tornado1 or tornado2.

A further question though, given the use of SockJS (or similarly, Socket.io), in case of WebSocket transport is not supported, how do we ensure the fallback transports (such as xhr-streaming or xhr-polling) also goes to tornado1 exclusively? One way I can think of is to use the second method (proxy based on URI) in your post.

What do you think? Thanks!

[Nick Jennings]

On Fri, Sep 21, 2012 at 9:23 PM, mrtn <mrtn...@gmail.com> wrote:
>
> Thanks a lot for the great posts! Very useful guides for me to get started.
>
>> Correct, HAProxy does not serve static files. Nginx does this great,
>> and right now I run both on my server. HAProxy is the "front-line" and
>> can route all websocket traffic directly to tornado, while any static
>> or other non-socket traffic gets directed to Nginx to handle.
>
>
> So my current design is this: have a HAProxy sit in front of a Nginx,
> tornado1 (server that mainly handles WebSockets), and tornado2 (server that
> mainly handles http requests).
>
> 1. If the incoming traffic is WebSockets (or any other transport listed
> here:
> https://github.com/sockjs/sockjs-client#supported-transports-by-browser-html-served-from-http-or-https),
> then HAProxy directs to tornado1.
>
> 2. If the traffic is for static file, then HAProxy directs to Nginx.
>
> 3. If the traffic is for HTTP requests, then based on the URLs, HAProxy
> directs to either tornado1 or tornado2.
>
> A further question though, given the use of SockJS (or similarly,
> Socket.io), in case of WebSocket transport is not supported, how do we
> ensure the fallback transports (such as xhr-streaming or xhr-polling) also
> goes to tornado1 exclusively? One way I can think of is to use the second
> method (proxy based on URI) in your post.
>
> What do you think? Thanks!

Hmm, I'm not sure. I haven't had to deal with these use-cases (yet).
So far, the project I'm working on requires flash so we can ensure
there will be websocket support no matter the browser version -- but
I'd be interested to know what your solution ends up being.

-Nick

[Serge S. Koval]

On Fri, Sep 21, 2012 at 10:23 PM, mrtn <mrtn...@gmail.com> wrote:

So my current design is this: have a HAProxy sit in front of a Nginx, tornado1 (server that mainly handles WebSockets), and tornado2 (server that mainly handles http requests).

1. If the incoming traffic is WebSockets (or any other transport listed here: https://github.com/sockjs/sockjs-client#supported-transports-by-browser-html-served-from-http-or-https), then HAProxy directs to tornado1.

Use URL based routing for SockJS. Here's sample HAProxy configuration file: https://github.com/sockjs/sockjs-node/blob/master/examples/haproxy.cfg

Configuration is pretty straightforward:

1. All requests that contain /echo, /broadcast, .., etc are considered SockJS requests. Put your SockJS endpoint here

2. Special case for /stats

3. Rest goes to 'static' backend

When you want to connect to SockJS, connect to your load balancer address with appropriate sockjs prefix, like /echo and it will be automatically forwarded to 'sockjs' backend which will have your tornado1 server.

This configuration being successfully used in production with 3 SockJS servers, few application servers and one nginx for static data.

Make sure you include these in your config though for SockJS to work properly:

# Fake connection:close, required in this setup.
    option http-server-close
    option http-pretend-keepalive

Hope it helps.

Serge.