Recommended way to run tornado at production

[Phyo Arkar]

What is the recommended way to run tornado at production enviorments?

Nginx vs Uwsgi vs Gunicorn?

Or just PyPy + Tornado ?

[Japhy Bartlett]

I've been using just nginx and HTTPServer.start() to handle process pooling.  No complaints so far!

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python-tornad...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jean Chassoul]

How about nginx and supervisord managing an N to N relationship between the number of tornado servers and the number of cores on your server/machine.

For extra fun, monit can monitor the nginx server and other stuffs on your system like data storage's or something else, the same can be achieved probably just with supervisord but I personally like to use supervisor just for python processes and monit for nginx and the other parts of the system.

[Phyo Arkar]

Thanks, nice ideas.

There no problems in using Nginx as frontend , right ?

No need for uWSGI to middle between nginx and tornado , like in traditional WSGI Apps ?

[Mehmet Köse]

my basic setup is; nginx + tornado ( + supervisor) + mongodb (with motor)

all frontend and media files on the amazon s3. It is serving on my media subdomain. (s.domain.com => KEY.cloudfront.net)

also, when I frontend development, my gulp tasks can minify css/js files, and upload ui files to S3, every change.
that. is. awesome.

Im really enjoying develop with this setup. Really fast and scalable.

[Japhy Bartlett]

nginx is great, and has been for years!

imo, you don't need uWSGI (especially if you're doing asynchronous stuff, WSGI is not compatible with this).

upstart is preferred over supervisord if it's part of your chosen Linux distro, imo, but supervisor is solid if you need something to make sure tornado starts on boot and stays alive.

[Phyo Arkar]

>also, when I frontend development, my gulp tasks can minify css/js files, and upload ui files to S3, every change. 
that. is. awesome.

Interesting.

For container , what is your choice ? Docker , Rkt . I gonna try RunC which just announced today.

[Russ Weeks]

The only problem I've found with using NGINX as a frontend is that it doesn't play well with Tornado's streaming upload feature. From what I've read, HAProxy is the way to go if you use streaming uploads.

I'd love to be wrong about that - if anybody has had a positive experience using streaming uploads with NGINX please share!

-Russ

[Phyo Arkar]

May be ben can chip in? ccing directly to ben too.

[Kevin LaTona]

On Jun 22, 2015, at 7:40 AM, Phyo Arkar <phyo.ar...@gmail.com> wrote:

> What is the recommended way to run tornado at production environments?


You have not described what you are trying to do yet in enough detail, so it's just almost impossible to offer any real feedback.

Tornado is mature and production ready today, how one decides to use it reveals what, if any other parts are needed.

Tornado is very capable to run on it's own or load balanced instances of it sitting behind any number of options like a NGINX.

Static content is best served by a static HTTP server IMO rather than having Tornado do that with logic calls at the same time.

But Tornado is more than capable of doing and delivering it all depending on your load, needs or desired outcome.

Using a LXC/D or a Docker instance makes better sense to me these days than running multiple simple Unix / Linux processes that has been shown in Tornado docs.

But those multiprocess examples are much simpler and easier to get going with and given Docker or LXC was not really around when those parts of the docs were written it makes some sense why they are shown or were used back then.

For simpler needs one can easily use basic round robbin DNS config's to load balance multiple Tornado instances, there by getting rid of having to sit behind a NGINX or HA Proxy box all together.

Or using a dedicated hardware load balancer like a F5 may make better sense if your load or needs are that demanding.

If you are running on any of the cloud setups like a AWS you can use Boto to deal with Route 53, Elastic Load Balancing and EC2's for you. And all the other services have some kind of similar solution on the racks.

Tell us more what you are hoping or wanting to do and maybe folks can talk more about it. Until then it's all just to general to make much more sense right now.



-Kevin

[Phyo Arkar]

Thanks a lot for detailed explanation , I am just preparing and wants input on what is recommended setup . All the posts here are very useful (And i hope they will be useful for many new comers to Tornado too).

Such information is hard to find for tornado as tornado , is although popular used by many , not talked about it much.

[Kevin LaTona]

Your welcome and yes, Tornado is one of the best kept Python secrets…… not sure why folks use Django…. Oh I didn't say……. that did I.  ;-)

These days depending on loads and needs really I see no reason to sit Tornado behind a front end HTTP server it seems so 90's to me.

But it could make sense in some or many cases where that front end is directing incoming calls for you.

These days all the craze is Docker, but for my needs and desires either legacy LXC or the new incoming( not production level yet ) Go based handler for containers known as LXD is the way to go.

The Ubuntu camp just annouced yesterday Fan which is one more take on dealing with containers and networking issues.

I think any one who gives you some stock answer as to what is the recommended or best way to setup Tornado is doing an injustice to you.

I for one have never even touched one of the Tornado's core original features …..long polling….. let alone used Websockets all that often …...as I really was after it's IOLoop more than anything else.

Any honestly I think it has one the nicer Python based url router ideas ( which I think was based or influnced on WebPy as I recall ).

It just makes better sense to me to look at using lots of LXC based Tornado instances when it comes to blocking calls than messing with Threads and Thread pools… but to each their own.

Don't forget Python( like Ruby, PHP and many others ) itself is single threaded and the GIL deals with all of Python's internal guts. 

So messing with thread pools is for people who want to think some how it's all working and are too lazy to figure out how to do Async style calling but at some point a processor can only do one thing at a time.

Just about everything I've read shows that the Async model is big, better, and faster than a process level thread model……. but it takes more thought and energy to get going with async model.

At times it's just okay to let Tornado block as, you have too or  there is no other better options….. that's where multiple instances solve the underlaying issues and they could be running on other processor cores.

Tornado itself is wicked fast… but it's not a C based HTTP engine either…. but don't let any one fool you it's smoking fast …... lean and clean.

A basic Tornado instance uses about 17 MB of ram… so figure it out you can run lots of LXC containters with lots of Tornado instances doing lots of things.

No reason you can run 100 to 500 LXC containers on decent processor with enough ram…… read up on it might change or open up your thoughts on that whole model of doing things.

A LXC allows you to control total ram, processor usage, disk access and lots of other stuff far better than simple Unix process.

With all the containers headed or going towards Atomic based images that opens up a whole better why to do things in production as well as development.

Plus Tornado's template engine is very nice and does the job without getting in the way, but not a lot not is said about it either.

I think the core reason why in the Python world Tornado is not talk about all that much is most Pythonista just have not caught on to Async yet. 

Twisted with it's depth and twists really has not made the topic all that approachable ……..so most people opted for Django the easy answer……. maybe some day I understand why, but Django for me is one huge set of hand cuffs that far too many people are using for all the wrong reasons or way, but hey it get's the job done for them, so it must be working for them.

Now with Torando being able to use the Python standard library's Asyncio calls who knows what the future may bring. Not sure what that is or why yet… but Tornado just works fine today.

I think Brett laid down a soild core and Ben has kept it alive and moving forward and made it solid. 

All along  keeping it simple and lean in the end is the best way to  do lots of projects and not enough Pythonista are using it like they should be. 

But maybe some day they might wake up… but hey maybe it's better that that masses don't come and want to mess and change it from it's core rasion d'etre.

-Kevin

[Ben Darnell]

My advice:

I recommend using some sort of load-balancing proxy instead of putting Tornado directly on the front line of the site. I've used both nginx and haproxy; haproxy is better at being a proxy but nginx has more non-proxy features. Sometimes it even makes sense to use both in the same deployment.

Don't serve static files from Tornado in production; either serve them from nginx or use a CDN.

I've never used uwsgi or gunicorn, and I haven't used pypy in production. For my next project I'd use python 3; probably cpython 3.5 instead of pypy3 (unless pypy3 has caught up to 3.5 by then).

I generally use supervisord as a process manager because I'm familiar with it and it handles process groups well (use an external process manager that can do rolling restarts, not HTTPServer.start()). Any process manager that doesn't require "daemonization" should be fine (runit, daemontools, etc)

I haven't done anything in production with containers but I think kubernetes is on the right track. 

--

[Phyo Arkar]

thanks a lot @ben and @ken,

Very helpful info.

@Ben , serving static files tru nginx is preferred way? So also use Nginx for uploads?

I think we should convince PyPy team to move directly to 3.5  , instead of doing 3.3 , 3.4 etc. (i saw a thread they are considering it).

Pypy 2.6 performance is Superb , you should give a try sometime.

For me , if pypy moved to 3.5, it will be the time i start using  python3 :P .

[Kevin LaTona]

On Jun 23, 2015, at 8:47 PM, Ben Darnell <b...@bendarnell.com> wrote:

> My advice:
>
> I recommend using some sort of load-balancing proxy instead of putting Tornado directly on the front line of the site.

Come on Ben… take a walk on the wild side. ;-)

Tornado like so many other HTTP engines are capable of standing on the front line… when it makes sense to do it that way.


Past that …...All problems have a solution, once the definition is clear and stated.

Tornado and Asyncio should have more Pythonista's using these approaches in their work.


If Python folks are waiting for another small town newspaper in Kansas to come along and allow some folks to build an Async based newspaper publishing HTTP content engine that needs to publish news everyday on a deadline.

It's just never going to happen.


Plus from my point of view any time any one grabs any product or open source project that was created to solve "a problem" for a startup or some other company's stated goals.

That now you inherit and need to deal with that DNA imprint.

If it solves your needs great.

But so many people grab and start using a tool with no solid idea why they grabbed it…… past it's popular or talked about…... so it must be good etc.

To each their own I guess to figure that all out.


TBL defined HTTP as a very simple protocol and it is.

It's all the geeks who pile on all kind of other needs and desires onto HTTP that makes it a mess at times.

-Kevin

[Jean Chassoul]

Hi,

Kevin, tornado it's awesome and fast!, but for production you really want to invest some time on setting that load balance in front of your traffic and some monitor/supervisor to give you restart and monitoring powers.

We all have many-cores in every device, there's no silver bullet but the N to N relation between your cores and tornado processes it's also important.

On a production environment you want to avoid idle cores and scale horizontally bringing up more servers to handle your load when needed or killing them to crash the parts of the system to be more $ cost effective, for development a tornado server handling all it's ok and again awesome and fast, but there's no way to recommend that setup to run tornado at production.

just and another opinionated point of view. 

(=

[Kevin LaTona]

Jean

Absolutely I would agree with you 101% for bigger and heavy loads like what you describe is solid solution ideas.

But not all sites or projects require full blown scaleable complex setups to use a HTTP engine like a Tornado.

Lots of Pythonista are missing out how to learn and use Async…… everyone needs to take the first baby steps to make the big leap.

Also as far as sitting NGINX in front don't forget it too is a single threaded Async HTTP engine.

It too has many of the same core weakness as Tornado faces but its pure C so those isssue may be slower to appear.

It blocks and locks like any other Async server can given the right circumstances, load and config settings.

Unless something has change in the world…. FreeBSD with it's proven long track record is the right kind of solution OS to be looking at or uisng for load balancing issues.

-Kevin

[A. Jesse Jiryu Davis]

I use exactly the same setup as Mehmet for http://emptysqua.re, naturally. =) I have four Tornado processes, since the Rackspace box is four-core. I just serve static files with nginx, not S3, since my site's very low-traffic. I have no experience running a high-traffic site, my setup works well for me as a simple way to keep a small site up on one server without a lot of administration or complexity.

[Japhy Bartlett]

Kevin -

I think the thing you're misunderstanding is that Nginx not only handles load balancing and serving static files, but also handles malformed and malicious requests.

There's an appropriate time to 'walk on the wild side', but port 80 on a public IP is risky enough as is.

- Japhy

[Ben Darnell]

On Thu, Jun 25, 2015 at 11:08 AM, Japhy Bartlett <ja...@pearachute.com> wrote:

Kevin -

I think the thing you're misunderstanding is that Nginx not only handles load balancing and serving static files, but also handles malformed and malicious requests.

Tornado should be fine with malformed requests; I recommend a proxy for reasons of performance and convenience, not for security (well, it's kind of for security: proxies can handle DDoS attacks much better than Tornado alone).

I say proxies are "convenient" even though they're an extra moving piece because they facilitate zero-downtime rolling restarts, so you can push new code more often. And in multi-machine deployments you'll have a bunch of tornado servers but only need a few LBs.

-Ben

[Jean Chassoul]

They are an extra moving piece but that pays for it self making your production deployment more robust, the same analogy can applied with the supervisord/monit pieces they gives you fault tolerance properties and for that you need at lest two individual tornado processes behind your LB of choice.

[Kevin LaTona]

On Jun 25, 2015, at 8:08 AM, Japhy Bartlett <ja...@pearachute.com> wrote:

> There's an appropriate time to 'walk on the wild side',

For me this means at times not forgetting to challenge myself to look and see.

Is there a better way of doing the same old thing… only better and easier now.

As when I do that, some days it leads me down paths that I never knew existed.


-Kevin

[Japhy Bartlett]

"""

Dear users -

We've been hacked and lost all of your passwords and personal data because one of our devs found an easier way to do encryption.

Now we're going to fire everyone and go out of business.  Bummer.

We really learned some important life lessons about ourselves though, it's a great time to be alive.

"""

good luck man

[Kevin LaTona]

Japhy,

I am sensing maybe some thing is being mis-read into my previous emails.

That last email was not directed back at you per se and I am now making sure that was clear in it's intentions.

Rather it was about a ongoing dialog that was I was hearing within that thread.

But do have to say your last post did make me chuckle a bit.

The upshot for me on the security topic is you hit nailed what so many companies are not doing about security + protection than what they are doing about it these days.

Thanks for that.

-Kevin