Running Django on Tornado's HTTP server

[Bret Taylor]

I am one of the authors of Tornado (http://www.tornadoweb.org/), the
web server/framework we built at FriendFeed that we open sourced last
week (see http://bret.appspot.com/entry/tornado-web-server).

The underlying non-blocking HTTP server is fairly high performance, so
I have been working this weekend to get other frameworks like Django
and web.py working on Tornado's server so existing projects could
potentially benefit from the performance. To that end, I just checked
in change to Tornado that enables you to run any WSGI-compatible
framework on Tornado's HTTP server. You can find it in a class called
WSGIContainer in our wsgi.py:

http://github.com/facebook/tornado/blob/master/tornado/wsgi.py#L188

You will have to check out Tornado from github to get the change; it
is not yet included in the tarball distribution.

Here is a template for running a Django app on Tornado's server using
the module:

import django.core.handlers.wsgi
import os
import tornado.httpserver
import tornado.ioloop
import tornado.wsgi

def main():
os.environ["DJANGO_SETTINGS_MODULE"] = 'myapp.settings'
application = django.core.handlers.wsgi.WSGIHandler()
container = tornado.wsgi.WSGIContainer(application)
http_server = tornado.httpserver.HTTPServer(container)
http_server.listen(8888)
tornado.ioloop.IOLoop.instance().start()

if __name__ == "__main__":
main()


I have only done very basic tests using the new module, so if any of
you are interested and start using Tornado with your Django projects,
please let us know what bugs you find so we can fix them. Any and all
feedback is appreciated.

Bret

[Antoni Aloy]

2009/9/13 Bret Taylor <bta...@gmail.com>:

We're acually using Cherrypy, so it would be quite easy to test and
check the performance. We'll post any issues we found.

Thank you!

--
Antoni Aloy López
Blog: http://trespams.com
Site: http://apsl.net

[Antoni Aloy]

2009/9/13 Antoni Aloy <anton...@gmail.com>:

Just few numbers:

ab -c 10 -t 60 http://localhost:8088/
Executing Django with Cherrypy with 3 threads on a PPC Computer 2 CPU 2 GHz

With Tornado WSGI:
ab -c 10 -t 60 http://localhost:8888/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 5000 requests
Completed 10000 requests
Completed 15000 requests
Completed 20000 requests
Completed 25000 requests
Finished 25022 requests

Server Software: TornadoServer/0.1
Server Hostname: localhost
Server Port: 8888

Document Path: /
Document Length: 266 bytes

Concurrency Level: 10
Time taken for tests: 60.019 seconds
Complete requests: 25022
Failed requests: 0
Write errors: 0
Total transferred: 9333206 bytes
HTML transferred: 6655852 bytes
Requests per second: 416.90 [#/sec] (mean)
Time per request: 23.987 [ms] (mean)
Time per request: 2.399 [ms] (mean, across all concurrent requests)
Transfer rate: 151.86 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.1 0 6
Processing: 3 24 1.9 24 68
Waiting: 0 24 1.9 23 68
Total: 7 24 1.9 24 68

Percentage of the requests served within a certain time (ms)
50% 24
66% 24
75% 24
80% 24
90% 25
95% 26
98% 27
99% 28
100% 68 (longest request)

Using CherryPy with 3 threads (the best configuration for my computer
in my tests)

ab -c 10 -t 60 http://localhost:8088/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 5000 requests
Completed 10000 requests
Completed 15000 requests
Completed 20000 requests
Finished 21500 requests

Server Software: CherryPy/3.0.3
Server Hostname: localhost
Server Port: 8088

Document Path: /
Document Length: 266 bytes

Concurrency Level: 10
Time taken for tests: 60.001 seconds
Complete requests: 21500
Failed requests: 0
Write errors: 0
Total transferred: 8299000 bytes
HTML transferred: 5719000 bytes
Requests per second: 358.33 [#/sec] (mean)
Time per request: 27.907 [ms] (mean)
Time per request: 2.791 [ms] (mean, across all concurrent requests)
Transfer rate: 135.07 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 20.5 0 2999
Processing: 3 28 10.7 26 413
Waiting: 3 26 10.3 24 412
Total: 3 28 23.1 26 3031

Percentage of the requests served within a certain time (ms)
50% 26
66% 28
75% 30
80% 31
90% 34
95% 38
98% 43
99% 48
100% 3031 (longest request)

To sumarize:

416.90 req/s for Tornado WSGI
358.33 req/s for CherryPy

Tornado makes one CPU up to 100% nearly at the same time of the ab
execution, Cherrypy takes longer. Perhaps some CherryPy guru could
configure it better.

The code is at

http://appfusedjango.googlecode.com/svn/trunk/hello_world for the
CherryPy (django-cerise) version
http://appfusedjango.googlecode.com/svn/trunk/hello_tornado for the
tornado version (includes tornado wsgi)

This is a sample helloword project with just the essential to run.
It's the same code I used to compare with PHP frameworks

http://trespams.com/2009/05/10/django-vs-php-framewors/

[Ismail Dhorat]

Interesting, i would like to see how tornado stacks up to Lighttpd?

Regards,
Ismail

[Brian]

I have a small Django app under development and was able to get this
running using the github update. My app runs and is very responsive
with Tornado.

Some of my admin site layout is out of whack, but it may be my
settings or my bug. Formatting and CSS I created on my own looks good
(e.g. graphics and tables created using the Google Visualization API.)
My app is able to go out onto the web, get files downloaded through
Tornado - it feels fast ...

It's terrific to have a Python-based server that could work in both
development and production. (Tornado installs/runs fine on a Netbook
with Ubuntu!)

The real-time / asynchronous capabilities look great. Any way Django
apps can take advantage of that functionality?

Thanks for open sourcing this!

-- Brian

On Sep 13, 4:26 pm, Bret Taylor <btay...@gmail.com> wrote:
> I am one of the authors of Tornado (http://www.tornadoweb.org/), the
> web server/framework we built at FriendFeed that we open sourced last

> week (seehttp://bret.appspot.com/entry/tornado-web-server).

[fruits]

media files for admin is handled by the django dev server. So when you
change the server, you have to serve admin media file yourself.

http://docs.djangoproject.com/en/dev/howto/deployment/modpython/#id3

[Brian]

Thanks ... this link pointed me in the right direction.

Should I be able to serve up admin media using just Django and
Tornado? If so, I haven't found the right combination of paths / urls
etc. in settings.py to get that working. The Tornado doc describes how
to "serve static files from Tornado by specifying the static_path
setting in your application" - so it sounds like it might eventually
work? The Tornado blog demo is one example, but I couldn't get that
approach working with my Django app.

I now have nginx serving up the Admin files, running alongside Tornado
and Django. My Admin Site formatting and layout looks fine with that
setup.

-- Brian

[Kenneth Gonsalves]

On Monday 14 Sep 2009 1:56:32 am Bret Taylor wrote:
> he module:
>
> import django.core.handlers.wsgi
> import os
> import tornado.httpserver
> import tornado.ioloop
> import tornado.wsgi
>
> def main():
> os.environ["DJANGO_SETTINGS_MODULE"] = 'myapp.settings'
> application = django.core.handlers.wsgi.WSGIHandler()
> container = tornado.wsgi.WSGIContainer(application)
> http_server = tornado.httpserver.HTTPServer(container)
> http_server.listen(8888)
> tornado.ioloop.IOLoop.instance().start()
>
> if __name__ == "__main__":
> main()

cool - this worked out of the box - but no css and js - can I serve media from
tornado, and if so, a pointer as to how?
--
regards
kg
http://lawgon.livejournal.com

[Antoni Aloy]

2009/9/19 Kenneth Gonsalves <law...@thenilgiris.com>:

Have you properly configured your MEDIA_URL to point to your Tornado
server instead of localhost:8000. I haven't made any test about
Tornado performance on serving statig files, but on production our
actual configuration with CherryPy in WSGI mode involves a nginx
server acting as a reverse proxy and serving the media files.

I use the configuration without nginx pointing directly to the WSGI
server just to make sure that all the media points to right server.
That is, that I have write all my media files as
{{MEDIA_URL}}/img/xxxx.jpg and so on.

Hope it helps!

[Kenneth Gonsalves]

solved - managed to run several django sites of an nginx instance using
virtual hosts and individual tornado servers for each virtual host. Coolness.

[Ramdas S]

Kenneth,

Why don't you doc it somewhere for future queries and for newbies switching? Also have you see any tangible performance improvement?
 

.

--
regards
kg
http://lawgon.livejournal.com

--
Ramdas S
+91 9342 583 065

[Kenneth Gonsalves]

On Tuesday 29 Sep 2009 2:44:26 pm Ramdas S wrote:
> can I serve media
>
> > > from tornado, and if so, a pointer as to how?
> >
> > solved - managed to run several django sites of an nginx instance using
> > virtual hosts and individual tornado servers for each virtual host.
> > Coolness
>
> Kenneth,
>
> Why don't you doc it somewhere for future queries and for newbies
> switching? Also have you see any tangible performance improvement?

will document after trying it in production - a zillion times faster.

[Javier Guerra]

On Tue, Sep 29, 2009 at 4:24 AM, Kenneth Gonsalves
<law...@thenilgiris.com> wrote:
> will document after trying it in production - a zillion times faster

faster than......... ? flup?

--
Javier

[Kenneth Gonsalves]

faster than apache/mod_wsgi

[Kenneth Gonsalves]

On Tuesday 29 Sep 2009 2:44:26 pm Ramdas S wrote:

> > > cool - this worked out of the box - but no css and js - can I serve
> > > media from tornado, and if so, a pointer as to how?
> >
> > solved - managed to run several django sites of an nginx instance using
> > virtual hosts and individual tornado servers for each virtual host.
> > Coolness
>
> Kenneth,
>
> Why don't you doc it somewhere for future queries and for newbies
> switching?

have put my code in djangosnippets:
http://www.djangosnippets.org/snippets/1748/, the site in question is here:
http://demo.ilugc.org.in