Django with CherryPy 3.0
noahz
3.0 WSGI server for local development. I was quite surprised how easy
this was to do, essentially replacing in core/management.py:
>>> from django.core.servers.basehttp import run
>>> run(addr, int(port), handler)
with:
>>> from cherrypy import wsgiserver
>>> server = wsgiserver.CherryPyWSGIServer((addr, int(port)), handler, server_name=addr)
>>> server.start()
I also added the Paste translogging middleware to log to the console,
so now for local development with no observable changes, I can do:
# ./manage.py runcherrypy
The biggest benefit of this in my case is that CherryPy handles
concurrent requests properly, making certain types of AJAX tasks
possible to develop locally, where django's built in server blocks
concurrent requests.
It seems like the performance is worse, possibly due to the fact that
I'm still using static.serve a lot locally. But the utility is
certainly higher for heavy ajax development, and even quick and dirty
deployment.
So I thought I might share my experiment with everyone for discussion.
Do others see the utility in this? The patch is following, and I could
be prompted to submit it to trac...
noahz
===================================================================
--- ../../django/django/core/management.py (revision 4306)
+++ ../../django/django/core/management.py (working copy)
@@ -1136,6 +1136,65 @@
inner_run()
runserver.args = '[--noreload] [--adminmedia=ADMIN_MEDIA_PATH]
[optional port number, or ipaddr:port]'
+def runcherrypy(addr, port, use_reloader=True, admin_media_dir=''):
+ "Starts a cherrypy Web server for development, with the paste
translogger middleware."
+ from django.core.servers.basehttp import run, AdminMediaHandler,
WSGIServerException
+ from django.core.handlers.wsgi import WSGIHandler
+ from cherrypy import wsgiserver
+ from paste import translogger
+
+ if not addr:
+ addr = '127.0.0.1'
+ if not port.isdigit():
+ sys.stderr.write(style.ERROR("Error: %r is not a valid port
number.\n" % port))
+ sys.exit(1)
+ quit_command = sys.platform == 'win32' and 'CTRL-BREAK' or
'CONTROL-C'
+
+ def inner_run():
+ from django.conf import settings
+ print "Validating models..."
+ validate()
+ fRed = chr(27) + '[31m'
+ fNone = chr(27) + '[0m'
+ print "\nDjango version %s, using settings %r" %
(get_version(), settings.SETTINGS_MODULE)
+ print "%sCherryPy server%s is running at http://%s:%s/" %
(fRed, fNone, addr, port)
+ print "Quit the server with %s." % quit_command
+ try:
+ import django
+ path = admin_media_dir or django.__path__[0] +
'/contrib/admin/media'
+ handler = AdminMediaHandler(WSGIHandler(), path)
+
+ # wrap handler in logger middleware
+ handler = translogger.TransLogger(
+ handler,
+ format = '[%(time)s] "%(REQUEST_METHOD)s
%(REQUEST_URI)s %(HTTP_VERSION)s" %(status)s %(bytes)s',
+ setup_console_handler=True
+ )
+ server = wsgiserver.CherryPyWSGIServer((addr, int(port)),
handler, server_name=addr)
+ server.start()
+ except WSGIServerException, e:
+ # Use helpful error messages instead of ugly tracebacks.
+ ERRORS = {
+ 13: "You don't have permission to access that port.",
+ 98: "That port is already in use.",
+ 99: "That IP address can't be assigned-to.",
+ }
+ try:
+ error_text = ERRORS[e.args[0].args[0]]
+ except (AttributeError, KeyError):
+ error_text = str(e)
+ sys.stderr.write(style.ERROR("Error: %s" % error_text) +
'\n')
+ sys.exit(1)
+ except KeyboardInterrupt:
+ sys.exit(0)
+ if use_reloader:
+ from django.utils import autoreload
+ autoreload.main(inner_run)
+ else:
+ inner_run()
+runcherrypy.args = '[--noreload] [--adminmedia=ADMIN_MEDIA_PATH]
[optional port number, or ipaddr:port]'
+
+
def createcachetable(tablename):
"Creates the table needed to use the SQL cache backend"
from django.db import backend, connection, transaction,
get_creation_module, models
@@ -1251,6 +1310,7 @@
'reset': reset,
'runfcgi': runfcgi,
'runserver': runserver,
+ 'runcherrypy': runcherrypy,
'shell': run_shell,
'sql': get_sql_create,
'sqlall': get_sql_all,
@@ -1386,6 +1446,16 @@
except ValueError:
addr, port = '', args[1]
action_mapping[action](addr, port, options.use_reloader,
options.admin_media_path)
+ elif action =='runcherrypy':
+ if len(args) < 2:
+ addr = ''
+ port = '8000'
+ else:
+ try:
+ addr, port = args[1].split(':')
+ except ValueError:
+ addr, port = '', args[1]
+ action_mapping[action](addr, port, options.use_reloader,
options.admin_media_path)
elif action == 'runfcgi':
action_mapping[action](args[1:])
else:
Graham King
Particularly I was trying to get the server to make a connection to
itself to cache the output HTML of a template, so as to serve it as a
static file.
+1 for anything that makes the dev server multi-threaded. I'm hoping
there might be a way to do this without introducing a dependency on
CherryPy.
Regards,
Graham
gordyt
> +1 for anything that makes the dev server multi-threaded. I'm hoping
> there might be a way to do this without introducing a dependency on
> CherryPy.
I believe the wsgiserver.py file from the CherryPy distribution is
independent from the rest of the framework. I know, for example, that
Chad Whitacre (from the Aspen project:
http://www.zetadev.com/software/aspen/) includes it in Aspen without
any of the rest of CherryPy.
If Django did the same thing then it should work without having to
require that users also install CherryPy.
--gordon
Istvan Albert
see the diff below:
Index: basehttp.py
===================================================================
--- basehttp.py (revision 4183)
+++ basehttp.py (working copy)
@@ -7,10 +7,16 @@
been reviewed for security issues. Don't use it for production use.
"""
-from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
+from BaseHTTPServer import BaseHTTPRequestHandler
from types import ListType, StringType
import os, re, sys, time, urllib
+import BaseHTTPServer, SocketServer
+class HTTPServer(SocketServer.ThreadingMixIn,
BaseHTTPServer.HTTPServer):
+ def __init__(self, server_address, RequestHandlerClass=None):
+ BaseHTTPServer.HTTPServer.__init__(self, server_address,
RequestHandlerClass )
+ print '*** MixinHTTPServer ***'
+
__version__ = "0.1"
__all__ = ['WSGIServer','WSGIRequestHandler','demo_app']
Adrian Holovaty
> There is an easy way to turn on multi-threading for the default server,
> see the diff below:
Graham and/or anybody else who wants multi-threading in the
development server, can you try this patch and see whether it solves
your problem?
Adrian
--
Adrian Holovaty
holovaty.com | djangoproject.com
Rob Hudson
> Graham and/or anybody else who wants multi-threading in the
> development server, can you try this patch and see whether it solves
> your problem?
I updated my Django trunk with the ThreadingMixIn patch and looking at
a couple projects that have media served statically (for development).
I think this is proof that it is multi-threaded. Notice how the
original request shows up 2nd instead of first:
Django version 0.96-pre, using settings 'innovate.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
[22/Jan/2007 10:15:29] "GET /media/css/style.css HTTP/1.1" 200 6743
[22/Jan/2007 10:15:29] "GET / HTTP/1.1" 200 8014
[22/Jan/2007 10:15:29] "GET /media/css/print.css HTTP/1.1" 200 471
[22/Jan/2007 10:15:29] "GET /media/img/orcas_logo.gif HTTP/1.1" 200
4788
Also, I looked at the Firebug Net view to see how the 4 pieces are
loaded. The threaded server shows the 4 pieces with the HTML
overlapping the requests for CSS and image. Whereas the default Django
never has complete overlap -- requests always finish after the previous
one.
(I'll attach the images in the next message since I'm in the Google
Groups web form right now...)
-Rob
Rob Hudson
server. Notice the staggered view of the requests.
Second image (threaded.png) is with the patch (also attached). Notice
how the requests overlap. The initial requests takes longer, but I'm
curious if that's because the other 3 requests are happening
concurrently slowing down the original request.
Note: The patch file is from Django's root and so should be easy to apply.
-Rob
Graham King
Adrian Holovaty wrote:
> On 1/22/07, Istvan Albert <istvan...@gmail.com> wrote:
>> There is an easy way to turn on multi-threading for the default server,
>> see the diff below:
>
> Graham and/or anybody else who wants multi-threading in the
> development server, can you try this patch and see whether it solves
> your problem?
>
> Adrian
>
Yes, it does. I telnet to 8000 to keep a request open, and make a
regular browser request. With the current dev server the browser will
hang until I kill the telnet. With the patch the browser responds.
+1 from me.
I'm going to keep the patch on my local checkout and will report if I
notice anything funny.
Graham
Jeremy Dunck
>
> On 1/22/07, Istvan Albert <istvan...@gmail.com> wrote:
> > There is an easy way to turn on multi-threading for the default server,
> > see the diff below:
>
> Graham and/or anybody else who wants multi-threading in the
> development server, can you try this patch and see whether it solves
> your problem?
I don't mean to FUD, but introducing threads could be trouble in the
runserver. If you're serving multiple requests in response to a
single primary view (like dynamic js in response to an html request),
if the views aren't thread-safe, stuff will go bad. This isn't an
issue if you single-thread in dev and prefork in prod.
That said, most of the time, this is good stuff. Perhaps a management
option to turn it off?
Istvan Albert
> That said, most of the time, this is good stuff. Perhaps a management
> option to turn it off?
My understanding is that no part of basehttp.py is used when Django is
deployed via modpython. If that is correct then this addition would
make no difference in a production environment.
I've researched this mainly because I needed this functionality in the
test server, used it for a month or so and it works surprisingly well.
I did just for fun a little experiment and it seems to outperform the
paste http server as well as cherrypy even under severe load.
Potential pitfall: no limit on the number of threads (this could be
easily added though), so if you were to use it as a production server
(which of course you shouldn't) then you could be DOS'ed simply by
having clients connect and not disconnect.
Istvan
Jeremy Dunck
>
> > That said, most of the time, this is good stuff. Perhaps a management
> > option to turn it off?
>
> My understanding is that no part of basehttp.py is used when Django is
> deployed via modpython. If that is correct then this addition would
> make no difference in a production environment.
Sorry, I meant that a multi-threaded server in dev could cause ghost
problems -in dev- when used to serve code that's not thread-safe.
Adrian Holovaty
> Yes, it does. I telnet to 8000 to keep a request open, and make a
> regular browser request. With the current dev server the browser will
> hang until I kill the telnet. With the patch the browser responds.
>
> +1 from me.
>
> I'm going to keep the patch on my local checkout and will report if I
> notice anything funny.
Keeping in mind the comments about possible thread-safety heisenbugs,
let's make it so that the threaded behavior is *optional*, but people
can turn it on if they're concerned about Ajax-induced stalling. What
do you think? Anybody willing to update the patch and post it into our
ticket system?
Rob Hudson
> Keeping in mind the comments about possible thread-safety heisenbugs,
> let's make it so that the threaded behavior is *optional*, but people
> can turn it on if they're concerned about Ajax-induced stalling. What
> do you think? Anybody willing to update the patch and post it into our
> ticket system?
Credit should definitely go to Istvan Albert, but I started a patch
with documentation update(s) here:
http://code.djangoproject.com/ticket/3357