rolling my own pylons request like thing
Iain Duncan
class WSGIApp(object):
def __init__(self, **kwargs):
self.request = <web ob Request that is thread local>
... rest of server start up initialization ...
def __call__(environ, start_response):
# make self.request be the current request wrapper
self.request = Request(environ)
... does some stuff, calls other methods, returns up the wsgi
chain...
def some_method(self):
"per request helper method"
self.request.foo = 1
bar = self.request.bar
So I guess I want to make something awfully similar to the Pylons
request global, but I'm not sure whether or how I can make my request by
either thread-local or a StackedObjectProxy, or if this idea is just
dumb. basically I wan't to be able to say to the coder who writes a new
method that self.request can be used like a per request version of self
to preserve state between the per request helper methods. Any tips much
appreciated!
Iain
Sergey Schetinin
You'll end up with something like this:
@context.setting
def request(value=None):
if value is None: raise RuntimeError("request global was not set up")
assert isinstance(value, webob.Request)
return value
def wrap_request_global(app):
def wrapped_app(environ, start_response):
with context.new():
request <<= webob.Request(environ)
return app(environ, start_response)
return wrapped_app
And then use request() where you intended to use self.request
Best Regards,
Sergey Schetinin
http://s3bk.com/ -- S3 Backup
http://word-to-html.com/ -- Word to HTML Converter
Iain Duncan
>
> Hi all, I am wishing to do something like this:
>
> class WSGIApp(object):
>
> def __init__(self, **kwargs):
> self.request = <web ob Request that is thread local>
> ... rest of server start up initialization ...
>
> def __call__(environ, start_response):
> # make self.request be the current request wrapper
> self.request = Request(environ)
> ... does some stuff, calls other methods, returns up the wsgi
> chain...
>
>
> def some_method(self):
> "per request helper method"
> self.request.foo = 1
> bar = self.request.bar
I want to avoid any extra dependencies if possible and also keep
everything out of global namespace if I can, mostly because this is
written for people like me to drop in anywhere and I find global imports
confusing ( where did that come from? etc ). So I came up with the
following and would love to hear feedback on whether it will work right.
If someone who understands the StackedObjectProxy properly can let me
know whether I have successfully created a thread safe self.request,
that would be great!
Thanks
Iain
<code>
from webob import Request, Response
from paste.registry import RegistryManager, StackedObjectProxy
class BaseController(object):
def __init__(self):
self.request = StackedObjectProxy()
self.response = StackedObjectProxy()
def __call__(self, environ, start_response):
request = Request(environ)
response = Response(status=200, content_type='text/html')
if environ.has_key('paste.registry'):
environ['paste.registry'].register( self.request, request )
environ['paste.registry'].register( self.response,
response )
self.response.body = self.do()
return self.response(environ, start_response)
def do(self):
# add to adhoc attributes of self.request
self.request.foo = 'bar'
# take some action based on request attributes
if self.request.environ.get('PATH_INFO') == '/foo':
return "Body returned when called from /foo"
else:
return "Body returned, default"
# add the registry manager middleware
base_app = BaseController()
dram_app = RegistryManager(base_app)
Mike Orr
What is your main goal? Are you trying to put a non-Pylons WSGI
controller into a Pylons application? Or are you trying to put some
Pylons-like features into a generic WSGI application?
The .__call__ is a WSGI application with the expected environ arg, and
returns a response. So that is where you should create your
webob.Request and Response objects. .__init__ should not know about
them.
After that, you have a choice. If you keep 'request' and 'response'
as local variables and pass them as arguments to every method that
uses them, the instance is thread safe, and you can just instantiate
the class to respond to all requests.
But if you set ``self.request`` and/or ``self.response``, the instance
is not thread safe because simultaneous requests would trample on each
other. In that case you would need to instantiate the object for
every request, or hide it inside a threadlocal.
If you look at the middleware in Paste and the ones Pylons invokes,
you'll see they are all thread safe. I.e., they do not put
request-specific data in 'self'. This makes it possible to
instantiate them as middleware a la: ``app = MyMiddleware(app,
...)``.
In contrast, Pylons assumes its native controllers are not thread
safe, and instantiates one for each request. This is done by the
PylonsApp "middleware", which calculates which controller to invoke.
For details see
http://wiki.pylonshq.com/display/pylonscookbook/Pylons+Execution+Analysis+0.9.6
. (The details are slightly different in 0.9.7, but the outline is
the same.)
Other frameworks handle request/response by passing arguments to the
controller action, or setting 'self' attributes, or providing a global
get_request() function, or using a magic global object that "knows"
about the current request. ``pylons.request`` is the latter. Because
it's a magic global in a multithreaded program, it suffers a
thread-safety problem. Two actions might access pylons.request
simultaneously, each expecting to see its own request. The simplest
solution is to make 'request' a threadlocal. But that fails if two
instances of the *same* application are running in the same process.
For instance, if you used Paste:Composite to mount two instances of a
blog application at different URLs. Then their 'app_globals' would be
different, but both would be accessing the same
``pylons.app_globals``. StackedObjectProxy is a way to overcome that
specific, rarely-encountered problem. The Pylons developers have
debated whether to ditch StackedObjectProxy in favor of ordinary
threadlocals because this use case is so rare. So far they haven't
done so, preferring to be extremely cautious.
The upshot of all this is, your life will be simpler if you avoid
self.request and global variables. If you do want to use
self.request, I'd recommend a two-class approach, where WSGI.__call__
instantiates a worker class to calculate the result. That way the
WSGI class can remain thread-safe while the worker class is not.
As for using Pylons' Request and Response objects in a non-Pylons WSGI
application, Pylons is not set up for that. It knows how to call a
WSGI controller, but not how to pass pylons.request and
pylons.response to it. So your WSGI application would have to
instantiate its own webob.Request and Response, even though that's
duplicative. But a Pylons controller action *can* pass ``request``
and ``response`` to anything it chooses, so you could transform your
WSGI application into a "WebOb application" that takes a request &
response as arguments, and invoke it from an action.
--
Mike Orr <slugg...@gmail.com>
Iain Duncan
>
> What is your main goal? Are you trying to put a non-Pylons WSGI
> controller into a Pylons application? Or are you trying to put some
> Pylons-like features into a generic WSGI application?
My goal is to make my administrative app thing as easy as possible to
drop into multiple environments ( pylons, tg, grok, repoze.bfg, etc ),
yet also keep it as easy to understand as possible. So I do not want
users to have to provide magic-globals as integrating with a framework
requires a pretty solid understanding of the framework. ( And the users
will often be my people on our dime! ) And I want the distinction
between server instantiation time stuff and request stuff to be very
clear.
>
> The .__call__ is a WSGI application with the expected environ arg, and
> returns a response. So that is where you should create your
> webob.Request and Response objects. .__init__ should not know about
> them.
Right, got that, on the right track so far.
>
> After that, you have a choice. If you keep 'request' and 'response'
> as local variables and pass them as arguments to every method that
> uses them, the instance is thread safe, and you can just instantiate
> the class to respond to all requests.
>
> But if you set ``self.request`` and/or ``self.response``, the instance
> is not thread safe because simultaneous requests would trample on each
> other. In that case you would need to instantiate the object for
> every request, or hide it inside a threadlocal.
Ok, so I was hoping that I could make self.request be thread local, and
avoid having long method signatures that need request in each one. So my
docs could explicitly say:
- all attributes of self are not thread safe and should be set only on
startup in __init__, they persist for the live of the app
- *except* self.request, however, which you can treat as a thread safe
version of self. Feel free to add adhoc attributes to self.request, it
will always proxy to this requests request. ( likely to be my own thin
wrapper over WebOb request, much like pylons ).
A prior attempt had used a thread local called self.local, but then I
thought, heck 'request' has to be local anyway, all methods need to be
able to get at environ, and it's very obvious that 'request' is per
request. Plus, storing my request data there means we can inspect it
with the paste debugger.
Sooo, I guess I want a thread-safe self-contained object to hold the
request and other stuff, without resorting to magic globals or anything
provided behind-the-scenes by the framework. ( I expect I will add
adapter classes later for the various target platforms to connect to the
magic-globals if need be. ).
I'm pretty sure I want to keep the pattern of long-lived instantiated
objects that are __called__ per request, because it matches the Webob
example docs, and I don't want to rewrite that stuff, I want to be able
to say: read this first.
>
> Other frameworks handle request/response by passing arguments to the
> controller action, or setting 'self' attributes, or providing a global
> get_request() function, or using a magic global object that "knows"
> about the current request. ``pylons.request`` is the latter. Because
> it's a magic global in a multithreaded program, it suffers a
> thread-safety problem. Two actions might access pylons.request
> simultaneously, each expecting to see its own request. The simplest
> solution is to make 'request' a threadlocal. But that fails if two
> instances of the *same* application are running in the same process.
> For instance, if you used Paste:Composite to mount two instances of a
> blog application at different URLs. Then their 'app_globals' would be
> different, but both would be accessing the same
> ``pylons.app_globals``. StackedObjectProxy is a way to overcome that
> specific, rarely-encountered problem. The Pylons developers have
> debated whether to ditch StackedObjectProxy in favor of ordinary
> threadlocals because this use case is so rare. So far they haven't
> done so, preferring to be extremely cautious.
>
> The upshot of all this is, your life will be simpler if you avoid
> self.request and global variables. If you do want to use
> self.request, I'd recommend a two-class approach, where WSGI.__call__
> instantiates a worker class to calculate the result. That way the
> WSGI class can remain thread-safe while the worker class is not.
What about the alternative, where the wsgi class has a singleton factory
helper to always get the request? I could have self.request be a
property to the factory to get the thread safe request. That's what I
was hoping to achieve somehow with the stacked object proxy, but I am
definitely not fully understanding that approach. And it's a wee bit
hard to test threading edge cases...
If you ( or anyone who really understands this stuff ) could post an
example of how to make sure I set up a thread safe request object, that
would be great.
Thanks again Mike,
Iain
Ian Bicking
Ok, so I was hoping that I could make self.request be thread local, and
avoid having long method signatures that need request in each one. So my
docs could explicitly say:
- all attributes of self are not thread safe and should be set only on
startup in __init__, they persist for the live of the app
- *except* self.request, however, which you can treat as a thread safe
version of self. Feel free to add adhoc attributes to self.request, it
will always proxy to this requests request. ( likely to be my own thin
wrapper over WebOb request, much like pylons ).
I'd do it like Pylons, which instantiates the controller on each request. This means people will be much less likely to have thread problems in their code, and anything you can do to avoid that would make things easier.
But if you aren't doing this, but just want an attribute to be request-local, I wouldn't used a proxy object. Instead use a property that gets the current request. But really, if you can avoid the whole thing by instantiating the controller on each request, that would be best.
--
Ian Bicking | http://blog.ianbicking.org