Tools and WSGI

[dbrattli]

Hi,

I'm trying to make some stuff and I'm not sure if it's even possible
CherryPy What I want to do is to sort of merge tools and wsgi
middleware to get the best of both worlds. I often find myself
wrapping WSGI middleware as a CP tool for WSGI stuff I want to have
the same flexibility as with CP tools and be able to have finer
granularity and not having to run though all WSGI middleware for all
exposed methods. But I need to do a lot of work wrapping WSGI
middleware as tools and and it's not a scalable solution to wrap all
WSGI pluings as tools.

So I would love to have the possibility of having a *single* CP tool
that could wrap any WSGI middleware and that you could thus enable the
wsgi middleware in the same way as you do with tools (_cp_config,
decorator etc). Or even specify the WSGI complete pipeline (or append
to) for each exposed method using config. Something like:

class Controller():
def index(self):
return "hello world"
index._cp_config = { 'tools.wsgi.pipeline' : [('mywsgi', MyWSGI)] }

or decorate:

@tools.wsgi.pipeline([('mywsgi', MyWSGI)] )
def index ...

I tried making this by having a CP ool that would set wsgi config on
an exposed method or class, and have one WSGI plugin to rule them all.
The WSGI plugin would read the config and decide the WSGI pipeline to
use based on the config. The crux is that dispatch has not yet run
when the WSGI pipeline is being run, and I don't want to go back into
the WSGI world at tool time. I think I could probably make this work
by making a WSGI middleware dispatcher that runs
(cherrypy.request.dispatch) first, so we know the method
(request.handler) and config and thus be able to create the WSGI
pipeline "on-the-fly":

1) CPWSGIApp
2) WSGI middleware, any wsgi that needs to run before dispatch
3) WSGI dispatcher (get_resource etc)
4) WSGI pipeline decided by config [(name, class),..]
5) WSGI -> CP adaptor
6) Tools
7) Handler

But there are other parts of request handling in 5) that needs to run
before request.get_resource() such as request.run() and I would have
to do all that stuff in the first WSGI plugin to be able to dispatch,
and I don't want to to do all this stuff twice. So all of this would
require a bit of changes to CherryPy, and I thought I should probably
ask this list to check if I'm "far-out-fishing", or if this could
actually be a nice idea? If it's a nice and workable idea, then what's
the best way to go ahead? Comments?

-- Dag

[Sylvain Hellegouarch]

> But there are other parts of request handling in 5) that needs to run
> before request.get_resource() such as request.run() and I would have
> to do all that stuff in the first WSGI plugin to be able to dispatch,
> and I don't want to to do all this stuff twice. So all of this would
> require a bit of changes to CherryPy, and I thought I should probably
> ask this list to check if I'm "far-out-fishing", or if this could
> actually be a nice idea? If it's a nice and workable idea, then what's
> the best way to go ahead? Comments?
>

My first reaction is that it is indeed far fetched. With that said, the
context of your requirements escape me so it might be more relevant than
at first sight.

Could you highlight what middlewares you are actually using? I'm surprised
you have so many of them, I think I miss most of the context.

- Sylvain

--
Sylvain Hellegouarch
http://www.defuze.org

[dbrattli]

Hi,

I'm thinking about the next generation Python web frameworks that uses
a lot of WSGI components such as:

- repoze.who (authentication)
- repose.what (authorization)
- repoze.tm (transactions)
- toscawidgets (forms, js)
- error handling
- sessing handling
- caching

For many more see http://repoze.org/repoze_components.html#middleware

CherryPy has no problem handling this, but the stack gets big, and
every controller in your framework will have to suffer because of the
one that needs them all. It would be very nice if you configure the
pipeline per class/method using _cp_config, so your "simple"
controllers does not need to run through them all. But it's not that
easy when dispatching is done after all the wsgi stuff. I know that CP
tools can be configured this way, and I've wrapped stuff like
toscawidgets and repoze.tm http://trac.turbogears.org/attachment/ticket/1065/transactions-tg15-2.diff.
But I haven't got time to wrap them all, and would really like to find
a better way. It possible to mount several applications and have a
different pipeline for each of them, but it's not as flexible and easy
as doing it per class/method with config.

-- Dag

[Sylvain Hellegouarch]

I see.

My immediate question then is why using CherryPy in such case? I'm finding
the idea of having ready-made tools like you suggest attractive but one
might argue we could as easily use Pylons for that. No?

Anyway, I digress from your initial question I guess. At first sight I
don't much idea on how to do what you described. Or perhaps could we turn
the problem the other way around, do those middlewares expose APIs (not a
WSGI one per se) that mean we could directly right tools calling those
APIs instead?

I'm thinking along the lines of how cherrypy.lib.* is used within the
cherrypy._cptools module.

The functionalities are not implemented by the tools themselves, the tools
only interface them to CP.

- Sylvain