Paste/PasteDeploy, TurboGears 0.9 and CherryPy 2.2
Kevin Dangoor
doesn't ship software.
TurboGears 0.9 has *a lot* of great work that people have done. Right
now, use of 0.9 is limited to the folks who are knowledgeable enough
and brave enough to follow the TurboGears trunk as it undergoes
changes. I think that's a shame, because many people are missing out
on the excellent features we have because of rough areas in certain
spots. DataController, for one, is just a sketch of what it can
become. But, I don't want to hold up everything for DataController.
Jeff has designed an identity API that is amazingly user friendly for
common cases. I would like to brainstorm a bit on how this API might
be tweaked to use RuleDispatch syntax which would provide more
flexibility going forward. That brainstorming should be in a separate
thread. I'd like to do that brainstorming before 0.9, because I'd
rather not have such a large backwards-incompatible change afterwards.
Other than that, the tweaks and fixes that are in Trac are the goal
for 0.9 right now. Keep in mind that 0.9 is not the end, it's just a
major step up on the way to 1.0 1.0 is not the end, either.
There is one other thing that needs to appear in 0.9: CherryPy 2.2.
TurboGears 1.0 will use CherryPy 2.2, and making the switch sooner
than later is important so that we can get some testing in and make
sure any problems we encounter are fixed while 2.2 is in development.
Which brings me to PasteDeploy. Ian has been keeping up with the
CherryPy 2.2 changes and trying to make sure that the level of WSGI
support needed for PasteDeploy is met by CP 2.2's more flexible
routing. The CP2.2 transition is something we can make for TG0.9, but
the PasteDeploy transition is not. I do think that using PasteDeploy
is a good goal and that it should be done right when CP is ready for
it.
I just wanted to bring a bit more definition to "what is 0.9?" so that
it's clear that we're heading to the end of the cycle there. We can
open up branches in Subversion for people who want to experiment with
post 0.9 features...
(As with anything, these are my thoughts and it's all open for
discussion. But I, for one, am *very* keen to ship 0.9 *soon*.)
Kevin
--
Kevin Dangoor
Author of the Zesty News RSS newsreader
email: k...@blazingthings.com
company: http://www.BlazingThings.com
blog: http://www.BlueSkyOnMars.com
Jonathan LaCour
> Which brings me to PasteDeploy. Ian has been keeping up with the
> CherryPy 2.2 changes and trying to make sure that the level of WSGI
> support needed for PasteDeploy is met by CP 2.2's more flexible
> routing. The CP2.2 transition is something we can make for TG0.9, but
> the PasteDeploy transition is not. I do think that using PasteDeploy
> is a good goal and that it should be done right when CP is ready for
> it.
Does this only apply to 0.9, or does it also apply for 1.0? I have
no issue with not having Paste Deploy in 0.9, as I can see the clear
need to get a release out the door. But, I am worried about having
the first really frozen release go out the door without Paste Deploy.
Just curious. I know I am harping on this stuff lately, but its
something that I think is a clear win for TurboGears.
--
Jonathan LaCour
http://cleverdevil.org
Kevin Dangoor
>
> On Jan 11, 2006, at 10:06 AM, Kevin Dangoor wrote:
> > Which brings me to PasteDeploy. Ian has been keeping up with the
> > CherryPy 2.2 changes and trying to make sure that the level of WSGI
> > support needed for PasteDeploy is met by CP 2.2's more flexible
> > routing. The CP2.2 transition is something we can make for TG0.9, but
> > the PasteDeploy transition is not. I do think that using PasteDeploy
> > is a good goal and that it should be done right when CP is ready for
> > it.
>
> Does this only apply to 0.9, or does it also apply for 1.0? I have
> no issue with not having Paste Deploy in 0.9, as I can see the clear
> need to get a release out the door. But, I am worried about having
> the first really frozen release go out the door without Paste Deploy.
Assuming all of the CP issues are dealt with, I can see TurboGears 1.0
using PasteDeploy.
Kevin
Dave Warnock
Thanks for your response on these issues. I think you have made a good
call on this. The project benefits greatly from being clearly managed in
this kind of way and I think the direction is a good one.
Paste deploy is going to be really good and the way TurboGears uses
external components is fantastic so it is tempting for those of us who
are keen on wsgi and paste to keep pushing them. yet is is also
important to get 0.9 released and I think you are right that it will be
good to get CP 2.2 into 0.9 and then have a little stability before
moving identity into Paste middleware once Paste deploy is used by TG
sometime before 1.0
Dave
Michael Schneider
for cherrypy 2.2
I am fine with putting off paste for 1.0
It seems like 2.2 + paste would be a big step for 0.9.
Michele Cella
let many more people (that ATM don't want to use a svn thing because
they feel it as unstable) play with the new features and give a
realistic feedback to developers.
As you said CP 2.2 is on the right track for WSGI/Paste support and
0.9/1.0 is just the beginning not the end.
Let's make 0.9a ASAP and move gradually (not slowly) towards 1.0 as you
already said some times ago.
Ciao
Michele
Kevin Dangoor wrote:
> A project that tries to make everything perfect is a project that
> doesn't ship software.
>
> TurboGears 0.9 has *a lot* of great work that people have done. Right
> now, use of 0.9 is limited to the folks who are knowledgeable enough
> and brave enough to follow the TurboGears trunk as it undergoes
> changes. I think that's a shame, because many people are missing out
> on the excellent features we have because of rough areas in certain
> spots. DataController, for one, is just a sketch of what it can
> become. But, I don't want to hold up everything for DataController.
>
> ...
Ian Bicking
> Jeff has designed an identity API that is amazingly user friendly for
> common cases. I would like to brainstorm a bit on how this API might
> be tweaked to use RuleDispatch syntax which would provide more
> flexibility going forward. That brainstorming should be in a separate
> thread. I'd like to do that brainstorming before 0.9, because I'd
> rather not have such a large backwards-incompatible change afterwards.
If you are thinking about RuleDispatch, you should probably consider
peak.security: http://peak.telecommunity.com/DevCenter/SecurityRules
> Which brings me to PasteDeploy. Ian has been keeping up with the
> CherryPy 2.2 changes and trying to make sure that the level of WSGI
> support needed for PasteDeploy is met by CP 2.2's more flexible
> routing. The CP2.2 transition is something we can make for TG0.9, but
> the PasteDeploy transition is not. I do think that using PasteDeploy
> is a good goal and that it should be done right when CP is ready for
> it.
Right now CherryPy is the blocker, and once it is working there I think
that should transfer to TG fairly easily, possibly (hopefully) even
retroactively (e.g., if you are running TG 0.9 with whatever version of
CherryPy where this is working). Right now CherryPy isn't offering a
sufficient WSGI-related API, and I don't know if that's because of
underlying architectural issues, or that the APIs just don't expose what
they should.
--
Ian Bicking / ia...@colorstudy.com / http://blog.ianbicking.org
Michele Cella
>
> If you are thinking about RuleDispatch, you should probably consider
> peak.security: http://peak.telecommunity.com/DevCenter/SecurityRules
>
This in fact was already suggested by PJE, I opened a ticket some days
ago:
http://trac.turbogears.org/turbogears/ticket/313
>
> Right now CherryPy is the blocker, and once it is working there I think
> that should transfer to TG fairly easily, possibly (hopefully) even
> retroactively (e.g., if you are running TG 0.9 with whatever version of
> CherryPy where this is working). Right now CherryPy isn't offering a
> sufficient WSGI-related API, and I don't know if that's because of
> underlying architectural issues, or that the APIs just don't expose what
> they should.
>
Just wondering if you have already seen last Fumanchu's reply to your
questions on ticket #145:
http://www.cherrypy.org/ticket/145
Ciao
Michele
Kevin Dangoor
>
> Kevin Dangoor wrote:
> > Jeff has designed an identity API that is amazingly user friendly for
> > common cases. I would like to brainstorm a bit on how this API might
> > be tweaked to use RuleDispatch syntax which would provide more
> > flexibility going forward. That brainstorming should be in a separate
> > thread. I'd like to do that brainstorming before 0.9, because I'd
> > rather not have such a large backwards-incompatible change afterwards.
>
> If you are thinking about RuleDispatch, you should probably consider
> peak.security: http://peak.telecommunity.com/DevCenter/SecurityRules
Yep, I've looked at that. What I go after, design-wise, with
TurboGears APIs is that they should be super simple for the common
case and, as much as possible, provide a graceful migration path
upwards as needs get more complex. That's a tough balancing act, but I
also think that doing that well (or at least trying to!) leads to the
best possible product.
For typical roles/permissions usage applied as needed to various parts
of this system, Jeff's API is awfully convenient. But, I think that
judicious use of RuleDispatch could make it possible to go beyond the
current API without immediately having to jump to a completely custom
IdentityProvider.
Kevin
Ian Bicking
>>Right now CherryPy is the blocker, and once it is working there I think
>>that should transfer to TG fairly easily, possibly (hopefully) even
>>retroactively (e.g., if you are running TG 0.9 with whatever version of
>>CherryPy where this is working). Right now CherryPy isn't offering a
>>sufficient WSGI-related API, and I don't know if that's because of
>>underlying architectural issues, or that the APIs just don't expose what
>>they should.
>>
>
>
> Just wondering if you have already seen last Fumanchu's reply to your
> questions on ticket #145:
>
> http://www.cherrypy.org/ticket/145
I had not seen that, thanks for noting that. Added myself to CC as
well. As I look through the CherryPy code, maybe it would be possible
to figure this out with a sufficiently magical cherrypy.root object,
though I suspect some serious monkeypatching to cherrypy.config might
also be necessary (to make up for no complete and global set of
configuration), and I might just need to make cherrypy.root a
threadlocal object. That's probably the thing most likely to work.
A positive part of a monkeypatch is that it will be immediately usable
with TG, probably/hopefully without TG needing to know that. Though I
suspect there may be some configuration issues... is TG still using
CherryPy's configuration, or is it starting to handle that on its own?
Kevin Dangoor
> well. As I look through the CherryPy code, maybe it would be possible
> to figure this out with a sufficiently magical cherrypy.root object,
> though I suspect some serious monkeypatching to cherrypy.config might
> also be necessary (to make up for no complete and global set of
> configuration), and I might just need to make cherrypy.root a
> threadlocal object. That's probably the thing most likely to work.
I'm curious to know what the use cases are that actually don't work
with the new CP implementation.
cherrypy.request and cherrypy.response are threadlocal proxy objects
already, if you feel that such an approach is required. That was what
I had originally expected would be done in CP, but fumanchu decided to
change the API a little bit with the tree mounting stuff (which is
fine).
> A positive part of a monkeypatch is that it will be immediately usable
> with TG, probably/hopefully without TG needing to know that. Though I
> suspect there may be some configuration issues... is TG still using
> CherryPy's configuration, or is it starting to handle that on its own?
TurboGears still uses CherryPy's configuration, but the files have
changed to Python modules that return a nested dictionary. This can be
transformed if need be.
Kevin
Ian Bicking
> On 1/11/06, Ian Bicking <ia...@colorstudy.com> wrote:
>
>>Kevin Dangoor wrote:
>>
>>>Jeff has designed an identity API that is amazingly user friendly for
>>>common cases. I would like to brainstorm a bit on how this API might
>>>be tweaked to use RuleDispatch syntax which would provide more
>>>flexibility going forward. That brainstorming should be in a separate
>>>thread. I'd like to do that brainstorming before 0.9, because I'd
>>>rather not have such a large backwards-incompatible change afterwards.
>>
>>If you are thinking about RuleDispatch, you should probably consider
>>peak.security: http://peak.telecommunity.com/DevCenter/SecurityRules
>
>
> Yep, I've looked at that. What I go after, design-wise, with
> TurboGears APIs is that they should be super simple for the common
> case and, as much as possible, provide a graceful migration path
> upwards as needs get more complex. That's a tough balancing act, but I
> also think that doing that well (or at least trying to!) leads to the
> best possible product.
I think the basic setup would be a set of rules based on data. The data
might look like:
{
'/admin': ['admin', 'editor'],
'/comment': ['valid-user'],
}
And a rule would check if the URL was prefixed by something in that
dictionary, and check against the roles for that URL. Another
complimentary option would be to add the resource (exposed object) to
the context of the action, and use an attribute on that to indicate
roles. These rules can probably just be installed globally, and then
specialized if necessary, especially because there's the two-phase
aspect of peak.security -- the rules indicating whether a function even
applies, and then an indication about whether the user is then
permitted. So these default rules would, if the user didn't use them
(implementing their own system) simply not apply.
Kevin Dangoor
My thought is to maintain the same basic idea that Jeff has come up
with: when locking down a single method, put the lock right there
(with a decorator). When locking down an object or tree, put the lock
right on that class. Use an external configuration system for it only
as needed (and many apps don't need it).
Kevin
Ian Bicking
> On 1/11/06, Ian Bicking <ia...@colorstudy.com> wrote:
>
>>I had not seen that, thanks for noting that. Added myself to CC as
>>well. As I look through the CherryPy code, maybe it would be possible
>>to figure this out with a sufficiently magical cherrypy.root object,
>>though I suspect some serious monkeypatching to cherrypy.config might
>>also be necessary (to make up for no complete and global set of
>>configuration), and I might just need to make cherrypy.root a
>>threadlocal object. That's probably the thing most likely to work.
>
>
> I'm curious to know what the use cases are that actually don't work
> with the new CP implementation.
With Paste Deploy, an application only knows where it lives when a
request is run, which is the primary issue. I imagine the basic setup
looking like:
[main:app]
root_object = mypackage.root
Though I'm not sure exactly how TG does it, I'm just thinking from
CherryPy now. When a request is sent to this application, SCRIPT_NAME
indicates where the application was found -- it isn't known ahead of
time. This means various programmatic dispatching tricks are possible.
And it also just happens to be the only way Paste Deploy communicates
this base URL to the application.
The tree stuff in CherryPy assumes that there is a single fixed url
where each object is mounted, and then I think builds up the
cherrypy.root object based on that.
The way I am experimenting with now is for cherrypy.root itself to be a
threadlocal object, and then a request comes in we make that object act
like whatever the configured root is (i.e., dispatching all attribute to
mypackage.root). Then we don't have to build up a fake tree at all,
instead it always looks to CherryPy as though there is only one
application being served. (Configuration also has to be fixed up like
this, which seems less obvious to do, but probably reasonable too).
It's a little wonky feeling -- if the root object was just an argument
instead of a global (which is really all I'm looking for) then this
would feel much more elegant. And frankly the code would look better
too. But a threadlocal isn't so bad, all considered -- it feels a lot
cleaner to me than constructing a tree, and is fairly easy to put into
place.
I haven't tested this at all (not even syntax ;) and I really should be
doing other things at the moment so I won't test it for a while, but a
basic prototype is at http://svn.colorstudy.com/home/ianb/CherryPaste/trunk
> cherrypy.request and cherrypy.response are threadlocal proxy objects
> already, if you feel that such an approach is required. That was what
> I had originally expected would be done in CP, but fumanchu decided to
> change the API a little bit with the tree mounting stuff (which is
> fine).
>
>
>>A positive part of a monkeypatch is that it will be immediately usable
>>with TG, probably/hopefully without TG needing to know that. Though I
>>suspect there may be some configuration issues... is TG still using
>>CherryPy's configuration, or is it starting to handle that on its own?
>
>
> TurboGears still uses CherryPy's configuration, but the files have
> changed to Python modules that return a nested dictionary. This can be
> transformed if need be.
Paste Deploy uses pretty dumb ini files, which isn't that far from
CherryPy's normal configuration, which is good. OTOH, I see the issue
you are trying to deal with -- lots of "configuration" is really about
describing how the application is built, and isn't something you
configure on a per-installation basis. The ambiguity comes in when
"application" becomes confused -- I think the original CherryPy
perspective was that CherryPy is the application, and you configure it
with respect to your Python code. As the perspective shifts towards a
framework, the application's configuration and the framework's
configuration can become confused.
As I've been encountering this, I've been writing translation functions
basically. They take the (minimal) application configuration, and apply
defaults (which are often calculated) and often add in framework
configuration (like that 'root_object'). So with Paste Deploy you would
probably not point to CherryPaste in your entry point, but to some
module (I've been calling it wsgiapp) and doing:
def make_app(global_conf, **app_conf):
return cherrypaste.pastify.make_app(
global_conf, root_object='myapp.root', **app_conf)
Another way to handle this would be to import a module, and add all the
global variables from that module, which approaches what I think you are
doing.
But anyway, if you can handle a degree of ini-based configuration (with
the slightly different semantics that implies, such as things like lists
and dicts are hard to construct compared to Python code) then I think
that should make it easier to apply Paste later.
Ian Bicking
make CherryPy applications compatible with Paste Deploy. I haven't
really used it seriously, but my very tiny and incomplete test works
okay; feedback is very much welcome. The idea isn't to magically
support all CherryPy applications, but to support those that try.
Specifically messing with cherrypy.root won't work, but you use the
Paste Deploy configuration to let it set up cherrypy.root for you (it
ends up being threadlocal, and each application is the "root" for its
request).
Anyway, you can install it with:
easy_install CherryPaste==dev
Probably better to use "-e -b .", because this really isn't ready for
any real use, I'm just looking for feedback.
TG 0.9 doesn't have to include this, but it would be good if TG worked
okay with this, or had some way you could get to that would work well
with this (something that does less magic).
A small website at: http://pythonpaste.org/cherrypaste/
--
Ian Bicking | ia...@colorstudy.com | http://blog.ianbicking.org