A project that tries to make everything perfect is a project that doesn't ship software.
TurboGears 0.9 has *a lot* of great work that people have done. Right now, use of 0.9 is limited to the folks who are knowledgeable enough and brave enough to follow the TurboGears trunk as it undergoes changes. I think that's a shame, because many people are missing out on the excellent features we have because of rough areas in certain spots. DataController, for one, is just a sketch of what it can become. But, I don't want to hold up everything for DataController.
Jeff has designed an identity API that is amazingly user friendly for common cases. I would like to brainstorm a bit on how this API might be tweaked to use RuleDispatch syntax which would provide more flexibility going forward. That brainstorming should be in a separate thread. I'd like to do that brainstorming before 0.9, because I'd rather not have such a large backwards-incompatible change afterwards.
Other than that, the tweaks and fixes that are in Trac are the goal for 0.9 right now. Keep in mind that 0.9 is not the end, it's just a major step up on the way to 1.0 1.0 is not the end, either.
There is one other thing that needs to appear in 0.9: CherryPy 2.2. TurboGears 1.0 will use CherryPy 2.2, and making the switch sooner than later is important so that we can get some testing in and make sure any problems we encounter are fixed while 2.2 is in development.
Which brings me to PasteDeploy. Ian has been keeping up with the CherryPy 2.2 changes and trying to make sure that the level of WSGI support needed for PasteDeploy is met by CP 2.2's more flexible routing. The CP2.2 transition is something we can make for TG0.9, but the PasteDeploy transition is not. I do think that using PasteDeploy is a good goal and that it should be done right when CP is ready for it.
I just wanted to bring a bit more definition to "what is 0.9?" so that it's clear that we're heading to the end of the cycle there. We can open up branches in Subversion for people who want to experiment with post 0.9 features...
(As with anything, these are my thoughts and it's all open for discussion. But I, for one, am *very* keen to ship 0.9 *soon*.)
Kevin
-- Kevin Dangoor Author of the Zesty News RSS newsreader
On Jan 11, 2006, at 10:06 AM, Kevin Dangoor wrote:
> Which brings me to PasteDeploy. Ian has been keeping up with the > CherryPy 2.2 changes and trying to make sure that the level of WSGI > support needed for PasteDeploy is met by CP 2.2's more flexible > routing. The CP2.2 transition is something we can make for TG0.9, but > the PasteDeploy transition is not. I do think that using PasteDeploy > is a good goal and that it should be done right when CP is ready for > it.
Does this only apply to 0.9, or does it also apply for 1.0? I have no issue with not having Paste Deploy in 0.9, as I can see the clear need to get a release out the door. But, I am worried about having the first really frozen release go out the door without Paste Deploy.
Just curious. I know I am harping on this stuff lately, but its something that I think is a clear win for TurboGears.
On 1/11/06, Jonathan LaCour <jonathan-li...@cleverdevil.org> wrote:
> On Jan 11, 2006, at 10:06 AM, Kevin Dangoor wrote: > > Which brings me to PasteDeploy. Ian has been keeping up with the > > CherryPy 2.2 changes and trying to make sure that the level of WSGI > > support needed for PasteDeploy is met by CP 2.2's more flexible > > routing. The CP2.2 transition is something we can make for TG0.9, but > > the PasteDeploy transition is not. I do think that using PasteDeploy > > is a good goal and that it should be done right when CP is ready for > > it.
> Does this only apply to 0.9, or does it also apply for 1.0? I have > no issue with not having Paste Deploy in 0.9, as I can see the clear > need to get a release out the door. But, I am worried about having > the first really frozen release go out the door without Paste Deploy.
Assuming all of the CP issues are dealt with, I can see TurboGears 1.0 using PasteDeploy.
> On 1/11/06, Jonathan LaCour <jonathan-li...@cleverdevil.org> wrote:
>>On Jan 11, 2006, at 10:06 AM, Kevin Dangoor wrote:
>>>Which brings me to PasteDeploy. Ian has been keeping up with the >>>CherryPy 2.2 changes and trying to make sure that the level of WSGI >>>support needed for PasteDeploy is met by CP 2.2's more flexible >>>routing. The CP2.2 transition is something we can make for TG0.9, but >>>the PasteDeploy transition is not. I do think that using PasteDeploy >>>is a good goal and that it should be done right when CP is ready for >>>it.
>>Does this only apply to 0.9, or does it also apply for 1.0? I have >>no issue with not having Paste Deploy in 0.9, as I can see the clear >>need to get a release out the door. But, I am worried about having >>the first really frozen release go out the door without Paste Deploy.
> Assuming all of the CP issues are dealt with, I can see TurboGears 1.0 > using PasteDeploy.
Thanks for your response on these issues. I think you have made a good call on this. The project benefits greatly from being clearly managed in this kind of way and I think the direction is a good one.
Paste deploy is going to be really good and the way TurboGears uses external components is fantastic so it is tempting for those of us who are keen on wsgi and paste to keep pushing them. yet is is also important to get 0.9 released and I think you are right that it will be good to get CP 2.2 into 0.9 and then have a little stability before moving identity into Paste middleware once Paste deploy is used by TG sometime before 1.0
Agreed, shipping 0.9 soon is really an important step for TG that will let many more people (that ATM don't want to use a svn thing because they feel it as unstable) play with the new features and give a realistic feedback to developers.
As you said CP 2.2 is on the right track for WSGI/Paste support and 0.9/1.0 is just the beginning not the end.
Let's make 0.9a ASAP and move gradually (not slowly) towards 1.0 as you already said some times ago.
Kevin Dangoor wrote: > A project that tries to make everything perfect is a project that > doesn't ship software.
> TurboGears 0.9 has *a lot* of great work that people have done. Right > now, use of 0.9 is limited to the folks who are knowledgeable enough > and brave enough to follow the TurboGears trunk as it undergoes > changes. I think that's a shame, because many people are missing out > on the excellent features we have because of rough areas in certain > spots. DataController, for one, is just a sketch of what it can > become. But, I don't want to hold up everything for DataController.
Kevin Dangoor wrote: > Jeff has designed an identity API that is amazingly user friendly for > common cases. I would like to brainstorm a bit on how this API might > be tweaked to use RuleDispatch syntax which would provide more > flexibility going forward. That brainstorming should be in a separate > thread. I'd like to do that brainstorming before 0.9, because I'd > rather not have such a large backwards-incompatible change afterwards.
> Which brings me to PasteDeploy. Ian has been keeping up with the > CherryPy 2.2 changes and trying to make sure that the level of WSGI > support needed for PasteDeploy is met by CP 2.2's more flexible > routing. The CP2.2 transition is something we can make for TG0.9, but > the PasteDeploy transition is not. I do think that using PasteDeploy > is a good goal and that it should be done right when CP is ready for > it.
Right now CherryPy is the blocker, and once it is working there I think that should transfer to TG fairly easily, possibly (hopefully) even retroactively (e.g., if you are running TG 0.9 with whatever version of CherryPy where this is working). Right now CherryPy isn't offering a sufficient WSGI-related API, and I don't know if that's because of underlying architectural issues, or that the APIs just don't expose what they should.
> Right now CherryPy is the blocker, and once it is working there I think > that should transfer to TG fairly easily, possibly (hopefully) even > retroactively (e.g., if you are running TG 0.9 with whatever version of > CherryPy where this is working). Right now CherryPy isn't offering a > sufficient WSGI-related API, and I don't know if that's because of > underlying architectural issues, or that the APIs just don't expose what > they should.
Just wondering if you have already seen last Fumanchu's reply to your questions on ticket #145:
On 1/11/06, Ian Bicking <i...@colorstudy.com> wrote:
> Kevin Dangoor wrote: > > Jeff has designed an identity API that is amazingly user friendly for > > common cases. I would like to brainstorm a bit on how this API might > > be tweaked to use RuleDispatch syntax which would provide more > > flexibility going forward. That brainstorming should be in a separate > > thread. I'd like to do that brainstorming before 0.9, because I'd > > rather not have such a large backwards-incompatible change afterwards.
Yep, I've looked at that. What I go after, design-wise, with TurboGears APIs is that they should be super simple for the common case and, as much as possible, provide a graceful migration path upwards as needs get more complex. That's a tough balancing act, but I also think that doing that well (or at least trying to!) leads to the best possible product.
For typical roles/permissions usage applied as needed to various parts of this system, Jeff's API is awfully convenient. But, I think that judicious use of RuleDispatch could make it possible to go beyond the current API without immediately having to jump to a completely custom IdentityProvider.
Michele Cella wrote: >>Right now CherryPy is the blocker, and once it is working there I think >>that should transfer to TG fairly easily, possibly (hopefully) even >>retroactively (e.g., if you are running TG 0.9 with whatever version of >>CherryPy where this is working). Right now CherryPy isn't offering a >>sufficient WSGI-related API, and I don't know if that's because of >>underlying architectural issues, or that the APIs just don't expose what >>they should.
> Just wondering if you have already seen last Fumanchu's reply to your > questions on ticket #145:
I had not seen that, thanks for noting that. Added myself to CC as well. As I look through the CherryPy code, maybe it would be possible to figure this out with a sufficiently magical cherrypy.root object, though I suspect some serious monkeypatching to cherrypy.config might also be necessary (to make up for no complete and global set of configuration), and I might just need to make cherrypy.root a threadlocal object. That's probably the thing most likely to work.
A positive part of a monkeypatch is that it will be immediately usable with TG, probably/hopefully without TG needing to know that. Though I suspect there may be some configuration issues... is TG still using CherryPy's configuration, or is it starting to handle that on its own?
On 1/11/06, Ian Bicking <i...@colorstudy.com> wrote:
> I had not seen that, thanks for noting that. Added myself to CC as > well. As I look through the CherryPy code, maybe it would be possible > to figure this out with a sufficiently magical cherrypy.root object, > though I suspect some serious monkeypatching to cherrypy.config might > also be necessary (to make up for no complete and global set of > configuration), and I might just need to make cherrypy.root a > threadlocal object. That's probably the thing most likely to work.
I'm curious to know what the use cases are that actually don't work with the new CP implementation.
cherrypy.request and cherrypy.response are threadlocal proxy objects already, if you feel that such an approach is required. That was what I had originally expected would be done in CP, but fumanchu decided to change the API a little bit with the tree mounting stuff (which is fine).
> A positive part of a monkeypatch is that it will be immediately usable > with TG, probably/hopefully without TG needing to know that. Though I > suspect there may be some configuration issues... is TG still using > CherryPy's configuration, or is it starting to handle that on its own?
TurboGears still uses CherryPy's configuration, but the files have changed to Python modules that return a nested dictionary. This can be transformed if need be.
Kevin Dangoor wrote: > On 1/11/06, Ian Bicking <i...@colorstudy.com> wrote:
>>Kevin Dangoor wrote:
>>>Jeff has designed an identity API that is amazingly user friendly for >>>common cases. I would like to brainstorm a bit on how this API might >>>be tweaked to use RuleDispatch syntax which would provide more >>>flexibility going forward. That brainstorming should be in a separate >>>thread. I'd like to do that brainstorming before 0.9, because I'd >>>rather not have such a large backwards-incompatible change afterwards.
> Yep, I've looked at that. What I go after, design-wise, with > TurboGears APIs is that they should be super simple for the common > case and, as much as possible, provide a graceful migration path > upwards as needs get more complex. That's a tough balancing act, but I > also think that doing that well (or at least trying to!) leads to the > best possible product.
I think the basic setup would be a set of rules based on data. The data might look like:
And a rule would check if the URL was prefixed by something in that dictionary, and check against the roles for that URL. Another complimentary option would be to add the resource (exposed object) to the context of the action, and use an attribute on that to indicate roles. These rules can probably just be installed globally, and then specialized if necessary, especially because there's the two-phase aspect of peak.security -- the rules indicating whether a function even applies, and then an indication about whether the user is then permitted. So these default rules would, if the user didn't use them (implementing their own system) simply not apply.
> And a rule would check if the URL was prefixed by something in that > dictionary, and check against the roles for that URL. Another > complimentary option would be to add the resource (exposed object) to > the context of the action, and use an attribute on that to indicate > roles. These rules can probably just be installed globally, and then > specialized if necessary, especially because there's the two-phase > aspect of peak.security -- the rules indicating whether a function even > applies, and then an indication about whether the user is then > permitted. So these default rules would, if the user didn't use them > (implementing their own system) simply not apply.
My thought is to maintain the same basic idea that Jeff has come up with: when locking down a single method, put the lock right there (with a decorator). When locking down an object or tree, put the lock right on that class. Use an external configuration system for it only as needed (and many apps don't need it).
Kevin Dangoor wrote: > On 1/11/06, Ian Bicking <i...@colorstudy.com> wrote:
>>I had not seen that, thanks for noting that. Added myself to CC as >>well. As I look through the CherryPy code, maybe it would be possible >>to figure this out with a sufficiently magical cherrypy.root object, >>though I suspect some serious monkeypatching to cherrypy.config might >>also be necessary (to make up for no complete and global set of >>configuration), and I might just need to make cherrypy.root a >>threadlocal object. That's probably the thing most likely to work.
> I'm curious to know what the use cases are that actually don't work > with the new CP implementation.
With Paste Deploy, an application only knows where it lives when a request is run, which is the primary issue. I imagine the basic setup looking like:
[main:app] root_object = mypackage.root
Though I'm not sure exactly how TG does it, I'm just thinking from CherryPy now. When a request is sent to this application, SCRIPT_NAME indicates where the application was found -- it isn't known ahead of time. This means various programmatic dispatching tricks are possible. And it also just happens to be the only way Paste Deploy communicates this base URL to the application.
The tree stuff in CherryPy assumes that there is a single fixed url where each object is mounted, and then I think builds up the cherrypy.root object based on that.
The way I am experimenting with now is for cherrypy.root itself to be a threadlocal object, and then a request comes in we make that object act like whatever the configured root is (i.e., dispatching all attribute to mypackage.root). Then we don't have to build up a fake tree at all, instead it always looks to CherryPy as though there is only one application being served. (Configuration also has to be fixed up like this, which seems less obvious to do, but probably reasonable too).
It's a little wonky feeling -- if the root object was just an argument instead of a global (which is really all I'm looking for) then this would feel much more elegant. And frankly the code would look better too. But a threadlocal isn't so bad, all considered -- it feels a lot cleaner to me than constructing a tree, and is fairly easy to put into place.
I haven't tested this at all (not even syntax ;) and I really should be doing other things at the moment so I won't test it for a while, but a basic prototype is at http://svn.colorstudy.com/home/ianb/CherryPaste/trunk
> cherrypy.request and cherrypy.response are threadlocal proxy objects > already, if you feel that such an approach is required. That was what > I had originally expected would be done in CP, but fumanchu decided to > change the API a little bit with the tree mounting stuff (which is > fine).
>>A positive part of a monkeypatch is that it will be immediately usable >>with TG, probably/hopefully without TG needing to know that. Though I >>suspect there may be some configuration issues... is TG still using >>CherryPy's configuration, or is it starting to handle that on its own?
> TurboGears still uses CherryPy's configuration, but the files have > changed to Python modules that return a nested dictionary. This can be > transformed if need be.
Paste Deploy uses pretty dumb ini files, which isn't that far from CherryPy's normal configuration, which is good. OTOH, I see the issue you are trying to deal with -- lots of "configuration" is really about describing how the application is built, and isn't something you configure on a per-installation basis. The ambiguity comes in when "application" becomes confused -- I think the original CherryPy perspective was that CherryPy is the application, and you configure it with respect to your Python code. As the perspective shifts towards a framework, the application's configuration and the framework's configuration can become confused.
As I've been encountering this, I've been writing translation functions basically. They take the (minimal) application configuration, and apply defaults (which are often calculated) and often add in framework configuration (like that 'root_object'). So with Paste Deploy you would probably not point to CherryPaste in your entry point, but to some module (I've been calling it wsgiapp) and doing:
Another way to handle this would be to import a module, and add all the global variables from that module, which approaches what I think you are doing.
But anyway, if you can handle a degree of ini-based configuration (with the slightly different semantics that implies, such as things like lists and dicts are hard to construct compared to Python code) then I think that should make it easier to apply Paste later.
Spurned by this discussion I've created a package CherryPaste, to try to make CherryPy applications compatible with Paste Deploy. I haven't really used it seriously, but my very tiny and incomplete test works okay; feedback is very much welcome. The idea isn't to magically support all CherryPy applications, but to support those that try. Specifically messing with cherrypy.root won't work, but you use the Paste Deploy configuration to let it set up cherrypy.root for you (it ends up being threadlocal, and each application is the "root" for its request).
Anyway, you can install it with:
easy_install CherryPaste==dev
Probably better to use "-e -b .", because this really isn't ready for any real use, I'm just looking for feedback.
TG 0.9 doesn't have to include this, but it would be good if TG worked okay with this, or had some way you could get to that would work well with this (something that does less magic).
