Overview of of the sprint...
Mark Ramm
1) CherryPy style Object Dispatch in a pylons friendly way.
2) the core decorator syntax from TurboGears on top of Pylons
Here are a few ideas which are floating around in my head after some
talks with Jonathan an Kevin in the last couple of days.
1) We can tell people to use the Pylons context object to get stuff to
templates that they don't want returned in the JSON dictionary that is
returned by the method.
2) We can implement an ultra simple template engile that returns a
python dictionary, usefull for programatica access to controler
methods, an therefore for testing!.
3) It would be great if we could implement Object Dispatch in a way
that made it easy to overide with a routes style config for a
particular path. (This would make REST style programming easier).
4) This style of web programing is awesome for the "new web server"
which returns XML, or JSON or binary Flex objects, or whatever as
often as it does rendered HTML.
Alberto Valverde
First of all: Count me in! :) I'm +100 on seeing what comes out of
this and willing to help out. Some comments:
On 17 jun, 23:27, Mark Ramm <mark.mchristen...@gmail.com> wrote:
> The basic plan for this sprint is to implement:
>
> 1) CherryPy style Object Dispatch in a pylons friendly way.
I see two ways of doing this:
a) Use CP3 itself. Since a Pylons controller is a full-fledged WSGI
app, a cherrypy._cptree.Application object can be dropped in place of
a controller and let it handle dispatching from there.
b) emulate CP's dispatching ourselves (a la Rhubarbtart but leveraging
Paste's API improvements which have ocured since RhubarbTart was
written). I've got some experimental code doing this here [1]
> 2) the core decorator syntax from TurboGears on top of Pylons
I'd like to suggest here to concentrate on the external API only and
simplify the implementation with the ideas that have been circulating
the trunk ML a couple of months ago (registering behavior and avoid
the complexity the current expose decorator has)
>
> Here are a few ideas which are floating around in my head after some
> talks with Jonathan an Kevin in the last couple of days.
>
> 1) We can tell people to use the Pylons context object to get stuff to
> templates that they don't want returned in the JSON dictionary that is
> returned by the method.
I'm not much of a fan of Pylons context object and all the
StackedObjectProxies on the whole. The simplicity and explicitness of
returning a dict is something I've been missing from TG on my Pylons
wanderabouts... If those things that shouldn't be jsonified are things
like helper functions, widgets, etc.. (not data) I think it's cleaner
to just import them from the template itself.
>
> 2) We can implement an ultra simple template engile that returns a
> python dictionary, usefull for programatica access to controler
> methods, an therefore for testing!.
Hmm. The way I envision this is that the dictionary returned by
methods gets processed in a lower layer in a way that controller
methods actually return a dictionary instead of processed output . In
other words: expose just registers behavior instead of mangling the
output with the template engine. That is:
1) request arrives
2) Routes route it to the controller (which is a WSGI app)
3) Controller's __call__ delegates to controller method
4) the output (a dict) form the controller method is processed
according to annotations set by expose
5) processed output is returned following the WSGI protocol
>
> 3) It would be great if we could implement Object Dispatch in a way
> that made it easy to overide with a routes style config for a
> particular path. (This would make REST style programming easier).
I'd go as far to suggest that we use Routes as a primary dispatcher.
Since, as I mentioned, Pylons controllers are WSGI apps we can swap
the standard Pylons controller with a CP dispatch style one on a per-
controller basis. This can be a CP Application object or one that
emulates it (without the tools, etc... if we're not going to need
them)
>
> 4) This style of web programing is awesome for the "new web server"
> which returns XML, or JSON or binary Flex objects, or whatever as
> often as it does rendered HTML.
Like good 'ol TG! :)
Alberto
Alberto Valverde
Forgot to include the link I mentioned, [1]. Here it is:
http://svn.toscat.net/TurboMiscStuff/tg2/
Alberto
Noah Gift
Jonathan LaCour
>> 2) the core decorator syntax from TurboGears on top of Pylons
>
> I'd like to suggest here to concentrate on the external API only and
> simplify the implementation with the ideas that have been circulating
> the trunk ML a couple of months ago (registering behavior and avoid
> the complexity the current expose decorator has)
I was thinking about this as well, and it seems like just setting
attributes on an exposed method, and handling it in WSGI middleware
makes the most sense.
>> 1) We can tell people to use the Pylons context object to get stuff
>> to templates that they don't want returned in the JSON dictionary
>> that is returned by the method.
>
> I'm not much of a fan of Pylons context object and all the
> StackedObjectProxies on the whole. The simplicity and explicitness of
> returning a dict is something I've been missing from TG on my Pylons
> wanderabouts... If those things that shouldn't be jsonified are things
> like helper functions, widgets, etc.. (not data) I think it's cleaner
> to just import them from the template itself.
I am +1 on this. I suppose we can't really get rid of the context
object, but I think we should at least alias it to something more
understandable, like "context" ... single-character globals irritate me.
>> 2) We can implement an ultra simple template engile that returns
>> a python dictionary, usefull for programatica access to controler
>> methods, an therefore for testing!.
>
> Hmm. The way I envision this is that the dictionary returned by
> methods gets processed in a lower layer in a way that controller
> methods actually return a dictionary instead of processed output . In
> other words: expose just registers behavior instead of mangling the
> output with the template engine.
Yes, if we handle the interpretation of settings made within an expose
decorator from inside WSGI middleware, then exposed methods will pretty
much always just return what they return when they are called, whether
it be a dict or a string or whatever. This is one of the benefits to me
of the approach, is that you don't have to create a "python engine" as
Mark suggested, or do anything else to work-around the expose decorator
when doing your testing.
It should be the same way for validation, IMO.
>> 3) It would be great if we could implement Object Dispatch in a
>> way that made it easy to overide with a routes style config for a
>> particular path. (This would make REST style programming easier).
>
> I'd go as far to suggest that we use Routes as a primary dispatcher.
> Since, as I mentioned, Pylons controllers are WSGI apps we can swap
> the standard Pylons controller with a CP dispatch style one on a per-
> controller basis. This can be a CP Application object or one that
> emulates it (without the tools, etc... if we're not going to need
> them)
I am not sure how this would work, but I personally like the immediacy
and simplicity of object dispatch, but often find myself having to
override the default method of my controllers when doing things in
a more RESTful way. Also, I have always fond Routes difficult to
understand, whereas object dispatch is very easy to understand.
I am not sure how I feel about this yet...
All in all, this should be a very fun experiment, and I think we could
end up in a nice place where we could benefit from a lot of the great
things going on in Pylons right now, without losing all that TurboGears
flavor we have all grown to love.
--
Jonathan LaCour
http://cleverdevil.org
Mark Ramm
> designing TG 2.0 in a way that all components are interchangeable as much as
> possible with Pylons, then advantage shifts to these two frameworks.
Exactly our goal. We want interchangeable components, and we want as
similar a standard set of components as we can achieve without
compromising on either the Pylons way or the TurboGears way.
Of course, our hope is that Django actually starts using WSGI
middleware rather than their own variety, at which point we could
begin sharing optional components with them.
It would also be great if things like the Django template engine were
seprately useable, and conformed to the buffet API. But I guess
that's why somebody forked off and created Jinja.
But at this point we can only work better with the people who are
actively willing to work better with us. So we want to make TG
widgets into something that plays well in Zope, and Pylons, and
otherwise make components that are really reusable -- across
frameworks.
> As smart as 10 people are....10,000 are smarter...
Honestly I think the Django guys have more than 10 people ;)
Alberto Valverde
On Jun 18, 2:57 pm, Jonathan LaCour <jonathan-li...@cleverdevil.org>
wrote:
> Alberto Valverde wrote:
(...)
>
> > I'd go as far to suggest that we use Routes as a primary dispatcher.
> > Since, as I mentioned, Pylons controllers are WSGI apps we can swap
> > the standard Pylons controller with a CP dispatch style one on a per-
> > controller basis. This can be a CP Application object or one that
> > emulates it (without the tools, etc... if we're not going to need
> > them)
>
> I am not sure how this would work, but I personally like the immediacy
> and simplicity of object dispatch, but often find myself having to
> override the default method of my controllers when doing things in
> a more RESTful way. Also, I have always fond Routes difficult to
> understand, whereas object dispatch is very easy to understand.
>
> I am not sure how I feel about this yet...
Just to make the point clearer. I'm not talking about replacing object
style dispatch with Routes but to stick routes above CP style
controllers with a default route pointing to the Root controller,
something like:
map.connect('/*url', controller="root")
This way, the RootController which this route dispatches to (or should
dispatch to) is a normal CP controller that further dispatches the
remaining PATH_INFO (possibly to other subcontrollers grafted as
attributes).
Overriding this with Routes-like dispatching for some paths should be
as easy as adding more specific routes above it.
The advantage I see with this approach is that it's almost free in the
sense that Routes already can handle this and lets both dispatching
styles cohabitate nicely and, most importantly in my POV, is that the
url generation facilities Routes has can be used (no need to reinvent
the wheel to handle reverse proxies etc...) therefore having a
consistent interface for all url-generation needs. Something like this
should be possible I believe:
url_for(controller="root", url="some/path/to/a/cp/sub/controller") --
> ${webroot}/some/path/to/a/cp/sub/controller
(wrapping this in a "url" function with a similar (if not identical)
interface to turbogears.url for the common case of object-style
dispatch only should be trivial I believe)
The only downside I can think of is the extra indirection Routes
imposes regardless Routes-like dispatching is used or not.
Alberto
Alberto Valverde
On Jun 18, 4:54 pm, "Mark Ramm" <mark.mchristen...@gmail.com> wrote:
> cti> Both approaches have their place, but if some thought could be put into
>
> > designing TG 2.0 in a way that all components are interchangeable as much as
> > possible with Pylons, then advantage shifts to these two frameworks.
>
> Exactly our goal. We want interchangeable components, and we want as
> similar a standard set of components as we can achieve without
> compromising on either the Pylons way or the TurboGears way.
Mark has already said it but just to create more traffic in the
recently born list I'll repeat what he said ;)
Pylons is already very modular and this won't change. What we're
aiming to do (as I understand it) is to build a nice and easy, TG-
flavored, API on top of Pylons. Components should be inherently
interchangeable since Pylons will still be there under TG
Alberto
Jonathan LaCour
> Just to make the point clearer. I'm not talking about replacing
> object style dispatch with Routes but to stick routes above CP style
> controllers with a default route pointing to the Root controller,
> something like:
>
> map.connect('/*url', controller="root")
>
> This way, the RootController which this route dispatches to (or should
> dispatch to) is a normal CP controller that further dispatches the
> remaining PATH_INFO (possibly to other subcontrollers grafted as
> attributes).
>
> Overriding this with Routes-like dispatching for some paths should be
> as easy as adding more specific routes above it.
Now I see your point, and it makes complete sense. Basically, all we
need to do in our sprint to make this happen is craft a RootController
base class that does the object-style dispatching based upon the
remaining PATH_INFO, which should be as easy as borrowing some code from
Paste, CherryPy 3, RhubarbTart, or the code that you sent out earlier.
I really like the idea that you can mix and match URL dispatch, but
that the core dispatching method is the same across regular Pylons and
Pygears projects.
Mark Ramm
> that the core dispatching method is the same across regular Pylons and
> Pygears projects.
Yea, that will be fantastic for when you want to do REST the official
way, or for when you have to map ann application to a legacy URL
structure. (Both of which are much easier in Routes) but will still
be super-easy for building new applications the cherrypy way.
And it seems like it should be easier to implement than the inversion
of the same thing which I thought of earlier (object dispatch as the
main dispatch with optional routes down the line).
--Mark Ramm
Alberto Valverde
On Jun 18, 2007, at 7:45 PM, Jonathan LaCour wrote:
>
> Alberto Valverde wrote:
>
>> Just to make the point clearer. I'm not talking about replacing
>> object style dispatch with Routes but to stick routes above CP style
>> controllers with a default route pointing to the Root controller,
>> something like:
>>
>> map.connect('/*url', controller="root")
>>
>> This way, the RootController which this route dispatches to (or
>> should
>> dispatch to) is a normal CP controller that further dispatches the
>> remaining PATH_INFO (possibly to other subcontrollers grafted as
>> attributes).
>>
>> Overriding this with Routes-like dispatching for some paths should be
>> as easy as adding more specific routes above it.
>
> Now I see your point, and it makes complete sense. Basically, all we
> need to do in our sprint to make this happen is craft a RootController
> base class that does the object-style dispatching based upon the
> remaining PATH_INFO, which should be as easy as borrowing some code
> from
> Paste, CherryPy 3, RhubarbTart, or the code that you sent out earlier.
Exactly. Unfortunately, for our egos, I believe this is going to be
more of a copy&paste mission than any serious design and engineering
effort ;)
Some sources of inspiration for the controller/wsgi part and general
brainstorming:
A Pylon's WSGI app. This does the context set-up (populating the
StackedOPs). Maybe we need to subclass to do some TG specific context
set up... dunno
Pylon's controller. We'll probably need to write a new one to do the
CP style dispatch while preserving the (Pylons') interface so a
project can mix and match both and have both styles of dispatch.
There's a quick-and-lousy implementation at the code I sent, (more
precisely here [1]). RhubarbTart does something quite similar but, as
far as I could tell, duplicates some functionality already in Paste
(the response/request wrappers... I guess because it wasn't at Paste
at the time it was written or RT was trying to be more "cherrypyish",
really guessing though...)
Now talking about expose:
I'd rather favor expose being something as simple as:
def expose(**expose_args):
def wrapper(func, *args, **kw):
# Register behavior (only once)...
func.expose_args = expose_args
# ...nothing more, nothing less
return func(*args, **kw)
# leverage Pylons dep of Schimionato's Decorator module
# to preserve signature, etc...
# I'd rather avoid the [] syntax since... well, the impl. is
# complex, magical and...
# who's using 2.3 nowadays?? ;)
return decorator(wrapper)
Then the CP clone controller could do something like this non-
executable pseudocode:
class TGBaseController(...):
def __call__(self, env, sr):
# peek into routing args and see if dest. method
# belongs in this controller
if not_a_subcontroller:
meth = self.get_exposed_meth(env)
args, kw = get_routing_args(meth, env)
output = meth(*args, **kw)
p_output = process_output(meth, output)
return build_response(p_output, env, sr)
else:
return self.delegate_to_subcontroller(env, sr)
The key points are the "get_routing_args" part which does the request
params marshalling into function args and the "process_output" which
applies templating, etc.. based on annotations expose left on the
method (now I just wrote this I see that it should build the response
too since Content-Type might vary... anyway, just a sketch... ;) This
is more or less what I did on [1]
Alberto
Jonathan LaCour
> Exactly. Unfortunately, for our egos, I believe this is going to be
> more of a copy&paste mission than any serious design and engineering
> effort ;)
That just means it should be easier than we thought, which is good news
as far as I am concerned :)
> Now talking about expose:
>
> I'd rather favor expose being something as simple as:
>
> def expose(**expose_args):
> def wrapper(func, *args, **kw):
> # Register behavior (only once)...
> func.expose_args = expose_args
> # ...nothing more, nothing less
> return func(*args, **kw)
> # leverage Pylons dep of Schimionato's Decorator module
> # to preserve signature, etc...
> # I'd rather avoid the [] syntax since... well, the impl. is
> # complex, magical and...
> # who's using 2.3 nowadays?? ;)
> return decorator(wrapper)
Yes, this is exactly what I had envisioned as well. Just a bunch of
attributes on an exposed function which "register" behavior that can be
applied somewhere else.
> Then the CP clone controller could do something like this non-
> executable pseudocode: [snip, snip]
I actually don't like this happening inside the object-dispatch root
controller. This means that cannot use the expose decorator in regular
controllers that don't use object-dispatch. I was thinking that this
would be better suited for WSGI middleware, but I am not sure if that is
possible or not.
If we use WSGI middleware, then you could use the @expose decorator in
regular Pylons controllers too, giving you the ability to mix and match
styles to your heart's content.
Kevin Dangoor
It's worth noting that if this stuff is being done in call like this,
expose does not need to be a true decorator. It could just add its
attributes to the function and return the original function.
Not having used Pylons, I can't comment on Jonathan's notion of using
@expose in a Pylons controller. If such a thing makes sense, that
sounds really cool :)
Kevin
Alberto Valverde
> regular (Pylons'*)
> controllers that don't use object-dispatch.
(* quote invasion by me to add context)
Good point. However, I don't see how expose could be used in Pylons'
controllers unless expose did a little bit more than just set those
attributes.
Pylon's controllers can return WSGI apps (though it's a little bit
hacky since it seems the code was only expecing Response objects,
which are WSGI apps too) so maybe "expose" could wrap what the method
returns in some sort of Response/WSGI-app object (with a reference to
the original output from the method for testing purposes) and a some
middleware doing the transformation afterwards.
However, with this approach, testing become a little more difficult
since to get the original output you'll have to dig into the response
object expose returns and expose becomes a little less transparent.
Another thing I see is that, unless the middleware is applied to the
WSGI app returned by the "exposed" function, no context (db
connections, config, view engines, etc...) will be available to the
"output processing" middleware (since it'll have to be above
PylonsWSGIApp)
Perhaps it's possible or easier to make a Pylons' Controller subclass
with the machinery needed to handle those "expose" annotations for
those cases a more "Pylonic" controller is wanted?
Alberto
Alberto Valverde
Indeed!
(happens that I'm using Deocrator much lately... I'm beginning to see
nails everywhere!... :)
Alberto
Jonathan LaCour
> (* quote invasion by me to add context)
>
> Good point. However, I don't see how expose could be used in Pylons'
> controllers unless expose did a little bit more than just set those
> attributes.
Off the top of my head, we could basically do this for the expose
decorator:
import pylons
def expose(template=None, format=None, transact=True):
def decorate(f):
def newfunc(*args, **kwargs):
# register behavior in the WSGI environment
pylons.request.environ['pygears.template'] = template
pylons.request.environ['pygears.format'] = format
pylons.request.environ['pygears.transact'] = transact
# call original controller function
return f(*args, **kwargs)
return newfunc
return decorate
... and then the middleware could just read the appropriate keys from
the WSGI environment to decide what to do. So, yes, its doing a bit
more than setting attributes on a function: its setting the attributes
in the WSGI environment.
> Pylon's controllers can return WSGI apps (though it's a little bit
> hacky since it seems the code was only expecing Response objects,
> which are WSGI apps too) so maybe "expose" could wrap what the method
> returns in some sort of Response/WSGI-app object (with a reference to
> the original output from the method for testing purposes) and a some
> middleware doing the transformation afterwards.
>
> However, with this approach, testing become a little more difficult
> since to get the original output you'll have to dig into the response
> object expose returns and expose becomes a little less transparent.
>
> Another thing I see is that, unless the middleware is applied to
> the WSGI app returned by the "exposed" function, no context (db
> connections, config, view engines, etc...) will be available to
> the "output processing" middleware (since it'll have to be above
> PylonsWSGIApp)
I haven't dug enough into Pylons to know about the above, but I'll take
a look and see if I can understand this later this week :)
> Perhaps it's possible or easier to make a Pylons' Controller subclass
> with the machinery needed to handle those "expose" annotations for
> those cases a more "Pylonic" controller is wanted?
Thats certainly another option. I'd really like to make it possible to
use the expose decorator in regular Pylons controllers, but I don't know
enough about the internals of Pylons (yet) to say whether or not this is
possible.
Alberto Valverde
On Jun 18, 2007, at 10:20 PM, Jonathan LaCour wrote:
>
> Alberto Valverde wrote:
>
>> (* quote invasion by me to add context)
>>
>> Good point. However, I don't see how expose could be used in Pylons'
>> controllers unless expose did a little bit more than just set those
>> attributes.
>
> Off the top of my head, we could basically do this for the expose
> decorator:
>
> import pylons
>
> def expose(template=None, format=None, transact=True):
> def decorate(f):
> def newfunc(*args, **kwargs):
> # register behavior in the WSGI environment
> pylons.request.environ['pygears.template'] = template
> pylons.request.environ['pygears.format'] = format
> pylons.request.environ['pygears.transact'] = transact
>
> # call original controller function
> return f(*args, **kwargs)
> return newfunc
> return decorate
>
> ... and then the middleware could just read the appropriate keys from
> the WSGI environment to decide what to do. So, yes, its doing a bit
> more than setting attributes on a function: its setting the attributes
> in the WSGI environment.
This could be another option. However, I don't see clearly where the
middleware that processes the output could be stacked since it'll
need to be below the Pylons app so it has access to the app's
context. IMO, it would be easier for expose to return a WSGI app
(with the original output at an attribute for easier testing).
Anyway, the devil is in the details... the sprint will tell better :)
I agree that being able to use expose in pylons regulart controllers
will be a great plus.
Alberto
Mark Ramm
> will be a great plus.
Ben suggested subclassing the pylons controller might be our best bet.
And for the sprint this weekend, I'm generally in favor of taking
the simpler path and getting something done quickly.
--Mark
Jonathan LaCour
+1
I looked into making my way work, and it would make the expose decorator
a bit more complex than I would like. I think we should go with the
simpler route that Alberto suggested.
Mike Schinkel
http://blog.welldesignedurls.org/2007/02/05/urls-for-multilingual-web-sites/
http://h3h.net/2007/01/designing-urls-for-multilingual-web-sites/
--
-Mike Schinkel
http://www.mikeschinkel.com/blogs/
http://www.welldesignedurls.org
http://atlanta-web.org - http://t.oolicio.us