Mapping/Handling paths with variables in the middle _without_ writing custom dispatcher logic? (/foo/bar/123/baz/bam)
Thomas Wittek
I'm using TurboGears for my web-app, which uses CP 2.2.1 to map and
handle the requests.
Now I wonder, what is the best way to map URLs like `/user/123/message/
send`, `/some/object/123/with/some/numeours/subcontrollers/and/
actions`.
This question probably occured several time on this list, but I didn't
find any answer (using my search terms).
I know that I can use positional parameters and write my own
dispatcher logic:
@expose()
def default(self, user_id, *args, **kwargs):
"""Custom dispatcher for URLs like /user/<id>[/<action>]"""
if not args:
return self.show(user_id)
elif args[0] == "edit":
return self.edit(user_id)
elif ....
But the purpose of @expose is to free me from writing my own
dispatchers.
Especially when there are many pages/methods for an object, you really
don't want to write a huge dispatcher yourself.
CP could easily handle e.g. `/user/123/message/send` when it's
formulated as `/user/message/send?user_id=123`, but specifying the
resource (user 123) as a query parameter is ugly (and not very
RESTful).
So could it be (easily) possible to translate the beautiful URL `/user/
123/message/send` to the CP friendly URL `/user/message/send?
user_id=123`?
Preferably without using Routes, as I find @expose() very convenient
instead of the above mentioned case.
What would be ideal to me, would be some kind of a parametrized
controller:
class User(Controller):
@expose()
def show(self, **kwargs):
# will have kwargs['user_id'] set
class Root(RootController):
user = User(params=['user_id']) # will take a parameter and
put it in kwargs['user_id'] for every sub-controller
So `/user/123/message/send` will call
root.user.message.send(user_id=123, <other args>).
Does something like this exist.
Would it be hard to implement? Unfortunately I feel that I cannot
implement it. I'm using TurboGears for 3 weeks now and didn't do much
more in Python besides my little web-app.
Thank you for any advice!
Pete H
Well, the easy way is to re-order your URI so that the '123' comes
after the generic resource elements of the URI, eg
instead of `/user/123/message/send` use `/user/message/send/123`
That way 123 can be picked up from *args. But of course you would have
to redesign your URI scheme which may not be possible.
As to re-mapping kwargs to args or vice-versa it would be interesting
to know if someone has a generic solution - a new tool perhaps?
As an aside, shouldn't the 'send' element properly be the PUT or POST
HTTP method in a RESTful design?
Thomas Wittek
> Well, the easy way is to re-order your URI so that the '123' comes
> after the generic resource elements of the URI, eg
> instead of `/user/123/message/send` use `/user/message/send/123`
> That way 123 can be picked up from *args. But of course you would have
> to redesign your URI scheme which may not be possible.
It is possible, as it would be possible to put the ID in a query
param.
But it's more like a workaround than a proper solution.
>From `/user/message/send/123` I cannot see directly that 123 is the ID
of the user, where I can see it clearly in `/user/123/message/send`.
The latter URL layout is just more desirable.
> As to re-mapping kwargs to args or vice-versa it would be interesting
> to know if someone has a generic solution - a new tool perhaps?
Yes, you could fake it using Apache/mod_rewrite -- but it still is a
fake and adds an external "dependency".
> As an aside, shouldn't the 'send' element properly be the PUT or POST
> HTTP method in a RESTful design?
Yes, you're right. Currently I don't use the "verbs" properly. I
didn't investigate it further, but are PUT/DELETE fully supported by
all browser? Currently I stick to GET/POST.
Pete H
On Oct 28, 12:59 pm, Thomas Wittek <streawkc...@googlemail.com> wrote:
> On Oct 28, 1:43 pm, Pete H <pe...@ssbg.zetnet.co.uk> wrote:
>
> > Well, the easy way is to re-order your URI so that the '123' comes
> > after the generic resource elements of the URI, eg
> > instead of `/user/123/message/send` use `/user/message/send/123`
> > That way 123 can be picked up from *args. But of course you would have
> > to redesign your URI scheme which may not be possible.
>
> It is possible, as it would be possible to put the ID in a query
> param.
> But it's more like a workaround than a proper solution.>From `/user/message/send/123` I cannot see directly that 123 is the ID
>
> of the user, where I can see it clearly in `/user/123/message/send`.
> The latter URL layout is just more desirable.
Well the URI is by definition positional so it can only be parsed by
making assumptions. You just have a different set of assumptions,
although
I agree that in what I guess to be your resource set having the user
id directly after the element 'user' makes more sense. But to do it
your way would reqire the mapper to have a lot more logic in it, tied
to your particular URI scheme, than the tree mapper that comes with
CherryPy. Maybe a regex parse like Routes would be able to handle it?
Maybe your resource URI really stops at /user/, with everything else
being parameters?
I've not had to go into regex mapping since the variables in my URIs
fall naturally after the fixed resource path - /sheep/
data/'sheep_specifier' for example with a GET returns sheep data
(pedigree and so on) or a list of sheep depending on
'sheep_specifier', with POST adds a new instance of sheep, with PUT
updates an existing sheep, and in my case DELETE is not allowed on
this resource. Easy to do with CherryPy. And only GET allows
unauthenticated access, again easy to do with CherryPy.
>
> > As to re-mapping kwargs to args or vice-versa it would be interesting
> > to know if someone has a generic solution - a new tool perhaps?
>
> Yes, you could fake it using Apache/mod_rewrite -- but it still is a
> fake and adds an external "dependency".
Well, my page handlers just call a method to do the mapping, which has
no external dependencies. It turns the kwargs into positional args
based on a list passed to the function.
> > As an aside, shouldn't the 'send' element properly be the PUT or POST
> > HTTP method in a RESTful design?
>
> Yes, you're right. Currently I don't use the "verbs" properly. I
> didn't investigate it further, but are PUT/DELETE fully supported by
> all browser? Currently I stick to GET/POST.
HTML only allows POST and GET methods from forms, so if your client is
a web browser you have to tunnel PUT and DELETE through POST using
hidden fields on the HTML form, or else use the javascript
XMLHttpRequest() object. Neither are very nice, but neither is having
the verb in the URI.
Its all a bit pedantic really, but the point of REST is to use HTTP as
it was designed and to impose a little discipline.
Thomas Wittek
> [..] to do it
> your way would reqire the mapper to have a lot more logic in it, tied
> to your particular URI scheme, than the tree mapper that comes with
> CherryPy.
Do you really think so?
I'm not into the internals of CP. But at least from a user perspective
it could be relatively easy:
Basically, it's a default method that puts one (or several) positional
parameters into `kwargs` and then "somehow" uses CP dispatching on the
rest of the URI:
@expose()
def default(self, user_id, *args, **kwargs):
"""Custom dispatcher for URLs like /user/<id>[/<rest handled
by other controller>]"""
kwargs['user_id'] = user_id
other_controller.dispatch(*args, **kwargs)
Where the last line is the tricky part, where I don't know how it
could be done.
As stated in my first post, this default method could also be
automated by a definition of parametrized controllers:
user = User(params=['user_id'])
Still, I don't know how hard it would be to patch CP to do something
like that (where the parametrized controller could be consideres as
syntactical sugar that's not really needed).
> Maybe a regex parse like Routes would be able to handle it?
Certainly, but I really like the idea of defining my URI scheme with
classes/methods.
> Maybe your resource URI really stops at /user/, with everything else
> being parameters?
My users example is a bit simplified.
A user could have several sub-resources that in turn have their own
controllers.
E.g. images, tags, files, messages, bookmarks etc.
But you are right. You could list the sub-resources in the `user`
controller:
GET /user/123/images
Where the links point to another controller that handles the image
logic:
GET /image/42
PUT /image/42
If each image (or bookmark, file, ...) has a unique ID (that is
independent of the user ID, you could do it like that (and maybe even
should do it like that).
But you still have to write a custom dispatcher in user.default() that
handles 123/images, 123/messages, 123/etc, what I find quite
inconvenient.
At least automating this task would easy my pain a lot. ;)
Paweł Stradomski
That could be done quite simply, but requires some changes to the dispatcher -
so you should probably subclass the default one and create your own. My idea
is to change this part in _cpdispatch.py:
nodeconf = {}
node = getattr(node, objname, None)
if node is not None:
# Get _cp_config attached to this node.
if hasattr(node, "_cp_config"):
nodeconf.update(node._cp_config)
To something like this:
nodeconf = {}
if hasattr(node, '_my_magic_method'):
node = node._my_magic_method(objname)
node = getattr(node, objname, None)
if node is not None:
# Get _cp_config attached to this node.
if hasattr(node, "_cp_config"):
nodeconf.update(node._cp_config)
And now you could define _my_magic_method on your User controller so that it
accepts user id and returns new object that already knows the user id:
class UserController:
def _my_magic_method(id):
user = find_the_user(id)
if user is None:
return None
else:
return BoundUserController(user)
class BoundUserController:
def __init__(self, user):
self.user = user
@expose
def images():
pass
@expose
def messages():
pass
Make sure you do not expose _my_magic_method.
--
Paweł Stradomski
Thomas Wittek
> > A user could have several sub-resources that in turn have their own
> > controllers.
> > E.g. images, tags, files, messages, bookmarks etc.
>
> That could be done quite simply, but requires some changes to the dispatcher -
> so you should probably subclass the default one and create your own. My idea
>
> nodeconf = {}
> if hasattr(node, '_my_magic_method'):
> node = node._my_magic_method(objname)
> node = getattr(node, objname, None)
Doesn't `node` get overwritten here? Should this line be in an `else`
branch?
> [..]
> # Get _cp_config attached to this node.
> if hasattr(node, "_cp_config"):
> nodeconf.update(node._cp_config)
>
> And now you could define _my_magic_method on your User controller so that it
> accepts user id and returns new object that already knows the user id:
>
> class UserController:
> def _my_magic_method(id):
> [..]
> return BoundUserController(user)
So I have to create a new controller instance at each request?
Wouldn't it be better to create them once at startup?
I still like the idea more, to have some kind of a magic controller
that translates a positional arg into a kwarg, as it is is easier to
undestand, I think.
So that a call to /user/123/images would be identical to /user/images?
user_id=123.
That could then be easily handled by a plain old controller structure
(I can only talk from a user perspective, I have no clue about the CP
internals):
class Images:
@expose()
def POST(self, **kwargs):
user_id = kwargs['user_id']
#...
class User:
images = Images()
@expose()
def default(self, user_id, *args, **kwargs):
kwargs['user_id'] = user_id
self.dispatch(*args, **kwargs)
# OR
self.dispatch(translate_args=['user_id'], *args, **kwargs)
# OR even:
@expose(translate_args=['user_id'])
def default(self, user_id, *args, **kwargs):
pass
# OR even:
@translate_args('user_id')
class User:
images = Images()
The arg translation will be called whenever a default method would be
called, so only when no direct match for an existing method/attribute
exists.
I still don't know the best way to declare that behaviour in the
controller. Probably, there is a better (more pythonic) way than my
proposals above. I'm also not quite happy with the name "translation".
I'm still very new to Python (and CP).
You could do something similar by translating positional args to other
positional args:
/user/123/images => /user/images/123.
Cheers
Robert Brewer
> I still like the idea more, to have some kind of a magic controller
> that translates a positional arg into a kwarg, as it is is easier to
> undestand, I think.
> So that a call to /user/123/images would be identical to /user/images?
> user_id=123.
class Controller:
popargs = ['user_id']
def images(user_id):
return foo
class Dispatcher(cherrypy.Dispatcher):
...
node = root
names = [x for x in path.strip('/').split('/') if x] + ['index']
for i in range(len(names)):
for key in getattr(node, 'popargs', []):
objname = names[i].replace('.', '_')
cherrypy.request.params[key] = objname
i += 1
objname = names[i].replace('.', '_')
Robert Brewer
fuma...@aminus.org
Thomas Wittek
> Translating a positional arg into a kwarg is a fine goal. Just do it in a dispatcher. The controller can declare that it is to be traversed in a special way, but it shouldn't be traversing itself. Use a Dispatcher for that; something like:
>
>
> class Dispatcher(cherrypy.Dispatcher):
It seems like cherrypy.Dispatcher is only available in CP 3.0+.
I'm using the TurboGears framework which still uses CP 2.2 -- no
cherrypy.Dispatcher here.
How could I do it in CP 2.2?
Thanks!
Robert Brewer
Subclass _cphttptools.Request and override mapPathToObject would be how
I would do it. Not sure how to plug that into TG though.
Robert Brewer
fuma...@aminus.org
Thomas Wittek
> Subclass _cphttptools.Request and override mapPathToObject would be how
> I would do it. Not sure how to plug that into TG though.
I created a base class `ParametrizedController` that has a `default`
method that translates any specified parameters into `kwargs` and then
"re-dispatches" to a modified URL.
Below is my code with a simple example in the docstring.
This solution is probably quite hacky (and maybe buggy as I don't know
much about CP), but it seems to work.
Also, note that you maybe don't want URLs like `/user/123/images/321`
but instead a URL `/user/123/images` that contains links to URLs like
`/images/312`.
Still, with the `ParametrizedController` you can easily handle URLs
like `/user/123/images`, `/user/123/message`, `/user/123/tasks`, `/
user/123/XYZ`.
import cherrypy
class ParametrizedController(object):
"""
Offers a default method that will translate positional arguments
into
keyword arguments and then "re-dispatches" the request.
You can define the names for the arguments that should be
translated at
instantiation.
Example:
class Images(ParametrizedController):
def __init__(self):
super(Images, self).__init__()
self.params=['image_id']
def index(self, user_id, image_id=None):
if image_id:
return "image %s for user %s" % (image_id, user_id)
else:
return "images for user " + user_id
index.exposed = True
class Users(ParametrizedController):
images = Images()
def __init__(self):
super(Users, self).__init__()
self.params=['user_id']
def edit(self, user_id, **kwargs):
return "edit user " + user_id
edit.exposed = True
def index(self, user_id=None):
if user_id:
return "display user " + user_id
else:
return "list of all users"
index.exposed = True
class Root(object):
users = Users()
The URL `/users/123/edit` will call Users.edit(self,
user_id='123').
The URL `/users/123` will call Users.index(self, user_id='123').
The URL `/users/123/images` will call Images.index(self,
user_id='123').
The URL `/users/123/images/321` will call Images.index(self,
user_id='123', image_id='321').
"""
def __init__(self, params=[]):
self.params = params
def default(self, *args, **kwargs):
# Check if any parameters are defined and put them into
cp.req.params
args = list(args)
for k in self.params:
cherrypy.request.params[k] = args.pop(0)
# Generate new URL without the translated arguments
new_path = '/'.join(cherrypy.request.object_path.split('/')
[:-1] + args + [''])
# "Re-dispatch" using the new URL
cherrypy.request.main(new_path)
return cherrypy.response.body
default.exposed = True
Michele Cella
Sorry if I'm coming so late into this discussion, I've seen you've
found a solution to your problem anyway you may want to take a look at
TGNewTraversal that does exactly what you've been looking for (taking
inspiration from Nevow), you can find it right there:
https://projects.isotoma.com/tgnewtraversal
Ciao
Michele
Sylvain Hellegouarch
available in cheeseshop. That may be a simple way to achieve what you
want to do.
- Sylvain
Thomas Wittek
> def default(self, *args, **kwargs):
> # Check if any parameters are defined and put them into
> cp.req.params
It's a good idea to check, if we already translated the arguments.
Updated method:
def default(self, *args, **kwargs):
# Don't translate parameters, if already done before
if all([k in cherrypy.request.params.keys() for k in
self.params]):
raise cherrypy.NotFound
Thomas Wittek
> I would also suggest trying the selector for CherryPy dispatcher
> available in cheeseshop. That may be a simple way to achieve what you
> want to do.
then I cannot use it, as it's for CP 3 only.
I use TurboGears, which in turn uses CP 2.2.1
Thanks for the hint, anyway!
May be I can use it later.
Sylvain Hellegouarch
Ah okay. My bad. Well maybe the link Michele suggested would be a good
option.
- Sylvain
Thomas Wittek
> https://projects.isotoma.com/tgnewtraversal
Is it right that I have to instantiate each and every controller
dynamically at every request?
At least the examples seem to so so.
Isn't this quite some overhead?
Generally, the controller structure stays the same over all requests,
so instantiating the controllers once at compile time seems more
logical to me.
What I quite like is the abstraction of the resources:
/document/[id]/edit
-> document = Collection of document resources. Methods: index, create
-> [id] = One document resource. Methods: read, edit, delete