Limited traversal
Mike Orr
the benefits would outweigh the costs.
I'm converting a Pylons application that has two database-related
routes for read-only access:
/incidents/{id} => Incident ORM class
/entries/{id} => Entry ORM class
I can use URL dispatch in Pyramid, but then the views have to do the
work of converting the ID into an int, and aborting 404 if it's
invalid or the record doesn't exist, and check whether the user's
permissions are compatible with the record's properties. (I don't have
to check perms in this application because the database views hide
non-public records, but I would in another application so I'm thinking
ahead.) On the other hand, it's easy to generate a URL to any
incident: request.route_url("incident", id=incident_id)
Using traversal, the resource tree would take care of fetching the
database record and reporting if it doesn't exist, and it could also
tie in with ACL permissions. I've verified that SQLAlchemy ORM
instances don't have .__name__ or .__parent__ attributes so I could
set these for location awareness. I'm thinking of a resource structure
like this:
===
class Resource(dict):
def __init__(self, name, parent):
self.name = name
self.parent = parent
class Root(Resource):
"""The root resource."""
def __init__(self, request):
self.request = request
self["incident"] = RecordGetter(self, "incident", request,
model.Incident)
self["entry"] = RecordGetter(self, "entry", request, model.Entry)
root_factory = Root
class RecordGetter(dict):
"""Traversal component tied to a SQLAlchemy ORM class.
Calling .__getitem__ calls ``orm_class.get``, adds certain attributes
to the object, and returns it. (My classes define the .get method as a
wrapper for query get.)
"""
def __init__(self, name, parent, request, orm_class):
self.__name__ = name
self.__parent__ = parent
self.request = request
self.orm_class = orm_class
def __getitem__(self, key):
try:
key = int(key)
except ValueError:
raise KeyError(key)
obj = self.orm_class(key)
if obj is None:
raise KeyError(key)
obj.__name__ = key
obj.__parent__ = self
return obj
===
So first, is this the right way to to it? Second, how can I generate
URLs to resources without loading the objects through this interface?
The home page queries the 30 most recent incidents and displays links
to them. request.resource_url() requires a resource object, but the
home page or browse pages query the database directly and get plain
ORM objects rather than location-aware ones. I could make the URLs
manually but that sucks. I could somehow get the "incident" resource
and append the ID, or get the root resource and append "incident" and
the ID, but that's not much better.
Creating a traversal path ("/incident/123"), asking for the resource,
and then generating the URL from it (which would be the same
"/incident/123"), would be equally silly, and would also trigger a
redundant query.
And that raises another issue, the "incident" resource doesn't really
exist. I.e., "/incident" should return an error, not connect to a
regular view or default view. How do I do that? Or how do I make the
default view be "not found"?
--
Mike Orr <slugg...@gmail.com>
Andrey Popp
> I'm weighing whether to use traversal in a limited way, and whether
> the benefits would outweigh the costs.
>
> I'm converting a Pylons application that has two database-related
> routes for read-only access:
> /incidents/{id} => Incident ORM class
> /entries/{id} => Entry ORM class
>
> I can use URL dispatch in Pyramid, but then the views have to do the
> work of converting the ID into an int, and aborting 404 if it's
> invalid or the record doesn't exist, and check whether the user's
> permissions are compatible with the record's properties. (I don't have
> to check perms in this application because the database views hide
> non-public records, but I would in another application so I'm thinking
> ahead.) On the other hand, it's easy to generate a URL to any
> incident: request.route_url("incident", id=incident_id)
>
> Using traversal, the resource tree would take care of fetching the
> database record and reporting if it doesn't exist, and it could also
> tie in with ACL permissions. I've verified that SQLAlchemy ORM
> instances don't have .__name__ or .__parent__ attributes so I could
> set these for location awareness.
As a side note, you could also use pyramid_traversalwrapper[1] "which wraps each
traversed object in a location-aware proxy", so you don't need to manage
__parent__ and __name__ attributes by yourself, I've found it pretty useful.
> So first, is this the right way to to it?
I think it's fine.
> Second, how can I generate URLs to resources without loading the objects
> through this interface? The home page queries the 30 most recent incidents
> and displays links to them. request.resource_url() requires a resource
> object, but the home page or browse pages query the database directly and get
> plain ORM objects rather than location-aware ones. I could make the URLs
> manually but that sucks. I could somehow get the "incident" resource and
> append the ID, or get the root resource and append "incident" and the ID, but
> that's not much better.
That's interesting question. I'm thinking of the following construct:
1. Make your resource graph lazy, e.g. producing only proxies which only
knows its location (pk in database in case of SQLAlchemy) and how to load
itself (table and parent resources, which impose some query constraints or
JOIN clauses).
2. Reuse your resource graph in pyramid's traversal engine and in your home
page view:
def homepage():
entries_ids = db.entries_ids_for_homepage()
entries = [root["entries"][id] for id in entries_ids]
So you get ``entries`` list of location-aware Entry objects which are loaded
only if you access their fields, but not __name__ or __parent__ which allows
you to generate URL for them w/o any overhead.
> Creating a traversal path ("/incident/123"), asking for the resource,
> and then generating the URL from it (which would be the same
> "/incident/123"), would be equally silly, and would also trigger a
> redundant query.
Why not memoize results of traversing? This kind of caching seems pretty
natural to me.
> And that raises another issue, the "incident" resource doesn't really
> exist. I.e., "/incident" should return an error, not connect to a
> regular view or default view. How do I do that? Or how do I make the
> default view be "not found"?
I'm sure it should exist -- it's a container of incidents! Any reasons you
don't want to expose it? If yes -- just adapt it to HTTPForbidden or something
like this.
[1]: http://pypi.python.org/pypi/pyramid_traversalwrapper/0.1
Andrey Popp
Andrey Popp
Mike Orr
>> And that raises another issue, the "incident" resource doesn't really
>> exist. I.e., "/incident" should return an error, not connect to a
>> regular view or default view. How do I do that? Or how do I make the
>> default view be "not found"?
>
> I'm sure it should exist -- it's a container of incidents! Any reasons you
> don't want to expose it? If yes -- just adapt it to HTTPForbidden or something
> like this.
In a full REST application I would, but in this case the home page has
the most recent incidents (which is all most people care about), and
there are separate browse pages if somebody want to page through all
5000 incidents by date or name. That's also why the URL is
"/incident/1234" instead of "/incidents/1234", because there is no
(valid) "/incidents" page. I suppose that in itself may be an argument
against traversal.
--
Mike Orr <slugg...@gmail.com>
Chris McDonough
Coming to this discussion late but I'm not sure I understand the
problem. If there's no view registered for the type of object implied
by "/incident", it will 404. I don't think there's any subtlety here.
- C
Mike Orr
What about the default view? How do I know which of my views is the
default? Is it the first one registered?
--
Mike Orr <slugg...@gmail.com>
Chris McDonough
It's the one that names the object type represented "/incident" as its
"context" argument and which has no "name" argument. If there isn't one
of those, it's a 404.
- C
Mike Orr
Ah, so there's one default view per 'context' argument, rather than
just one global default view. That makes more sense now. I'm intending
to register all my views with a 'context' argument and without a name,
at least for the views I've needed so far. So in that sense, all of
them are the default views for their respective contexts.
--
Mike Orr <slugg...@gmail.com>
Daniel Holth
Mike Orr
I could do that, but if I have to define the routes anyway, it pushes
me toward just using URL dispatch.
--
Mike Orr <slugg...@gmail.com>
Andrey Popp
> On Thu, Feb 16, 2012 at 12:09:36PM -0800, Mike Orr wrote:
> > Second, how can I generate URLs to resources without loading the objects
> > through this interface? The home page queries the 30 most recent incidents
> > and displays links to them. request.resource_url() requires a resource
> > object, but the home page or browse pages query the database directly and get
> > plain ORM objects rather than location-aware ones. I could make the URLs
> > manually but that sucks. I could somehow get the "incident" resource and
> > append the ID, or get the root resource and append "incident" and the ID, but
> > that's not much better.
>
> That's interesting question. I'm thinking of the following construct:
>
> 1. Make your resource graph lazy, e.g. producing only proxies which only
> knows its location (pk in database in case of SQLAlchemy) and how to load
> itself (table and parent resources, which impose some query constraints or
> JOIN clauses).
>
> 2. Reuse your resource graph in pyramid's traversal engine and in your home
> page view:
>
> def homepage():
> entries_ids = db.entries_ids_for_homepage()
> entries = [root["entries"][id] for id in entries_ids]
>
> So you get ``entries`` list of location-aware Entry objects which are loaded
> only if you access their fields, but not __name__ or __parent__ which allows
> you to generate URL for them w/o any overhead.
If someone is still interested in this approach, I've implemented[1]
sqlalchemy.Query.lazy_get(ident) method which works that way. My use-case is
different -- to allow User object to be attached on request without touching
database (user id is encoded in authc cookie), but I think it should work for
constructing lazy resource graph as well.
Implementation based on SQLAlchemy mailing list post[2].
[1]: https://gist.github.com/77eb31670a1842f301c2
[2]: https://groups.google.com/forum/?fromgroups#!topic/sqlalchemy/DvofQn6uVI4
Michael Merickel
My use-case is
different -- to allow User object to be attached on request without touching
database (user id is encoded in authc cookie), but I think it should work for
constructing lazy resource graph as well.
Andrey Popp
Yeah, thank you, that would work too, but already has custom Query class with
othe goodies and also want to access user's id as request.user.id.
Fabio Dive
this simplify a lot the concept and the design I use with my applications with user data.
Do you think I can write a class to retrieve data from the beaker session if it is loaded instead of dbconn? In case of None I can proceed with dbconn..is this the best approach to minimize queries to the db and using the beaker session in an elegant and efficient way?
thank you
f.
--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To post to this group, send email to pylons-...@googlegroups.com.
To unsubscribe from this group, send email to pylons-discus...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
Michael Merickel
Yeah, thank you, that would work too, but already has custom Query class with
othe goodies and also want to access user's id as request.user.id.
Andrey Popp
> On Sun, Feb 19, 2012 at 3:20 AM, Andrey Popp wrote:
> > Yeah, thank you, that would work too, but already has custom Query class
> > with othe goodies and also want to access user's id as request.user.id.
>
> That's fine of course. request.user.id works with this pattern though, just
> to clarify, because the User object is returned as request.user.
Yeah, but it touches the database if dbconn['users'].query({'id':userid}) isn't
implemented as Query.lazy_get I've posted in gist, so both patterns are useful
and orthogonal for each other :-).
Mike Orr
There is one issue with this approach. If the record doesn't exist,
the tree shouldn't be retturning a resource at all, the caller should
raise KeyError. But it doesn't know if the record exists without
querying the database. So it's kind of a catch-22, unless you assume
the resource object means "syntactically valid ID" rather than "record
that definitely exists".
--
Mike Orr <slugg...@gmail.com>
Chris McDonough
> On Sat, Feb 18, 2012 at 10:44 PM, Michael Merickel <mmer...@gmail.com> wrote:
> > On Sat, Feb 18, 2012 at 3:53 PM, Andrey Popp <8ma...@gmail.com> wrote:
> >>
> >> My use-case is
> >> different -- to allow User object to be attached on request without
> >> touching
> >> database (user id is encoded in authc cookie), but I think it should work
> >> for
> >> constructing lazy resource graph as well.
> >
> >
> > In Pyramid 1.3 I've updated the cookbook to directly support this idea of a
> > "lazy get" via attaching lazily evaluated properties to the request object.
> > You may consider it as an alternative to a custom Query class.
> >
> > http://docs.pylonsproject.org/projects/pyramid_cookbook/en/latest/auth/user_object.html#making-a-user-object-available-as-a-request-attribute
>
> There is one issue with this approach. If the record doesn't exist,
> the tree shouldn't be retturning a resource at all, the caller should
> raise KeyError.
You're applying traversal terminology to this cookbook entry, but the
cookbook entry is unrelated to traversal. Can you clarify?
- C
Mike Orr
My original question in this thread was using traversal to fetch a
record from a certain table, and the tradeoffs of doing so vs using a
route. The suggestion of using a lazy evaluator was brought up in this
context, and the cookbook example was shown as an example of a lazy
evaluator.
--
Mike Orr <slugg...@gmail.com>