proper use of traversal
114 views
Skip to first unread message
anurag sharma
Jan 30, 2013, 2:27:31 AM1/30/13
to pylons-...@googlegroups.com
Hi Friends,
I'm using traversal/zodb for my pyramid application. However I'm a bit confused whether I'm using traversal/resource/tree structure properly or not.
The thing which is confusing me the most is the parent concept in tree, where we have to define parents for our resources while creating them.
For explanation I'm trying to present a small problem and will like to know what should be the correct approach for this kind of situation:
Suppose we have a college class management system. There are two classes class A and class B, a student 'XYZ' can be a part of one class or both the classes.
How I would design it is like this : We have root as college and under root we have a container called classes and a container called students.
So when student XYZ is created in system it is added in a container called students, and when he was admitted for class A, he was also added to a container(for students in that class) inside class A resource. Both the objects are the same (students/XYZ is classes/classA/itsStudents/XYZ )
In this case I can't have a parent in student instance when it is created, so I will use pyramid_traversalwrapper which will allow to assign parent dynamically based on the request. If url is students/XYZ , locationproxy will have parent as students and if .../classA/itsstudents/xyz then it will be according to it.
My question here is that whether my approach and application/usage of traversal is correct here or will it be done differently from experienced pyramid traversal developers. My doubt stems from very little discussion of traversal wrapper in docs and IRC/forums and this problem seemed to me very common scenario.
Metaphor of traversal as file system will also tell that a file can be in two folders ( and I will believe the files will be 'same' not 'similar' - meaning the same object ).
Please clear my doubts.
Thanks,
Anurag
I'm using traversal/zodb for my pyramid application. However I'm a bit confused whether I'm using traversal/resource/tree structure properly or not.
The thing which is confusing me the most is the parent concept in tree, where we have to define parents for our resources while creating them.
For explanation I'm trying to present a small problem and will like to know what should be the correct approach for this kind of situation:
Suppose we have a college class management system. There are two classes class A and class B, a student 'XYZ' can be a part of one class or both the classes.
How I would design it is like this : We have root as college and under root we have a container called classes and a container called students.
So when student XYZ is created in system it is added in a container called students, and when he was admitted for class A, he was also added to a container(for students in that class) inside class A resource. Both the objects are the same (students/XYZ is classes/classA/itsStudents/XYZ )
In this case I can't have a parent in student instance when it is created, so I will use pyramid_traversalwrapper which will allow to assign parent dynamically based on the request. If url is students/XYZ , locationproxy will have parent as students and if .../classA/itsstudents/xyz then it will be according to it.
My question here is that whether my approach and application/usage of traversal is correct here or will it be done differently from experienced pyramid traversal developers. My doubt stems from very little discussion of traversal wrapper in docs and IRC/forums and this problem seemed to me very common scenario.
Metaphor of traversal as file system will also tell that a file can be in two folders ( and I will believe the files will be 'same' not 'similar' - meaning the same object ).
Please clear my doubts.
Thanks,
Anurag
Whit Morriss
Jan 30, 2013, 1:04:09 PM1/30/13
to <pylons-discuss@googlegroups.com>
howdy anurag,
On Jan 30, 2013, at 1:27 AM, anurag sharma <anurags...@gmail.com>
wrote:
> Hi Friends,
>
> I'm using traversal/zodb for my pyramid application. However I'm a bit confused whether I'm using traversal/resource/tree structure properly or not.
> The thing which is confusing me the most is the parent concept in tree, where we have to define parents for our resources while creating them.
>
> For explanation I'm trying to present a small problem and will like to know what should be the correct approach for this kind of situation:
> Suppose we have a college class management system. There are two classes class A and class B, a student 'XYZ' can be a part of one class or both the classes.
> How I would design it is like this : We have root as college and under root we have a container called classes and a container called students.
> So when student XYZ is created in system it is added in a container called students, and when he was admitted for class A, he was also added to a container(for students in that class) inside class A resource. Both the objects are the same (students/XYZ is classes/classA/itsStudents/XYZ )
[I'll give this a shot, with the proviso I may just make you more confused]
I would posit traversal is a system for modeling how clients will interact with your application, not necessarily a way to model how your application should interact with it's data (though they are related). The close coupling of the zodb persistence to the traversal mechanism blurs this separation a bit vs. say using normalized tables in SQL where you would start with these concepts uncoupled.
The critical difference lies between the concept of a context and a data "object" like an ORM would hand you. It is a bridge between your external representation (how/what a client can get out of your app) and your internal representation (how your app gets things done). Using traversal with the ZODB means that your internal representation is going to be pretty closely tied to your external representation, but not completely.
Considering your example, I think you actually have 2 different types of data for students. #1 /students/XYZ would be a representation of the student for the system (name, age, SSN, credits completed, etc), and #2 /classes/classA/XYZ would be representation of the student in that class (grades, teacher's notes, etc).
In this case, I think you want an object hierarchy with a containerish objects (maps, lists) for Students and Classes that holds Students and StudentClassInfo.
To do sensible things you would most likely want to maintain some sort of relationship between Students and StudentClassInfo. This could either use something as simple as ids (see example below) or perhaps by maintaining a simple index composed out of zodb BTrees depending on what kinds of lookups you need.
How would this work? say you had a view 'about' for a student in the class.
http://myapp/classes/BasketWeaving/Bob/about
It displays some basic information
- Name: Bob Johnson
- Current Grade: F
- Notes: Bob lit last basket on fire.
Traversal would return the "about" view with the context StudentClassInfo (sci). In this case, sci.__name__ == 'Bob', so the view could simply walk back up the object tree to 'root' then access the student instance ala 'root.students['Bob']' to retrieve data such as the full name. The grade and notes would accessible via the sci instance.
-w
d. "whit" morriss
Platform Codemonkey
wh...@surveymonkey.com
On Jan 30, 2013, at 1:27 AM, anurag sharma <anurags...@gmail.com>
wrote:
> Hi Friends,
>
> I'm using traversal/zodb for my pyramid application. However I'm a bit confused whether I'm using traversal/resource/tree structure properly or not.
> The thing which is confusing me the most is the parent concept in tree, where we have to define parents for our resources while creating them.
>
> For explanation I'm trying to present a small problem and will like to know what should be the correct approach for this kind of situation:
> Suppose we have a college class management system. There are two classes class A and class B, a student 'XYZ' can be a part of one class or both the classes.
> How I would design it is like this : We have root as college and under root we have a container called classes and a container called students.
> So when student XYZ is created in system it is added in a container called students, and when he was admitted for class A, he was also added to a container(for students in that class) inside class A resource. Both the objects are the same (students/XYZ is classes/classA/itsStudents/XYZ )
I would posit traversal is a system for modeling how clients will interact with your application, not necessarily a way to model how your application should interact with it's data (though they are related). The close coupling of the zodb persistence to the traversal mechanism blurs this separation a bit vs. say using normalized tables in SQL where you would start with these concepts uncoupled.
The critical difference lies between the concept of a context and a data "object" like an ORM would hand you. It is a bridge between your external representation (how/what a client can get out of your app) and your internal representation (how your app gets things done). Using traversal with the ZODB means that your internal representation is going to be pretty closely tied to your external representation, but not completely.
Considering your example, I think you actually have 2 different types of data for students. #1 /students/XYZ would be a representation of the student for the system (name, age, SSN, credits completed, etc), and #2 /classes/classA/XYZ would be representation of the student in that class (grades, teacher's notes, etc).
In this case, I think you want an object hierarchy with a containerish objects (maps, lists) for Students and Classes that holds Students and StudentClassInfo.
To do sensible things you would most likely want to maintain some sort of relationship between Students and StudentClassInfo. This could either use something as simple as ids (see example below) or perhaps by maintaining a simple index composed out of zodb BTrees depending on what kinds of lookups you need.
How would this work? say you had a view 'about' for a student in the class.
http://myapp/classes/BasketWeaving/Bob/about
It displays some basic information
- Name: Bob Johnson
- Current Grade: F
- Notes: Bob lit last basket on fire.
Traversal would return the "about" view with the context StudentClassInfo (sci). In this case, sci.__name__ == 'Bob', so the view could simply walk back up the object tree to 'root' then access the student instance ala 'root.students['Bob']' to retrieve data such as the full name. The grade and notes would accessible via the sci instance.
-w
d. "whit" morriss
Platform Codemonkey
wh...@surveymonkey.com
Mike Orr
Jan 30, 2013, 4:17:37 PM1/30/13
to pylons-...@googlegroups.com
I'm not a traversal expert but here's my 2c.
Traversal has a small niche where it's significantly better than URL
Dispatch, and a much larger niche where either Traversal or URL
Dispatch are equally effective but they lead to different code
organization.
The case where Traversal is significantly better is when you don't
know beforehand which object types will be at which URLs, especially
when users are allowed to create a tree structure rather than a flat
namespace. Content management systems are the quntessential example of
this.
However, many applications have a fixed URL structure, and a class
management system may be one of those. You said your URLs are like
"/classes/{class_id}/students/{student_id}". In that case, Traversal
vs URL Dispatch is mostly a preference choice. The advantage is that
the dispatcher will look up the resource for you and present it as the
"context", with parents. Otherwise you'd have to look those up
yourself in the view. How much difference does it make? Not much, just
using "context" vs another variable.
And you have to create a resource tree, which wouldn't be necessary
without Traversal. This adds a certain amount of work, so the question
is, how much unique benefit can you add at the same time? It gives the
opportunity to move some of the code in (URL Dispatch) views into the
resource and its parents, as methods or attributes. In other words, in
ordinary (URL Dispatch) model objects, you have your column
attributes, and you may also have some methods and non-persistent
attributes. In Traversal, you can add additional methods and
attributes beyond that to the context and its parents, things that
leverage the Traversal structure and lead to better organized code,
maybe. That "maybe" is important, because is it really better
organized, or are you trying to shoehorn things into the
resource/parent structure that don't really belong?
There hasn't been much written about what kinds of things are worth
pushing into the resource structure in a fixed-URL application, so I
don't really have an answer. Sometimes I think about putting index
wrappers; e.g., where "/articles" is an index page for "/articles/X".
In that case, I don't need the full child objects, but just enough
fields to generate the index page and do searches and the like. So
should I make the index result list (or a paginator of it) into the
"context" of the index view? But what have I gained? It's just moving
a query from one place to another. Whether it's called "context" or
not is perhaps unimportant. And when I go through the parent to a
child context, I don't want the parent generating a superfluous query
I'll never use.
Similar issues come up in building up breadcrumbs links or
neighbor-navigation links. If the parent generates a query just by
traversing it, then you're continually having root queries and section
queries on every request. And to get that list of neighbor-links
(siblings, etc), you'd have to go far and wide through the resource
tree on every request, and if you naively make it fetch all these
other nodes entirely rather than using a limited index query, you end
up fetching a whole lot of stuff you won't use in that request. So
these things *can* be done via the resource tree, but it's not
necessarily always the ideal way.
Another issue is generating URLs to other nodes. It's easy to generate
the context's URL or an ancestor's. But what about another node? Is it
really worth fetching that object through the tree and asking it its
URL? Or is it OK to just hardcode the URL down from a resource you
already know (the root or an ancestor); e.g.,
'request.resoufce_path(root, "other_view")' or
'request.resource_path(root, "other_section", "other_view")'. I used
to avoid that as improperly hardcoding names, but now I'm starting to
wonder, because the method *does* seem to explicitly have that
capability.
Anyway, those are my current thoughts about traversal.
Dispatch, and a much larger niche where either Traversal or URL
Dispatch are equally effective but they lead to different code
organization.
The case where Traversal is significantly better is when you don't
know beforehand which object types will be at which URLs, especially
when users are allowed to create a tree structure rather than a flat
namespace. Content management systems are the quntessential example of
this.
However, many applications have a fixed URL structure, and a class
management system may be one of those. You said your URLs are like
"/classes/{class_id}/students/{student_id}". In that case, Traversal
vs URL Dispatch is mostly a preference choice. The advantage is that
the dispatcher will look up the resource for you and present it as the
"context", with parents. Otherwise you'd have to look those up
yourself in the view. How much difference does it make? Not much, just
using "context" vs another variable.
And you have to create a resource tree, which wouldn't be necessary
without Traversal. This adds a certain amount of work, so the question
is, how much unique benefit can you add at the same time? It gives the
opportunity to move some of the code in (URL Dispatch) views into the
resource and its parents, as methods or attributes. In other words, in
ordinary (URL Dispatch) model objects, you have your column
attributes, and you may also have some methods and non-persistent
attributes. In Traversal, you can add additional methods and
attributes beyond that to the context and its parents, things that
leverage the Traversal structure and lead to better organized code,
maybe. That "maybe" is important, because is it really better
organized, or are you trying to shoehorn things into the
resource/parent structure that don't really belong?
There hasn't been much written about what kinds of things are worth
pushing into the resource structure in a fixed-URL application, so I
don't really have an answer. Sometimes I think about putting index
wrappers; e.g., where "/articles" is an index page for "/articles/X".
In that case, I don't need the full child objects, but just enough
fields to generate the index page and do searches and the like. So
should I make the index result list (or a paginator of it) into the
"context" of the index view? But what have I gained? It's just moving
a query from one place to another. Whether it's called "context" or
not is perhaps unimportant. And when I go through the parent to a
child context, I don't want the parent generating a superfluous query
I'll never use.
Similar issues come up in building up breadcrumbs links or
neighbor-navigation links. If the parent generates a query just by
traversing it, then you're continually having root queries and section
queries on every request. And to get that list of neighbor-links
(siblings, etc), you'd have to go far and wide through the resource
tree on every request, and if you naively make it fetch all these
other nodes entirely rather than using a limited index query, you end
up fetching a whole lot of stuff you won't use in that request. So
these things *can* be done via the resource tree, but it's not
necessarily always the ideal way.
Another issue is generating URLs to other nodes. It's easy to generate
the context's URL or an ancestor's. But what about another node? Is it
really worth fetching that object through the tree and asking it its
URL? Or is it OK to just hardcode the URL down from a resource you
already know (the root or an ancestor); e.g.,
'request.resoufce_path(root, "other_view")' or
'request.resource_path(root, "other_section", "other_view")'. I used
to avoid that as improperly hardcoding names, but now I'm starting to
wonder, because the method *does* seem to explicitly have that
capability.
Anyway, those are my current thoughts about traversal.
Anurag
Feb 7, 2013, 2:30:09 PM2/7/13
to pylons-...@googlegroups.com
Thanks a lot Whit and Mike, it helped me a lot in understanding the implementation of traversal and understanding the precautions to be taken.
However coming back to my original question of possibility of having same objects in two hierarchies in same application still puzzles me ( just leaving aside the example which I gave in my original question), moreover existence of traversal_wrapper has also caused another confusion to me i.e. unable to store those resources in repoze.catalog's document map due to lack of hardwired location awareness in them.
Don't you guys feel that having same object in two hierarchies is a common scenario or am I somewhat misplaced in my understanding (at least existence of traversal_wrapper suggests that there are scenarios) ?
Thanks,
Anurag
However coming back to my original question of possibility of having same objects in two hierarchies in same application still puzzles me ( just leaving aside the example which I gave in my original question), moreover existence of traversal_wrapper has also caused another confusion to me i.e. unable to store those resources in repoze.catalog's document map due to lack of hardwired location awareness in them.
Don't you guys feel that having same object in two hierarchies is a common scenario or am I somewhat misplaced in my understanding (at least existence of traversal_wrapper suggests that there are scenarios) ?
Thanks,
Anurag
Whit Morriss
Feb 7, 2013, 3:36:36 PM2/7/13
to <pylons-discuss@googlegroups.com>
On Feb 7, 2013, at 1:30 PM, Anurag <anurags...@gmail.com>
wrote:
Thanks a lot Whit and Mike, it helped me a lot in understanding the implementation of traversal and understanding the precautions to be taken.
However coming back to my original question of possibility of having same objects in two hierarchies in same application still puzzles me ( just leaving aside the example which I gave in my original question), moreover existence of traversal_wrapper has also caused another confusion to me i.e. unable to store those resources in repoze.catalog's document map due to lack of hardwired location awareness in them.
I sort of touched on this in my scree, but basically, the hierarchy that makes up the URL must be unique as does the address in the zodb. So the best you can do is fake it wrt two places at once. But that's why you have views...
-w
--
You received this message because you are subscribed to the Google Groups "pylons-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pylons-discus...@googlegroups.com.
To post to this group, send email to pylons-...@googlegroups.com.
Visit this group at http://groups.google.com/group/pylons-discuss?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
Reply all
Reply to author
Forward
0 new messages