TiddlyWeb: Support for groups (and thoughts on bags vs classes)
Michael Mahemoff
appopriate use of bags and recipes, and whether the framework needs to
adapt to support best practice.
On first seeing the tiddlyweb data model, my first impression was that
bags are classes (aka tables) and tiddlers are instances (aka
records). But I now realise it's not quite that. I was assuming a
tiddler "is a" bag member (instantiation) whereas the more fundamental
rule is that a bag "has a" tiddler (aggregation). It's perfectly
possible for different types of tiddler to live in the same bag, and
for tiddlers to move from one bag to another.
I'll walk through a hypothetical case study and offer my thoughts on
how I might implement it, based on what I've seen with TiddlyDocs,
Scrumptious, and TiddlyGuv; I am interested if others see things the
same way or not, and if I am taking advantage of the framework as much
as possible.
EXAMPLE - BAG ARCHITECTURE
One archetype structure that's emerging has two types of tiddler:
something about the domain, and comments. If was designing a site to
discuss games, I'd end up with "Game" tiddlers and "Comment" tiddlers.
STAGE 1: GLOBAL RESOURCES
First instinct says to create a "games" bag (readable by anyone in the
group) *and a "comments" bag.
STAGE 2: PRIVATE RESOURCES
Now, what if we want to support only private conversations? Then we
need a role per group - probably constructed from a GUID - and 2 bags
per group. Group 123 gets "games-123" (read/write etc by everyone in
role "123" and no-one else, perhaps delete and manage by a group
admin), and "comments-123" bags (create/read by everyone in role,
delete/write etc by group admin). These must be in different bags
because they need different policies.
In DB terms, this structure is not normalised - we will have "games-X"
and "comments-X" for every group, and if we delete the group or rename
it, we would have to remember to apply it to all "*-X". In this
particular example, I'd prefer a model where I could have a "123" bag
with a "games" sub-bag having permission "read/write by this bag's
group" and "comments" sub-bag having permission "create by this bags
group". It would feel more DRY that way. But I appreciate the current
model, with flat bags, does make the structure simpler to understand
in some respects, and with the concept of roles, it's not a big deal
(it would be a bigger deal if there weren't roles and each change to
group membership meant changing the policies of each bag).
STAGE 3a: 2 PRIVATE RESOURCE TYPES WITH SAME POLICY
Then we also allow people to talk about "Books". A book has different
fields to a game, so we might view it as a separate class, though I
realise we're in dynamic duck typing territory here and there's really
no concept of a class. But client code has certain expectations about
what it can do with a book, versus what it can do with a game; and
users want a list of books distinct from a list of games. The question
is, where do "books" and "games" go? 2 possibilities:
* "bag-per-policy" strategy: books and games both go in a common
"documents-123" bag.
* "bag-per-class" strategy: separate "books-123" and "games-123" bags.
To keep the structure simple, my preference is "bag-per-policy". But
how do I then get a list of all "games" from the "documents" bag? We
could have a "type:" field in the tiddlers, but the default URL
structure would be weird as it would always be /bags/documents-123/...
for games and books. And getting a list of all books would mean
filtering on the "type" field - slightly cumbersome, and moreover, I
imagine this is a real scaling issue until we have a high-performance
storage implementation.
STAGE 3b: PUBLIC AND PRIVATE RESOURCES
From Stage 2, we might also evolve to a situation where we let groups
publish their finest game articles for everyone to see. This means we
end up with "public-games-123" and "private-games-123". Again, I could
see the argument in favour of a single "123" bag with sub-bags for
each, but again it's not a big deal if we stick to naming conventions.
If we let the group say they will support editing certain articles by
anyone, we also end up with a "global-games" bag. All three bags
contain the same type of thing, but with different permissions.
THE ROLE OF RECIPES
From 3b, we could imagine a recipe to show someone the games they're
interested in, containing the last ten instances from each of "global-
games", "public-games-[group-ID]", and "private-games-[group-ID]". I
recall we have some kind of magic pattern-matching feature like this
in place? Does anyone have a relevant link? I'm looking for a recipe
identified as recipe-123, where the contents of a generic recipe,
perhaps called recipe-{id}, would be global-games, public-games-#{id},
private-games-#{id} type thing.
bengillies
useful, at least. Take the following example with permissions:
Basically, if someone publishes something (an article for instance),
then they effectively move it from their bag into another bag (user123
bag into published_articles bag). The issue then becomes that if that
user then wants to edit their article, they need either edit
permission on the published_articles bag, or they need delete
permission (so they can move it back into their bag, change it, and
put it back).
For one user, this process works fine, but for multiple users, this
becomes trickier, as having one published_articles bag means that
either published articles can *never* be changed, or, the user needs
to be given edit permissions to *everybody*'s articles. The other
options are to have one published bag for every user (thus viewing
published articles would involve looking in potentially thousands of
bags, as would trying to find a particular article), or you would need
a designated admin, whose role is to administrate the published bag,
and keep things in check.
As it currently stands, the latter option is the most feasible, though
clearly not ideal, requiring extra human intervention.
What would work well, IMHO, would be to have an owner permission
setting linked to ownership/creator of each tiddler. This would
essentially be one set of rules for the person who created the
tiddler, and one set of rules for everybody else. I realise that this
would require two sets of policy settings per bag, instead of just
one, and the concept of tiddler ownership in place (different to
modifier). However, it seems pertinent to the point that Mike has made
and would go some way to solving his problem.
Michael Mahemoff
> options are to have one published bag for every user (thus viewing
> published articles would involve looking in potentially thousands of
> bags, as would trying to find a particular article)
alluded to in 3b, but a different type of smart recipe. In this case,
the smart recipe contains a wildcard bag list like "public-*". I
actually think this would be a good way to retain the spirit of the
bag-recipe architecture while giving us the functionality we need.
Chris Dent
On Jul 31, 2009, at 12:24 AM, bengillies wrote:
> What would work well, IMHO, would be to have an owner permission
> setting linked to ownership/creator of each tiddler. This would
> essentially be one set of rules for the person who created the
> tiddler, and one set of rules for everybody else. I realise that this
> would require two sets of policy settings per bag, instead of just
> one, and the concept of tiddler ownership in place (different to
> modifier). However, it seems pertinent to the point that Mike has made
> and would go some way to solving his problem.
There are lots of interesting issues coming out in this thread, and it
is good that we are exploring them. I hope to dive into Mike's initial
message when my brain comes up to speed, but I thought I would go
ahead and respond to just this paragraph because the response to it is
fairly straightforward:
When Jeremy and I first hashed out the overall design and purpose of
TiddlyWeb, making bags the sole arbiter of who or what may access and
manipulate tiddlers was an explicit and intentional choice, knowing
full well that it was "different" and placed some unusual constraints
on how people would or could manage their data.
Adapting to those constraints and coming up with the best practices
for working with them is one of the primary near term challenges for
the TiddlyWeb project.
Chris Dent
Gonna take a stab at responding to just parts of these messages and
that seems a little more accessible.
It should be noted that these issues (in this message) are within the
domain of using TiddlyWeb as a store for content used in published
documents. This is but one of many domains of TiddlyWeb use, and it is
important to keep in mind that it is a generic store, not a CMS.
On Jul 31, 2009, at 12:24 AM, bengillies wrote:
> Basically, if someone publishes something (an article for instance),
> then they effectively move it from their bag into another bag (user123
> bag into published_articles bag). The issue then becomes that if that
> user then wants to edit their article, they need either edit
> permission on the published_articles bag, or they need delete
> permission (so they can move it back into their bag, change it, and
> put it back).
The original idea in the design was that the recipe cascade would take
care of this situation:
* You start with:
** One or more bags where one or more people can edit. Call them
editbags.
** One or more bags where people can't edit but everybody can read.
Call the publishbags.
** Two or more recipes.
*** One recipe puts the editbags _after_ the publishbags. Call this
the draft recipe.
*** One recipe puts the editbags _before_ the publishbags. Call this
the published recipe.
* People who want to edit, view and edit content via the draft recipe.
* When a draft is deemed ready for publication it is _copied_ to the
publishbags.
* When it is time to make changes to that document, the version that
is still in the editbag is edited. When it becomes correct, it is
_copied_ to the publishbags, again.
In the situation where there are multiple people who are editors, I
think wikis in general and wikipedia specifically have made it very
clear that it is perfectly reasonable for a large group of people to
all have edit permission on the same pile of stuff, so rather than
having edit bags for every single individual I think it would be fine
for there to be one edit bag in which people who have the EDITOR role
have the create and write and permissions. If the group of people who
are doing editing can't or won't be respectful to each others edits,
then they have larger problems that no piece of software is going to
able resolve. (Note I'm speaking of groups here, not the entire world
and it makes perfect sense for company A to be a group and company B
to a different group, given issues with silly things like trade
secrets and what not.)
TiddlyWeb very much comes out of the Wiki tradition, which fairly
firmly believes that people can be pretty good, that information is
better when shared and collaboratively developed.
FND
> it is important to keep in mind that [TiddlyWeb] is a generic store
I think that's a fairly fundamental aspect.
In this instance, it appears one option is to build a layer that does
permissions negotiation independent (or on top) of TiddlyWeb proper.
> it is perfectly reasonable for a large group of people to
> all have edit permission on the same pile of stuff
Indeed - it might be worth examining whether more granularity is really
required. (Not suggesting it isn't, but it's worth considering, as a
non-technical issue.)
-- F.
Chris Dent
On Jul 29, 2009, at 5:13 PM, Michael Mahemoff wrote:
> I'd like to open up a conversation about tiddlyweb design and the
> appopriate use of bags and recipes, and whether the framework needs to
> adapt to support best practice.
I'm glad you've started this thread, but as I've indicated elsewhere
I'm having trouble trying to engage with the details of the
conversation and come up with practical solutions to the issues. I
think this is because there's probably a fairly fundamental difference
between the processes and thinking that motivated the development of
TiddlyWeb, and how it is being used.
This is not a bad thing. It's a class example of a good problem to
have. TiddlyWeb has been useful to you in ways that are somewhat
different than it was intended. That's a good thing.
It does, however, lead to some confusion and differences of opinion
and understanding of how things work in theory, work in practice, and
work in expectation.
I'll run through your paragraphs making quick reactions, and then
hopeful be able to summarize at the end.
> On first seeing the tiddlyweb data model, my first impression was that
> bags are classes (aka tables) and tiddlers are instances (aka
> records). But I now realise it's not quite that. I was assuming a
> tiddler "is a" bag member (instantiation) whereas the more fundamental
> rule is that a bag "has a" tiddler (aggregation). It's perfectly
> possible for different types of tiddler to live in the same bag, and
> for tiddlers to move from one bag to another.
My feeling is that using OO or DBMS language in relation to the
TiddlyWeb data model is misplaced. The use of "Web" in the name is
intentional: TiddlyWeb is a document store with characteristics very
similar to the entire open web. What those documents are is variable,
but in the same way that a directory served by a webserver or your
local filesystem can contain a variety of stuff, so too can a bag. In
some ways a bag's policy is like the .htaccess file on a directory
served by Apache.
I think of bags and containers with gatekeepers.
But yes, I agree with your final sentence.
> EXAMPLE - BAG ARCHITECTURE
>
> One archetype structure that's emerging has two types of tiddler:
> something about the domain, and comments. If was designing a site to
> discuss games, I'd end up with "Game" tiddlers and "Comment" tiddlers.
Or you might have "tiddlers which make up the info about a game" (i.e.
the suite of information which describes an individual game is
multiple tiddlers and they all live in a a single bag which describes
that tiddler).
TiddlyWeb is more about composition and aggregation, less about
classes and units.
> STAGE 2: PRIVATE RESOURCES
> [snip]
> In DB terms, this structure is not normalised - we will have "games-X"
> and "comments-X" for every group, and if we delete the group or rename
> it, we would have to remember to apply it to all "*-X".
Yes. Neither is couchdb, hadoop, big table, or the web in general
(normalised). You can, if you like, maintain an index of your own that
does the tracking for you. You could even write code that automates
cascade on delete and things like that.
> In this
> particular example, I'd prefer a model where I could have a "123" bag
> with a "games" sub-bag having permission "read/write by this bag's
> group" and "comments" sub-bag having permission "create by this bags
> group". It would feel more DRY that way. But I appreciate the current
> model, with flat bags, does make the structure simpler to understand
> in some respects, and with the concept of roles, it's not a big deal
> (it would be a bigger deal if there weren't roles and each change to
> group membership meant changing the policies of each bag).
TiddlyWeb makes no attempt to impose DRY-ness in its data handling.
You can manage DRYness yourself if you like, if you can figure out
ways to do it, but as an unstructured document store, DRY and
normalization are not really in its remit.
> STAGE 3a: 2 PRIVATE RESOURCE TYPES WITH SAME POLICY
>
> Then we also allow people to talk about "Books". A book has different
> fields to a game, so we might view it as a separate class, though I
> realise we're in dynamic duck typing territory here and there's really
> no concept of a class.
I'm not sure we're even in duck typing territory. These are documents,
what their structure means is _entirely_ dependent on how the client
choose to interpret them. TiddlyWeb makes no pretensions to
classifying the stuff it stores. If you wish to manage classes and
classification, that's your code and your design that's doing that.
> To keep the structure simple, my preference is "bag-per-policy". But
> how do I then get a list of all "games" from the "documents" bag? We
> could have a "type:" field in the tiddlers, but the default URL
> structure would be weird as it would always be /bags/documents-123/...
> for games and books. And getting a list of all books would mean
> filtering on the "type" field - slightly cumbersome, and moreover, I
> imagine this is a real scaling issue until we have a high-performance
> storage implementation.
It will remain an issue even then. Because a bag is a container that
gets listed, there is no convenient way to access a single bag
containing a very large number of tiddlers in a speedy fashion without
the aid of an auxiliary index and sort of preprocessor. It's better to
have many smaller bags. One way to do that is to use what you're
calling "type" as a sort of hash on the bags. "I want all the tiddlers
which are 'type" book, so I look in the book bag."
(I'm well aware that this is all contrary to some standard ways of
doing data management and that it requires different ways of doing
design, but it is intentional. If, in the final analysis, this model
doesn't work for your use case, then there are many other tools
available.)
> STAGE 3b: PUBLIC AND PRIVATE RESOURCES
>
>> From Stage 2, we might also evolve to a situation where we let groups
> publish their finest game articles for everyone to see. This means we
> end up with "public-games-123" and "private-games-123". Again, I could
> see the argument in favour of a single "123" bag with sub-bags for
> each, but again it's not a big deal if we stick to naming conventions.
> If we let the group say they will support editing certain articles by
> anyone, we also end up with a "global-games" bag. All three bags
> contain the same type of thing, but with different permissions.
I think my response to Ben may address part of this?
> THE ROLE OF RECIPES
>
>> From 3b, we could imagine a recipe to show someone the games they're
> interested in, containing the last ten instances from each of "global-
> games", "public-games-[group-ID]", and "private-games-[group-ID]". I
> recall we have some kind of magic pattern-matching feature like this
> in place? Does anyone have a relevant link? I'm looking for a recipe
> identified as recipe-123, where the contents of a generic recipe,
> perhaps called recipe-{id}, would be global-games, public-games-#{id},
> private-games-#{id} type thing.
In IRC I pointed out the template handling in tiddlyweb.model.recipe.
If people trace that bit of code back to how it is used, it is
possible to see how the templating can be extended.
Here I am at the end of the message and I don't feel like we've made
much progress other than me saying, "It doesn't work that way, adapt
or use something else."
That's not particular productive, so I hope we'll continue hashing
this out.
One practical thing that I think might help all of us is if we figure
out how to use recipes more effectively. In the original conception of
TiddlyWeb it was assumed that recipes would be accessed by "users" far
more often than bags, and the recipe cascade would be be engaged in
many situations. Bags would be accessed more often by "builders". If
we can figure out how to make recipes more useful/accessible/
comprehensible that would probably be a good thing.
It's a known problem that long recipes can be a bit slow, but there
are optimizations that can happen there that have not yet been
explored simply because recipes have thus far been a bit of an
orphaned child: most of the optimization work has happened with
filters and bags.
Michael Mahemoff
> In the situation where there are multiple people who are editors, I
> think wikis in general and wikipedia specifically have made it very
> clear that it is perfectly reasonable for a large group of people to
> all have edit permission on the same pile of stuff, so rather than
> having edit bags for every single individual I think it would be fine
> for there to be one edit bag in which people who have the EDITOR role
> have the create and write and permissions. If the group of people who
> are doing editing can't or won't be respectful to each others edits,
> then they have larger problems that no piece of software is going to
> able resolve. (Note I'm speaking of groups here, not the entire world
> and it makes perfect sense for company A to be a group and company B
> to a different group, given issues with silly things like trade
> secrets and what not.)
>
> TiddlyWeb very much comes out of the Wiki tradition, which fairly
> firmly believes that people can be pretty good, that information is
> better when shared and collaboratively developed.
all, we have revision control and potentially auditing if a plugin
emerges to do that.
One class of systems though is the one I suggest above in stage 2
above, which is like WordPress-MU or Wikia with its multitude of
wikis. In the case of TiddlyDocs, we want different groups to
collaborate on private documents; and in the case of Scrumptious,
different groups to be able to conduct private conversations about
websites. So in this case, there definitely has to be walls between
certain clusters of recipes. Also, to gain acceptance in the
enterprise, certain bags do need some granular permissions within the
group - e.g. shadow tiddlers or anything related to look-and-feel can
only be changed by admin; and comments are append-only. So there does
need to be several of the same kind of bags for each group.
If the answer to this kind of thing is "deploy a separate instance for
each group", then fair enough; I just want to know what do we think
the limitations are.
The situation becomes a little more complex as in 3b when these
private groups can make things public too, but I'm less concerned
about that scenario for now.
> One practical thing that I think might help all of us is if we figure
> out how to use recipes more effectively. In the original conception of
> TiddlyWeb it was assumed that recipes would be accessed by "users" far
> more often than bags, and the recipe cascade would be be engaged in
> many situations. Bags would be accessed more often by "builders". If
> we can figure out how to make recipes more useful/accessible/
> comprehensible that would probably be a good thing.
>
> It's a known problem that long recipes can be a bit slow, but there
> are optimizations that can happen there that have not yet been
> explored simply because recipes have thus far been a bit of an
> orphaned child: most of the optimization work has happened with
> filters and bags.
designed to fit into this kind of territory, and are right now under-
utlised. I think there are two things we need to consider with
recipes:
- whether they're definition at present is adequate; because I feel we
may need something a little smarter with pattern-matching capabilities
- performance, as you suggest.
Chris Dent
> One class of systems though is the one I suggest above in stage 2
> above, which is like WordPress-MU or Wikia with its multitude of
> wikis. In the case of TiddlyDocs, we want different groups to
> collaborate on private documents; and in the case of Scrumptious,
> different groups to be able to conduct private conversations about
> websites. So in this case, there definitely has to be walls between
> certain clusters of recipes. Also, to gain acceptance in the
> enterprise, certain bags do need some granular permissions within the
> group - e.g. shadow tiddlers or anything related to look-and-feel can
> only be changed by admin; and comments are append-only. So there does
> need to be several of the same kind of bags for each group.
>
> If the answer to this kind of thing is "deploy a separate instance for
> each group", then fair enough; I just want to know what do we think
> the limitations are.
What's wrong with deploy with lots of different bags, be clever and
controlled with recipe management and have a good bag management system?
In fact one could use a recipe as the description for what bags needs
to be created for a particular group when a new group is added to the
system.
This may seem cumbersome, but keep in mind that TiddlyWeb is fairly
low level. You can put whatever management code, including
administrative data handling, on top of it that you like. Think of all
the code that is running for you when you run MySQL or something like
that. TiddlyWeb has very little of that stuff.
[recipes]
> - whether they're definition at present is adequate; because I feel we
> may need something a little smarter with pattern-matching capabilities
I hope what the smartness needs to be will be made more clear with time.
Michael Mahemoff
> > If the answer to this kind of thing is "deploy a separate instance for
> > each group", then fair enough; I just want to know what do we think
> > the limitations are.
>
> What's wrong with deploy with lots of different bags, be clever and
> controlled with recipe management and have a good bag management system?
>
> In fact one could use a recipe as the description for what bags needs
> to be created for a particular group when a new group is added to the
> system.
pattern matching concept.
> > - whether they're definition at present is adequate; because I feel we
> > may need something a little smarter with pattern-matching capabilities
>
> I hope what the smartness needs to be will be made more clear with time.
If those ideas appeal, I'd be willing to articulate a change request
for discussion.
Chris Dent
On Jul 31, 2009, at 6:31 PM, Michael Mahemoff wrote:
>>> - whether they're definition at present is adequate; because I
>>> feel we
>>> may need something a little smarter with pattern-matching
>>> capabilities
>>
>> I hope what the smartness needs to be will be made more clear with
>> time.
>
> The basic use cases are covered in earlier posts in this thread, no?
> If those ideas appeal, I'd be willing to articulate a change request
> for discussion.
I'm not sure I fully understand the use cases, nor am I certain the
use cases still hold with the deeper expression of the data model that
we've had today.
But, that said, will expanding the previously mentioned recipe
template stuff handle much of that?
bengillies
extending recipe-type syntax:
Basically, (as Chris is aware of) I'm calling recipes from within
other pages/bags/recipes/etc.
So, for example, If I go to /bags/foo/tiddlers then the code that
generates that page will, in turn, load in additional recipes as
specified by the user (ie - they can specify what recipes should be
called here from within a tiddler). What this means is that in
addition to all the tiddlers in foo being shown, additional tiddlers
may be included from the "bar" bag as well.
My point on recipes is that it would be particularly useful if I could
specify, within the recipe, some sort of syntax for changing that
recipe based on the original bag included. In my mind, that could look
like the following:
/bags/shadowtiddlers/tiddlers
/bags/config/tiddlers
/bags/{bagname}-bar/tiddlers
In this example then, if the user goes to the "foo" bag and that in
turn calls up this recipe, {bagname} would be replaced by "foo". I
think in Mike's example, this could work with the user going to /bags/
123/tiddlers (123 being the group), which in turn would load up a
recipe with:
/bags/games-{bagname}/tiddlers
/bags/books-{bagname}/tiddlers
Which would effectively aggregate both games and books without
actually needing to create a documents bag to keep them in.
I realise this sort of thing is possible if handcoded in Python
itself, but to that seems quite messy, as I don't expect many users
will want to (or be allowed to) delve into Python themselves. The main
issue that I can see with this though, is what you would then do when
you want to view that recipe on its own. Here, I would imagine you
would need to specify some default bag to use in its place. So
something like:
/bags/{bagname:foo}-bar/tiddlers
Where foo is the default, but overridden when a different bag is
visited.
Chris Dent
On Aug 1, 2009, at 1:13 PM, bengillies wrote:
> Talking about recipes more, I've come across a different use case for
> extending recipe-type syntax:
>
> Basically, (as Chris is aware of) I'm calling recipes from within
> other pages/bags/recipes/etc.
I think it would be helpful, at least to me, if you could repost most
of what you're explain here, in a bit more systematic way. I'm having
trouble following both the user's experience of the steps involved and
also the server side processing steps. Something like:
* User goes to URL /recipes/X/tiddlers
* Server gets recipe X out of the store, the contents of the recipe are
/bags/Y/tiddlers
/bags/{Z}/tiddlers
* Server replaces Z with <whatever it is it replaces it with>
In the current description I'm struggling to determine what goes in
the store and how the recipe engine processes it.
I _think_ but can't be sure that it is probably possible to do what
you suggest by making the tiddlyweb.control:_recipe_template() method
more powerful.
> So, for example, If I go to /bags/foo/tiddlers then the code that
> generates that page will, in turn, load in additional recipes as
> specified by the user (ie - they can specify what recipes should be
> called here from within a tiddler). What this means is that in
> addition to all the tiddlers in foo being shown, additional tiddlers
> may be included from the "bar" bag as well.
If you go to /bags/foo/tiddlers no other bags are involved. Do you
mean /recipes/foo/tiddlers? From below it sounds like not. Within the
core of TiddlyWeb it is unlikely that anything will be built in that
allows for nested bags, as bags are intentionally defined as
containers of tiddlers and the combination of a bag and a tiddler
makes for the unique name that identifies one tiddler.
It is, however, possible to consider things that happen either in
serializers or recipe processing that do what you want. I'm pretty
sure this is the road you've already gone down. Making a serializer
for a bag that triggers a recipe process is certainly possible.
In that case what's needed is someway of informing the standard recipe
engine how to do the processing.
> My point on recipes is that it would be particularly useful if I could
> specify, within the recipe, some sort of syntax for changing that
> recipe based on the original bag included. In my mind, that could look
> like the following:
>
> /bags/shadowtiddlers/tiddlers
> /bags/config/tiddlers
> /bags/{bagname}-bar/tiddlers
I think modifying _recipe_template is the way to go with this.
> I realise this sort of thing is possible if handcoded in Python
> itself, but to that seems quite messy, as I don't expect many users
> will want to (or be allowed to) delve into Python themselves. The main
> issue that I can see with this though, is what you would then do when
> you want to view that recipe on its own. Here, I would imagine you
> would need to specify some default bag to use in its place. So
> something like:
A plugin could provide a _recipe_template (or whatever we evolve that
to be) that takes configuration (rather than code) instructions from
tiddlywebconfig, allowing people to get these results without doing
their own coding in Python. Somebody has to do the initial
recipe_template coding though.
bengillies
>
> I think it would be helpful, at least to me, if you could repost most
> of what you're explain here, in a bit more systematic way. I'm having
> trouble following both the user's experience of the steps involved and
> also the server side processing steps. Something like:
idea.
>
> * User goes to URL /recipes/X/tiddlers
>
> * Server gets recipe X out of the store, the contents of the recipe are
>
> /bags/Y/tiddlers
> /bags/{Z}/tiddlers
>
> * Server replaces Z with <whatever it is it replaces it with>
>
> In the current description I'm struggling to determine what goes in
> the store and how the recipe engine processes it.
unless there's a bag with the same name and you replace Z with that,
as there'd be no bagname/whatever to insert into the recipe.
A probably better example would be if you created a custom URL that
includes bag names/tiddler names/whatever and you wanted to put them
into your recipe. So you go to
/url/books/warandpeace
and "books" and "warandpeace" could then be passed into the recipe as
{bagname} and {tiddlername}.
>
> I _think_ but can't be sure that it is probably possible to do what
> you suggest by making the tiddlyweb.control:_recipe_template() method
> more powerful.
>
> It is, however, possible to consider things that happen either in
> serializers or recipe processing that do what you want. I'm pretty
> sure this is the road you've already gone down. Making a serializer
> for a bag that triggers a recipe process is certainly possible.
>
> In that case what's needed is someway of informing the standard recipe
> engine how to do the processing.
>
> A plugin could provide a _recipe_template (or whatever we evolve that
> to be) that takes configuration (rather than code) instructions from
> tiddlywebconfig, allowing people to get these results without doing
> their own coding in Python. Somebody has to do the initial
> recipe_template coding though.
_recipe_template, and see about adding {bagname}, {tiddlername} and a
wildcard of some sort.
bengillies
> as I'm trying to do this already, I'll have a look at
> _recipe_template, and see about adding {bagname}, {tiddlername} and a
> wildcard of some sort.
I was able to change the bag name dynamically based on the URL, but
was unable to change anything else as the recipe template (returned
from _recipe_template) is not used on the filters. Also, the
templating used seems incompatible with wildcards.
Looking at the code in get_recipe, specifically this bit:
for bag_name, filter_string in our_list:
for key, value in template.items():
bag_name = bag_name.replace('{{ ' + key + ' }}',
value)
real_list.append([bag_name, filter_string])
It does not seem like a big job to add these in.
Chris - can you see any issues with doing this?
Chris Dent
On Aug 3, 2009, at 8:56 PM, bengillies wrote:
> I had a look at this today, and met with *some* success.
>
> I was able to change the bag name dynamically based on the URL, but
> was unable to change anything else as the recipe template (returned
> from _recipe_template) is not used on the filters. Also, the
> templating used seems incompatible with wildcards.
I think if you want to be doing dynamic generation of filter strings
that's something you should do definitely do so on the fly in a plugin
(akin to the dynarecipe plugin). The notion of fill in the blank bags
in recipes that are stored makes some pretty good sense and is fairly
easy to get one's mind around, but adding variables in the filter
strings on top of that starts getting pretty arcane: the
representations of the recipes loses a lot of inspectability.
You could probably make custom filters that would provide the
equivalent functionality as templated filters.
> Looking at the code in get_recipe, specifically this bit:
>
> for bag_name, filter_string in our_list:
> for key, value in template.items():
> bag_name = bag_name.replace('{{ ' + key + ' }}',
> value)
> real_list.append([bag_name, filter_string])
The thing I can see adding in here is support for the defaults syntax
that you suggested.
bengillies
directly, so yes it would make sense.
On the filtering syntax, I agree with you about wildcards in that
plugins adding new filters are a better way to go (http://is.gd/21aAl
this one for example accomplishes a similar effect). However, I'm not
sure that adding a key/value type syntax like that already in the bag
name is particularly confusing (for stored recipes at least). Adding a
line to a recipe like the following:
/bags/published_documents/tiddlers?select=modifier:{{ user }}
would make sense, as the extra syntax is clearly linked with recipes
and not filters. In this context, being told that you can use the
syntax only in part of the recipe would (in my mind) be more
confusing. I believe the differentiator in this respect would be that
the syntax is for recipes and not filters or bags specifically.
Chris Dent
On Aug 4, 2009, at 12:04 AM, bengillies wrote:
> Adding a
> line to a recipe like the following:
>
> /bags/published_documents/tiddlers?select=modifier:{{ user }}
>
> would make sense, as the extra syntax is clearly linked with recipes
> and not filters. In this context, being told that you can use the
> syntax only in part of the recipe would (in my mind) be more
> confusing. I believe the differentiator in this respect would be that
> the syntax is for recipes and not filters or bags specifically.
I can buy that. Could you make a ticket in trac explaining/requesting
an enhancement that:
1. extends the template syntax to enable defaults (giving examples of
the syntax)
2. extends the template code so that replacements are done in both the
bag part and the filter part
(and assign it to me, give it a component of tiddlyweb, etc)
And I'll likely make it happen sometime over the next couple of days.
Thanks.