[TW5] Get list of descendants by recursively assembling filter?

[Linus Johnsson]

Dear all,

I am working on a project in which I need to be able to get a list of a tiddler's descendants. I use tags to define the relationship in question. To be precise, B is a descendant of A if A=B or B is tagged with a descendant of A.

The list should be equal to what you get if you evaluate a filter that goes something like this:

[all[current]tag[concept]] [all[current]tag[concept]tagging[]tag[concept]] [all[current]tag[concept]tagging[]tag[concept]tagging[]tag[concept]] ...

The problem is of course that my current solution does not allow for arbitrary depth of search, which is what I want.

I have been trying to wrap my head around this problem and still can't judge whether or not it's possible to do this without resorting to JavaScript. If it is, I would be overjoyed if anyone would point me in the right direction. If it isn't, I would appreciate help even more since I am not (yet) proficient in JavaScript.


Many thanks in advance,
Linus

[c pa]

Linus,

Use something like this. It's a call to a macro that recursively calls itself.

\define listChildren(tag)
    <$list filter="[tag[$tag$]]">

        <!-- Change the following line to display whatever it is you want to be displayed for each child tiddler-->
        <$view field="title"/><br/>  

        <$macrocall $name="listChildren" tag=<<currentTiddler>> />
    </$list>
\end

!Top   <!-- The name of the top tiddler (or you could do an [all[current]] filter -->

<$macrocall $name="listChildren" tag="Top" />

[Linus Johnsson]

Dear c pa,

Thank you for your kind advice. Your code does what it is supposed to, of course. But since many of my tiddlers contain spaces, the resulting list will not work as a filter. (I need to perform a second filter run on the results rather than putting all the functionality in the macro, for to do the latter would in my case result in duplicates.) Obviously, to get this to work I need double square brackets around each title. For reasons that are obscure to me, [[<$view field="title />]] just renders as plain text. I have no idea how to escape those brackets so that <$view field="title /> is wikified. I cannot find any information on this on the web. Any help would be appreciated.

Best regards,

Linus

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/XH4tgLlr7fY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/89913824-5086-41a5-8d89-905970414975%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Evan Balster]

http://tiddlywiki.com/static/TextWidget.html

<$text text="[["/><$view field="title"/><$text text="]]">>

There might be a simpler way, but this works.

...So you're rendering this all to a string for use as a parameter to another filter?  If you manage that, I'm very curious as to how.  (I've not managed to figure out how to render WikiText to plaintext for use in string attributes.)

[c pa]

Linus,

Ahhh, the old create a list and then parse it trick. I have done that before. I think what I ended up doing is storing the list in a temporary tiddler's list  and then parsing that list to get the results

You can use &#91; and &#93; to create square brackets

So something like &#91;&#91;<$view field="title"/>&#93;&#93; will create the title

   

[Linus Johnsson]

Thank you for your kind replies. Both suggestions rendered brackets just fine. As you seem to have anticipated, this immediately landed me in a different kind of problem. I defined the following macro:

\define listDescendants(tag)
    <$list filter="[tag[$tag$]tag[concepts]]">

        &#91;&#91;<$view field="title" />&#93;&#93;

        <$macrocall $name="listDescendants" tag=<<currentTiddler>> />
    </$list>
\end

The macro works as intended. I do not understand, however, why the following (trivial) examples will not reproduce the list returned by the macro. I need to get this to work in order before I add filter runs. First,

<$set name="myList" value=<<listDescendants "NameOfConceptTiddler">> >
<$list filter=<<myList>> />
</$set>

returns the macro code itself, with the spaces between the tokens removed. As for my second attempt,

<$set name="myList" value=<<listDescendants "NameOfConceptTiddler">> >
<$list filter="$(myList)$" />
</$set>

it just returns "$(myList)$".

Wrapping the calling code in a macro does not work either:

\define myMacro
<$set name="myList" value=<<listDescendants "NameOfConceptTiddler">> >
<$list filter=$(myList)$ />
</$set>
\end

<<myMacro>>

All I get is "\define myMacro $(myList)$ \end".

I am thoroughly confused. Obviously there is something about the syntax that I have not yet grasped. Please enlighten me!

Best regards,

Linus

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/XH4tgLlr7fY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/c052b929-6b6b-492d-8b75-0bdf2df6b3d9%40googlegroups.com.

[Evan Balster]

A few notes:

1) Macros always have to be at the top of the file, before any WikiText.

2) The syntax $(variable)$ only applies inside macros.  Outside macros, use <<variable>>.

3) Your trick isn't working because there's no way to render WikiText into a filter expression.  Basically, the code inside <<myList>> doesn't get executed if it's passed to an attribute like <$list filter=<<mylist>> >.  I've just opened a topic suggesting a feature which would make this possible:  read here.  That said, it's likely there are other ways to solve your problem which don't involve composing an intermediate list.

[Linus Johnsson]

Through your suggestions I have made at least some progress. Given the code

\define listDescendants(tag)
    &#91;&#91;$tag$&#93;&#93;

    <$list filter="[tag[$tag$]tag[concepts]]">
        &#91;&#91;<$view field="title" />&#93;&#93;
        <$macrocall $name="listDescendants" tag=<<currentTiddler>> />
    </$list>
\end

\define myMacro()

<$set name="myList" value=<<listDescendants "NameOfConceptTiddler">> >

<$set name="finalFilter" value=<<assembleFilter>> >
<<myMacro2>>
</$set>
</$set>
\end

\define assembleFilter()
$(myList)$ +[tagging[]tag[memos]]
\end

\define myMacro2()
<$list filter=<<finalFilter>> >
<<showListItem>>
</$list>
\end

\define showListItem()
&raquo; <$link to={{!!title}}><$view field="caption" ><$view field="title" /></$view></$link><br>
\end

<<myMacro>>

I do get a list of memos tagged with concepts in the tree that I am traversing, but strangely enough it does not include those tagged only with the ancestor. Any ideas?

@Evan, some comments on your notes:

1) Check.

2) It appears that $(variable)$ does not always work inside macros either. For instance, given that finalFilter is a variable defined in the enclosing macro myMacro2,

\define myMacro2()
<$list filter=<<finalFilter>> >
<<showListItem>>
</$list>
\end

works, while

\define myMacro2()
<$list filter=$(finalFilter)$ >
<<showListItem>>
</$list>
\end

does not.

3) Using <$set> as in the above example appears to be a rather simple workaround given the limitations of <$filter>.

Best regards,

Linus

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/3e9b1041-7721-479d-96f3-a7e9eb4c7930%40googlegroups.com.

[Evan Balster]

2)  The reason for this behavior is that macros do not set variables.  They're a text replacement system, and widgets (like <$list> or <$set>) are not run until the macro has been applied.  $(variable)$ is a substitution that happens within the macro, while <<variable>> happens after.  These subtleties can make TiddlyWiki programming a bit of a headache at times.

3)  A good way to debug filter generation is to use a <$text> widget, which will display the output of a macro without further parsing.  IE, it will use the same rule to display the result of a macro as would be used if that macro was used as a filter.

As a test, I made a few small modifications to your code and pasted the following into a new tiddler on TiddlyWiki.com:

\define listDescendants(tag)

    &#91;&#91;$tag$&#93;&#93;

    <$list filter="[tag[$tag$]]">

        &#91;&#91;<$view field="title" />&#93;&#93;

        <$macrocall $name="listDescendants" tag=<<currentTiddler>> />

    </$list>

\end

\define myMacro()

<$set name="myList" value=<<listDescendants "HelloThere">> >

<$set name="finalFilter" value=<<assembleFilter>> >

<<<

<$text text=<<finalFilter>>/>

<<<

<<myMacro2>>

</$set>

</$set>

\end

\define assembleFilter()

$(myList)$

\end

\define myMacro2()

<$list filter=<<finalFilter>> >

<<showListItem>>

</$list>

\end

\define showListItem()

&raquo; <$link to={{!!title}}><$view field="caption" ><$view field="title" /></$view></$link><br>

\end

<<myMacro>>

The output of that includes a bunch of garbage information, and a listing of the tiddlers which are direct descendants of the "root concept".  The reason is that the filter you're generating, as indicated by the Text widget, is this:

&#91;&#91;HelloThere&#93;&#93; <$list filter="[tag[HelloThere]]"> &#91;&#91;<$view field="title" />&#93;&#93; <$macrocall $name="listDescendants" tag=<<currentTiddler>> /> </$list>

All of that goes into the filter parser, which interprets everything other than [tag[HelloThere]] as a tiddler to be included in the list.  The <$list> widget is not executed beforehand.  Because none of those junk tiddlers exist with a tag "concept", you don't see them when running your own code.  The children of HelloThere show up as normal, however.

See the attached screenshot, showing what the rendered page (with all junk tiddlers) looks like.

The take-away from all this is that there is no wikification of text that gets fed into attributes such as filters -- TiddlyWiki would need additional functionality for this to be possible.  It's another conceptual headache with the relation between macros and widgets that took me a long time to understand.

If you want to do what you're trying to do, you'll probably need to start over with a technique that doesn't involve generating a filter or list.  Try just printing the name of each item in your heirarchy individually.

[Evan Balster]

Here's an approach which doesn't require assembling an intermediate list.  Like my last example, this is most easily demonstrated by creating a new tiddler on TiddlyWiki.com and previewing it.

\define heirarchy_sub()

<$list filter="[tag{!!title}]">

<li> <$link>{{!!title}}</$link> </li>

<ul>

<$macrocall $name="heirarchy_sub">>

</ul>

</$list>

\end

\define heirarchy(ROOT)

<$tiddler tiddler="$ROOT$"><<heirarchy_sub>></$tiddler>

\end

<ul>

<<heirarchy "TableOfContents">>

</ul>

[Linus Johnsson]

Dear Evan,

Thanks for clarifying. It all makes sense to me now. I wonder how long I will be able to remember these distinctions though!

I have defined a macro that seems to be quite close to what you are suggesting:

\define traverseDescendants(tag, displayMacro)
    <$list filter="[tag[$tag$]tag[concept]]">
        <$macrocall $name="$displayMacro$" />
        <$macrocall $name="traverseDescendants" tag=<<currentTiddler>> displayMacro="$displayMacro$" />
    </$list>
\end

It takes a tag as a parameter (when called from the outside, the root of the tree to be traversed) and the name of a macro to invoke for each tiddler in the tree. I have added the tag "concept" to the filter in order to get just the tree of concepts because I use concepts to tag other kinds of tiddlers. The display macro passed to traverseDescendants could then display something else entirely. In my case, that would be all memos tagged with any concept in the tree.

Because memos in my wiki may be tagged with several concepts, I still face the problem of defining the display macro so that duplicates are removed. I am going to look into this now.

Best regards,

Linus

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/e52e2c41-93c3-4553-bfa2-26ea7d41a02d%40googlegroups.com.

[Linus Johnsson]

I just noticed that the above code does not call displayMacro for the root. This does the trick though:

\define traverseDescendants(tag, displayMacro)
    <$tiddler tiddler="$tag$">
        <$macrocall $name="$displayMacro$" />
    </$tiddler>
    <$list filter="[tag[$tag$]tag[§begrepp]]">

        <$macrocall $name="traverseDescendants" tag=<<currentTiddler>> displayMacro="$displayMacro$" />
    </$list> 
\end

Best regards,

Linus

[Tobias Beer]

This begs for a solution similar to x-plore [1] for TW2, which...

• only indexes any tiddler once (to avoid recursion)
• allows to exclude tiddlers from being indexed
• allows to truncate the traversal at specified tiddlers

Please excuse the non-commented spaghetti code from back then.

In short, I don't think the core provides an (output) interface to meaningfully traverse tiddlers across levels.

So, this will likely require a js macro, plugin, widget of its own, e.g.

<$traverse root="Foo" exclude="[tag[bar]][prefix[$]]" truncate="[tag[admin]]" output=flat mode=tagging/>

...whereas:

• root
• a filter expression specifying root nodes for the traversal (tree[s])
• default: default tiddlers
• exclude
• a filter expression specifying tiddlers that are not to be indexed or further traversed
  
• default: <none>
• truncate
• a filter expression specifying tiddlers that are indexed but not further traversed
  
• default: <none>
• output
• an output mode, e.g. a tree or a flat list
• default: tree
• mode
• what relation to index, e.g. tagging, tags, referencing, referenced by, listing, listed, any, etc...
• default: any

[1] click on x-plore toolbar button at http://tbgtd.tiddlyspot.com/#x-plore

— tb

[Linus Johnsson]

Evan and Tobias, after having done some thinking, I have finally decided that I agree with you. Without js, we seem to have hit a brick wall. (I could of course have just trusted you on this, but felt compelled to discover it for myself.)

Tobias, your plugin seems to be a good start. If I am not mistaken, it would need to be modified slightly to do what I want. Specifically, I would need it to have another filter parameter that specifies tiddlers to be traversed but not indexed.

Would it be possible to adapt x-plore to TW5?

Best regards,

Linus

2015-09-15 11:11 GMT+02:00 Tobias Beer <beert...@gmail.com>:

This begs for a solution similar to x-plore [1] for TW2, which...

• only indexes any tiddler once

• excludes tiddlers from being indexed
• truncates the output tree at specified tiddlers

Please excuse the non-commented spaghetti code from back then.

In short, I don't think the core provides an (output) interface to meaningfully traverse tiddlers across levels.

So, this will likely require a js macro, plugin, widget of its own, e.g.

<$traverse root="Foo" exclude="[tag[bar]][prefix[$]]" truncate="[tag[admin]]" output=flat mode=tagging/>

...whereas:

• root
• a filter expression specifying root nodes for the traversal (tree[s])
• default: default tiddlers
• exclude
• a filter expression specifying tiddlers that are not to be indexed or further traversed
  
• default: <none>
• truncate
• a filter expression specifying tiddlers that are indexed but not further traversed
  
• default: <none>
• output
• an output mode, e.g. a tree or a flat list
• default: tree
• mode
• what relation to index, e.g. tagging, tags, referencing, referenced by, listing, listed, any, etc...
• default: any

[1] click on x-plore toolbar button at http://tbgtd.tiddlyspot.com/#x-plore

— tb

--

You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/XH4tgLlr7fY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/aa931397-f2eb-4b60-80ef-f2df2f4a6607%40googlegroups.com.

[Tobias Beer]

Tobias, your plugin seems to be a good start. If I am not mistaken, it would need to be modified slightly to do what I want. Specifically, I would need it to have another filter parameter that specifies tiddlers to be traversed but not indexed.

Not sure I understand this part. Can you give an example (hierarchy) where you want tiddlers being traversed but not part of the returned list?

 

Would it be possible to adapt x-plore to TW5?

Would be more like a rewrite but not impossible. Right now I'm not up to speed on constructing output. These things are perhaps a tad bit more compliocated with widgets as compared to wikify(sometext, place) from TWc.

In your OP, you were talking about a list, not a (traversal-)tree. Can you expand on that and what precisely you have in mind?

— tb

[Linus Johnsson]

What I have in mind is a flattened tree (which I referred to as a "list" in my OP). The basic use case is like follows: I have a concept tree. Each concept tags zero or more memos. Each memo is tagged with zero or more concepts. I want a list of all memos that are tagged with any concept in a specified subtree. So I would need to traverse all concepts but only index the memos. 

Of course, if the plugin returns a list that can be used in further filter runs, I could just index everything that has been traversed and filter out the concepts afterwards. Maybe this is necessary anyway because my use case is really somewhat more complicated. I have defined a third tiddler type, "meaning unit" (I'm into qualitative research), that can be tagged with concepts and tag memos. I consider this indirect link between concept and memo semantically equivalent to a direct one. So it would seem that I need to make two separate runs and combine their results. 

Hope this is makes sense. Looking forward to your ideas!

Best regards,

Linus

Sent using CloudMagic

--

You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/XH4tgLlr7fY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/32f8516d-4496-423d-94b5-4041ef81a97b%40googlegroups.com.

[Tobias Beer]

Here's an example tagging tree and how I understand you wish to collect all "descendant memos":

• Foo (is category)
• Bar (tags: Foo Memo)
  
• Baz (tags: Bar Memo)
• Mumble (tags: Bar)
• Frotz (tags: Foo)
• Gronk (tags: Frotz Memo)

So, the recursive-flat-list would return for category Foo and tag Memo:

• Bar, Baz, Gronk
  

Not sure how exactly "Meaning Units" fit into this picture and how they relate to either categories or memos.

As you only wish a flat list as the output, the traversal could actually be accomplished with a new recursive filter, rather than a macro or widget.

Hope this is makes sense. Looking forward to your ideas!

Try to provide real-life examples equivalent to the above.
That tends to make things more clear, possibly to you as well. ;-)

— tb

[Linus Johnsson]

If we for the time being ignore "meaning units", the structure is rather something like this:

• Foo (tags: concept)
• Bar (tags: Foo concept)
• Baz (tags: Bar memo)
• Mumble (tags: Bar concept)
• Frotz (tags: Foo memo)

In other words, concepts tag other concepts, and memos tag concepts.

Meaning units ("mu") add some complexity. They can be thought of as snippets of text (for instance quotations) to be coded and categorised.

• Foo (tags: concept)
• Eeney (tags: Foo mu)
  
• Bar (tags: Foo concept)
• Meaney (tags: Bar mu)
• Gronk (tags: Meaney memo)
  
• Baz (tags: Bar memo)
• Mumble (tags: Bar concept)
• Frotz (tags: Foo Miney memo)

Where "Miney" is a meaning unit which is not directly tagged with any concept, but is nevertheless indirectly associated with Foo (through the memo Frotz). A meaning unit can thus be brought in under a concept either by tagging it directly with the concept, or by writing a memo about the concept and also tagging that memo with the meaning unit.

If I understand your notion of "recursive" filter correctly, one would still need something equivalent to x-plore to create a list that can be used in further filter runs (in my case, to weed out anything but the concepts)?

Best regards,

Linus

--

You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/XH4tgLlr7fY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/62b33306-7717-4f14-a019-b16a68a8e8dc%40googlegroups.com.

[Tobias Beer]

Hi Linus,

Thanks for the outline, your structure is much clearer now.

 

If I understand your notion of "recursive" filter correctly, one would still need something equivalent to x-plore to create a list that can be used in further filter runs (in my case, to weed out anything but the concepts)?

The complexity would be reduced to introducing an appropriate filter that is capable of doing the recursion... and then returning a flat list of corresponding (unique) tiddler titles, even though gathered through recursion. No need to fiddle with "tree" outputs, yet (think x-plore).

What is a bit tricky, if not impossible atm, is that such a filter would ask for two parameters being filter expressions themselves...

• travesal-filter
  
• a filter defining what nodes to recursively traverse
• e.g. "[all[current]tag[Concept]] [all[curent[tag[Memo]]"
• collect filter
• defining what tiddlers to collect during traversal
• defaults to the "traversal filter" but can be different
• e.g. "[all[curent[tag[Memo]]"

The problem with this is that currently there is no way to specify a filter as a parameter inside another filter. So, I guess, that brings us back to using something else other than a filter to begin with. Right now, I'm assuming it would, initially become a javascript macro with the above two parameters.

Best wishes,

— tb

[Majid al-Dosari]

these kinds of things are now pretty much solved with the wikify widget https://github.com/Jermolene/TiddlyWiki5/issues/2337

I fired up my tw from source and i now can construct recursive macros to generate filters and then have their output (through a wikify) go in a filter attribute of whatever widget that uses it.

[LorenzGL]

I'm resurrecting this thread because I'm stuck trying to feed the output of a recursive macro into a filter using wikify. The problem seems to be that the macro is outputting a formatted list but the filter expects a title list. I haven't been able to figure out how to solve the problem. The goal is to display a list of all individuals of a superclass for individuals that are only linked to subclasses but not the superclass. Here is my code:

\define class_sub()
<$list filter="[is_subclass_of{!!title}]">
<$link><<currentTiddler>></$link>
<$macrocall $name="class_sub">>
</$list>
\end

<$wikify name="all_subclasses" text=<<class_sub>> output=html>
<<list-links filter:"[enlist<all_subclasses>listed[is_individual_of]]">>
</$wikify>

Any help on this would be greatly appreciated.

[Michael Manti]

Hi,

I'm resurrecting this thread again because other threads suggest that wikify-then-enlist is the solution, but I can't seem to get it to work. I'm using the list field to manage a hierarchy; the descendants are in the list field.

\define wrap(text,left:"[[",right:"]]") $left$$text$$right$

\define list-descendants(parent,list:"list")

  <$tiddler tiddler="""$parent$""">

      <$list filter="""[all[current]has[$list$]$list$[]] -[all[current]]"""

             variable="child">

 <$macrocall $name="wrap"

                    text=<<child>>/>

        <!--

<$link to=<<child>>/>

-->

<$macrocall $name="list-descendants"

                    parent=<<child>>

                    list="$list$"/>

      </$list>

  </$tiddler>

\end

\define descendants() <<list-descendants parent:"A">>

\define list-descendant-terms()

  <$wikify text=<<descendants>> name="wikifiedDescendants">

  <$list filter="[enlist<wikifiedDescendants>]"/>

</$wikify>

\end

My test tiddlers are:

title: A

list: [[A 1]] [[A 2]]

title: A 1

list: [[A 1 i]] [[A 1 ii]]

title: A 2

list: [[A 2 i]]

title: A 1 i

title: A 1 ii

title: A 2 i

The wrap, list-descendants, and descendants macros work, but feeding their output into a filter in list-descendant-terms yields A1iii2, where A is a link to tiddler A.

Any help with this would be appreciated. Thanks!

Mike