Looking for way to generate a table with tiddlers sorted according to respective sets of tags

[steve]

Hello

I would like to use a list filter to generate a table with two columns:
* the first column is the name of the tiddlers that match the search criteria;
* the second column is the list of tag pills for each tiddler in the first column;
* the tag pills in the second column are listed in alphabetical order
* the tiddlers in the first column are sorted in order of their list of tag pills

For example

| TiddlerOne |TagA |
| TiddlerTwo |TagA, TabB
| TiddlerTwoB |TagA, TabB |
| TiddlerThree |TagA, TagB, TagC |
| TiddlerFour |TagB, TagD |
| TIddlerFIve |TagD |


The following almost works. By almost I mean that the tags for each tiddler
are listed in alphabetical order, the tiddlers are mostly listed in order of
their respective list of tags and that all the tiddlers with a tag=reference are
listed.

<table>
<$list filter="[!is[system]tag[reference]sort[tags]]">
<tr>
<td><$link to={{!!title}}><$view field="title"/></$link></td>
<td><$list filter="[all[current]tags[]sort[title]]" template="$:/core/ui/TagTemplate" storyview="pop"/></td>
</tr>
</$list>
</table>

I vaguely recall that it was possible to do this in TWC. Any ideas on how to do this in TW5.

Thanks
Steve W

[steve]

Update

You can get a better idea of what I'm talking about from the example at http://tiddlysquat2.tiddlyspot.com/

Steve Wharton

[Tobias Beer]

Hi steve,

I believe the crucial bit would be...

 the tiddlers in the first column are sorted in order of their list of tag pills

I don't think it makes much sense to sort tiddlers by their tags field;

the sequence of tags in it being pretty random.

Why are you trying to do that?

Tobias.

[steve]

Tobias

I agree that it does not make sense to sort the tiddlers by their unsorted tags.
What I was wanting to do is list tiddlers in order of their sorted list of tags.

My use case is to review the tiddlers that have a particular tag (or any other filter criteria)
so that I can identify "under described" tiddlers, e.g., a tiddler tagged "reference" and no
other tags (to identify tiddlers that need an additional tag to identify the type of reference
or alternatively combinations of tags that no longer make sense.

I'm good at writing lots of notes in my wiki. I'm less good at being able to find and organize
previous notes. Reviewing, i.e., curating the tags is helpful to me in keeping the tiddlers
easy to discover and avoiding creating redundant notes.

I hope this helps clarify my question.

Thanks for the response
Steve

[Tobias Beer]

Hi again, steve,

I think sorting should be by some other criteria.

What you may want is another "dynamic" filter, e.g. via a select widget that allows to further constrain the list to only those tiddlers having it, the selectable tags being any of those of items in the list.

Best wishes, Tobias.

[Jed Carty]

I made something sort of like what you are talking about here. It is slow and there are still a bunch of bugs but it does some of it anyway.

It can be a more involved process than you expect depending on what you allow as valid search criteria.

[steve]

Tobias, Jed:

Again, thanks for responding.

I believe that the available filter criteria (by tags, types, content, dates, etc) is not the
limiting factor. Using these I can whip up a list filter to do whatever I want. The example
on http://tiddlysquat2.tiddlyspot.com/ selects tiddlers that match tag[reference].

The part that seems to be missing is the means to:
[1] sort tags[] for each tiddler and concatenate the sorted tags into a string (maybe use a temporary field?) and
[2] sort the tiddlers according to their respective tag Strings

The example looks like it is doing #1 correctly, i.e., sorting the tags.
The example is only partially successful with #2, sorting the tiddlers.

Tobias: I'll look into your suggestion regarding the use of a dynamic filter.
Jed: Is there a way that I can view the tiddler from your example?

Thanks
Steve Wharton

[Jed Carty]

The tiddler is here.

For [1], I thought the tags field was put into alphabetical order when you saved the tiddler, but that doesn't seem to be the case. Those two steps are what makes it a more involved process than it seems like it should be, at least if you want to order things dynamically.

As an alternative you could use this button:

<$button>Sort Tags
<$list filter='[!is[system]]'>
<$set name=TagsString filter='[<currentTiddler>tags[]sort[]]'>
<$action-setfield tags=<<TagsString>>/>
</$set>
</$list>
</$button>

Which will sort the tags field of all non-system tiddlers in your wiki. You could just use that every so often to keep things in order.

[Erwan]

Hi all,

Steve I think that your use case is interesting because it shows a limitation of sorting with TW filters (although imho in this specific case the best solution would be to sort the list of tags automatically every time a tag is added to a tiddler). I think another case with the same limitation is when one wants to sort a list of tiddlers by the number of times they are used as tags (i.e. number of tiddlers tagged with them).

I'll try to describe the issue here: the "sort" operator (and the related nsort, sortcs, nsortcs) only work with fields, but sometimes it might make sense to compute the sort criterion with a "sub-filter" instead. As Jed's solution shows, currently the only way is to create the field containing the sort key: in this case the key is the result of the "sub-filter" "[<currentTiddler>tags[]sort[]]". In my other example about sorting tags by the number of times they are used, the "sub-filter" would be "[<currentTiddler>tagging[]]".

I wonder if it would be possible to implement new "sort" filter operators which would allow the usage of a sub-filter like this? The way I see it it would be used like this:

- Steve's case: "[....sort-list-filter[<currentTiddler>tags[]sort[]]]"

- sort by number of tagged tiddlers: "[....nsort-count-filter[<currentTiddler>tagging[]]]"

The idea would be that the argument of "sort-list-filter" would be computed for each tiddler resulting from the previous part of the filter, and the result would act like a temporary key on which the overall sorting is based. In the case of "nsort-count-filter", the sub-filter is interpreted as a <$count ...> filter instead of a <$list ...> one. It is important to notice that the variable <currentTiddler> would have to be interpreted inside the loop, i.e. applies to each resulting tiddler and not the current tiddler in which the whole filter is used.

What do you experts think?

ps: btw, does anyone know if there is a difference in performance between using [all[current]...] and [<currentTiddler>...] ?


Regards,
Erwan

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/6eceb037-a36d-429d-be81-ff5d2b5355c9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tobias Beer]

Hi Erwan,

 

Steve I think that your use case is interesting because it shows a limitation of sorting with TW filters (although imho in this specific case the best solution would be to sort the list of tags automatically every time a tag is added to a tiddler).

I think TiddlyWiki should *never* sort a list-field when adding or removing an item, unless specifically instructed to do so via something like Matabele's listops, the tags field being just a special case of a generic list-field.

 

I think another case with the same limitation is when one wants to sort a list of tiddlers by the number of times they are used as tags (i.e. number of tiddlers tagged with them). 

...

I wonder if it would be possible to implement new "sort" filter operators which would allow the usage of a sub-filter like this 

This sounds to me that -- for the purposes of sorting -- we wish for something like a "virtual field", by which we can sort after we've created it "in-memory". A sortby filter could "sort by the output of filter expression evaluated against an item". This filter expression could be passed to the operand as either a variable or text reference as we'd have no other way to chuck a "subfilter" within a filter expression, e.g.

<$vars sortfilter="[<currentTiddler>tagging[]]">
<<list-links filter:"[all[]sortby<sortfilter>]">>
</$vars>

'The above should cater for the OP's requirement, assuming the sortby filter would:

1. evaluate each item in the input set against the specified filter in the operand
   
2. sort the items in the input set with respect to whatever was returned

It should possibly be extendable via filter operator suffix to specify a core sort algorhithm other than the default...

<<list-links filter:"[all[]sortby:nsort<sortfilter>]">>

The count would be:

<$vars sortfilter="[<currentTiddler>tagging[]count[]]">
<<list-links filter:"[tags[]sortby<sortfilter>]">>
</$vars>

...requiring, well, a nonexisting "count" filter.

Best wishes,

Tobias.

[Erwan]

Yes, thank you Tobias for understanding and formalizing my idea!
Should we open an issue about that?

On 06/11/15 16:31, Tobias Beer wrote:

Hi Erwan,

 

Steve I think that your use case is interesting because it shows a limitation of sorting with TW filters (although imho in this specific case the best solution would be to sort the list of tags automatically every time a tag is added to a tiddler).

I think TiddlyWiki should *never* sort a list-field when adding or removing an item, unless specifically instructed to do so via something like Matabele's listops, the tags field being just a special case of a generic list-field.

 

I think another case with the same limitation is when one wants to sort a list of tiddlers by the number of times they are used as tags (i.e. number of tiddlers tagged with them). 

...

I wonder if it would be possible to implement new "sort" filter operators which would allow the usage of a sub-filter like this 

This sounds to me that -- for the purposes of sorting -- we wish for something like a "virtual field", by which we can sort after we've created it "in-memory". A sortby filter could "sort by the output of filter expression evaluated against an item". This filter expression could be passed to the operand as either a variable or text reference as we'd have no other way to chuck a "subfilter" within a filter expression, e.g.

<$vars sortfilter="[<currentTiddler>tagging[]]">
<<list-links filter:"[all[]sortby<sortfilter>]">>
</$vars>

'The above should cater for the OP's requirement, assuming the sortby filter would:

1. evaluate each item in the input set against the specified filter in the operand
   
2. sort the items in the input set with respect to whatever was returned

It could / perhaps should even be...

<<list-links filter:"[all[]sortby:nsort<sortfilter>]">>

...to avail any core sorting algorithms.

The count would be:

<$vars sortfilter="[<currentTiddler>tagging[]count[]]">
<<list-links filter:"[all[]sortby<sortfilter>]">>
</$vars>

...requiring, well, a nonexisting "count" filter.

Best wishes,

Tobias.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/0fd70eb7-1315-4e10-aff7-37ba9bd6af23%40googlegroups.com.

[Tobias Beer]

Hi Erwan,

 

Yes, thank you Tobias for understanding and formalizing my idea!
Should we open an issue about that?

I think we should, but let's go one more round beforehand, perhaps.

Now I'm beginning to think that it should not be a filter of its own but rather a flag (/suffix) to any available sort filter that instructs it to sort by the evaluated operand, e.g.:

{{{ [sort:eval<sortfilter>] }}}

In other words:

Sort the input elements by the output of evaluating the specified operand as a filter expression against each item in turn. This should be implemented by some global interface that does the evaluating that all sort operations use rather than each sort operand implementing that individually.

So, we have a map of key=>value pairs with the keys being the items (by default tiddler titles) and the values being the things to sort by, by default also tiddler title but possibly something else, e.g. some field or "an evaluated filter expression".

Equivalently:

{{{ [sortcs:eval<sortfilter>] }}}

etc...

Not sure if an "evaluated operand" would be relevant for other filter operations other than sorting and whether we could / should devise a syntax that always does evaluation, no matter what filter operator.

Best wishes,

— tb

[steve]

Hi all:

Jed: The button solution you suggested work fine (see listing below).

Erwan and Tobias: Thanks for your posts about (what I interpret to be) a more generalized approach for addressing my use case.

Steve Wharton
```

<$button>Sort Tags
<$list filter='[!is[system]]'>
<$set name=TagsString filter='[<currentTiddler>tags[]sort[]]'>
<$action-setfield tags=<<TagsString>>/>
</$set>
</$list>
</$button>

<table>
<$list filter="[!is[system]tag[BC]sort[tags]]">
<$set name="sorted_tags" filter="[is[current]tags[]sort[]]">

<tr>
<td><$link to={{!!title}}><$view field="title"/></$link></td>
<td><$list filter="[all[current]tags[]sort[title]]" template="$:/core/ui/TagTemplate" storyview="pop"/></td>
</tr>

</$set>
</$list>
</table>
```

[Tobias Beer]

Hi steve,

 

Jed: The button solution you suggested work fine (see listing below).

The thing is, it works, but it also modifies the wiki just so you get the items sorted the way you want them to. Imho, sorting should work without modifying the store, and thus without pushing a button. Sure, practically speaking, this works for now so you can test the practicality of your approach.

Best wishes,

— tb

[steve]

Tobias

I quite agree. My approach is simply expedient and not entirely satisfying. One disadvantage of this approach
is that the recent view gets filled up with all the tiddlers whose tags were sorted. I spent a few hours today
looking for alternatives and didn't find any so far. I did learn about  fields and buttons. I'll keep looking and reading.

I appreciate the interest and responsiveness of all the folks who respond to questions on this wiki.

STeve

[Erwan]

Hi Tobias,

this is even better indeed: for starters this avoids multiplying the number of operators variants (which could cause confusion), and I get a vague idea that this can open even more possibilities of combinations... even though I can't think of any actual use case right now...

Cheers
Erwan

On 07/11/15 09:41, Tobias Beer wrote:

Hi Erwan,

 

Yes, thank you Tobias for understanding and formalizing my idea!
Should we open an issue about that?

I think we should, but let's go one more round beforehand, perhaps.

Now I'm beginning to think that it should not be a filter of its own but rather a flag (/suffix) to any available sort filter that instructs it to sort by the evaluated operand, e.g.:

{{{ [sort:eval<sortfilter>] }}}

In other words:

Sort the input elements by the output of evaluating the specified operand as a filter expression.

Equivalently:

{{{ [sortcs:eval<sortfilter>] }}}

etc...

Not sure if an "evaluated operand" would be relevant for other filter operations other than sorting and whether we could / should devise a syntax that always does evaluation, no matter what filter operator.

Best wishes,

— tb

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/a6d40057-9187-4b10-bb6f-7fb9dc52502f%40googlegroups.com.

[Erwan]

I thought I found a new kind of problem, but when I was writing it I realized that this is just another case that could be solved with "sort:eval" actually. Just mentioning it here anyway, in case that inspires anyone:

I have a list of tiddlers tagged "author" and a list of "wiki" tiddlers, where each of the latter is tagged with an "author" tiddler" (kind of like that). The "wiki" tiddlers have a field "latest-modification". I would like to display a list of "wiki" tiddlers sorted by author, but with the authors sorted by latest modification.

Btw, another thing that I think would be nice is a simple way to display a table with the possibility to sort by any column in any order, for instance like many tables in wikipedia. (sorry, this might be completely unrelated, it just made me think about it)

Erwan

On 07/11/15 09:41, Tobias Beer wrote:

Hi Erwan,

 

Yes, thank you Tobias for understanding and formalizing my idea!
Should we open an issue about that?

I think we should, but let's go one more round beforehand, perhaps.

Now I'm beginning to think that it should not be a filter of its own but rather a flag (/suffix) to any available sort filter that instructs it to sort by the evaluated operand, e.g.:

{{{ [sort:eval<sortfilter>] }}}

In other words:

Sort the input elements by the output of evaluating the specified operand as a filter expression.

Equivalently:

{{{ [sortcs:eval<sortfilter>] }}}

etc...

Not sure if an "evaluated operand" would be relevant for other filter operations other than sorting and whether we could / should devise a syntax that always does evaluation, no matter what filter operator.

Best wishes,

— tb

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/a6d40057-9187-4b10-bb6f-7fb9dc52502f%40googlegroups.com.

[Tobias Beer]

Hi Erwan,

I created a new issue:

#2063 sort by evaluated filter expression / virtual fields

https://github.com/Jermolene/TiddlyWiki5/issues/2063

...adding the following to the idea:

Caching

Seeing as how we may want caching for the output of such computed (operand) expressions, perhaps instead we may want virtual fields and a way to use text-references to read them, with some place to define for each virtual field:

• a virtual field name
  
• a filter expression specifying how the virtual field, e.g. count, is computed

Best wishes,

— tb