Sorting a list by the quotient of two fields?

[si]

Hi everyone.

I'm trying to implement a particular type of task list using plain wikitext. I stole the idea from supermemo.

Each of my tasks is tagged task, and has the fields time and value. These fields represent the estimated time a task will take and the perceived value of a task respectively.

I'm trying to generate a task list that is sorted by the result of value divided by time for each task.

I thought this would be able to do this easily, then tried it and realised I couldn't. Can anyone help me out here?

[Mat]

I'm curious about this too, so I have no good answer. It is problematic because you don't have access to both the title and the calculated value at the same time in the listing. It's seems only possible to either sort by a directly accessible value or you already sorted the titles and calculate the value after. 

I have previously made a proposal for a filter operator that, mid-filter, stores a temporary value only to have it recalled later in the filter or in the succeeding listwidget content. (I will not link to it here because the proposal is flawed and need to rephrase it.)

I said I have no good answer. Here is a bad one:

Calculate the values and prepend them to the titles (e.g 7.53_MyTiddler). Sort by these new titles and display the names by splitting up the names to get the original name again. Not only do I not like this approach but I also don't know how all these new titles are to be temporarily stored. Maybe they could be appended into some physical list field? (That would be interesting, actually... but it's an "action" so it would have to be manually fired perhaps via some button click...)

<:-)

[Jeremy Ruston]

I've wondered about adding a variant of the sort operator that would take a subfilter and sort the incoming titles by the result of running that filter on each one in turn. So if the subfilter were [removeprefix[$:/]] then tiddlers would be sorted as A, $:/B, C (ie, system tiddlers would be interleaved with ordinary tiddlers).

So, the OP would then be possible with

<$set name="sortfilter" value="[{!!value}divide{!!time}]">

<$list filter="[tag[task]sort-by<sortfilter>]">

...

etc.

Performance might be an issue when sorting large numbers of items, but it would open up a lot of flexibility.

Best wishes

Jeremy

--

Jeremy Ruston

jer...@jermolene.com

https://jermolene.com

On 22 Apr 2020, at 22:12, Mat <matia...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/2a1b2d50-e081-49fa-8d56-20387993dcb8%40googlegroups.com.

[Jeremy Ruston]

I've wondered about adding a variant of the sort operator that would take a subfilter and sort the incoming titles by the result of running that filter on each one in turn. So if the subfilter were [removeprefix[$:/]] then tiddlers would be sorted as A, $:/B, C (ie, system tiddlers would be interleaved with ordinary tiddlers).

So, the OP would then be possible with

<$set name="sortfilter" value="[{!!value}divide{!!time}]">

<$list filter="[tag[task]sort-by<sortfilter>]">

...

etc.

Performance might be an issue when sorting large numbers of items, but it would open up a lot of flexibility.

Best wishes

Jeremy

--

Jeremy Ruston

jer...@jermolene.com

https://jermolene.com

On 22 Apr 2020, at 22:12, Mat <matia...@gmail.com> wrote:



--

[Saq Imtiaz]

Jeremy: I've wanted that exact thing only about 6 times in the last month alone, so a big plus +1.

[Mat]

Jeremy Ruston wrote:

I've wondered about adding a variant of the sort operator that would take a subfilter and [...]

Why, specifically, a sort operator? If this functionality was possible would think an unlimited number of use cases for "submanipulations" should come up. 

<:-)

 

[Jeremy Ruston]

Why, specifically, a sort operator?

Because that was what was asked.

If this functionality was possible would think an unlimited number of use cases for "submanipulations" should come up. 

We already support the "subfilter" operator which uses subfilters.

There may well be other use cases beyond filtering and sorting.

Best wishes

Jeremy

<:-)

 

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/18c91157-d57c-4ae1-b5b0-c7121c00dfb7%40googlegroups.com.

[Eric Shulman]

I did something similar to create a "tag cloud", where the tags are sorted by the number of tiddlers they are tagging.

See http://tiddlytools.com/filtergenerators.html#TiddlyTools%2FFilterGenerators%2FTagCloud

For your purposes, the code is actually quite a bit simpler than the TagCloud.  This should do the job:

\define getitems()
<$list filter="[tag[task]]">
   <$vars time={{!!time}} value={{!!value}}>
   <$text text="[["/>{{{ [<value>divide<time>divide[10000]] }}};<<currentTiddler>><$text text="]]"/>

   </$vars>
</$list>
\end

<$wikify name="items"  text=<<getitems>>  >
   <$list filter="[enlist<items>nsort[]]">
      {{{ [<currentTiddler>split[;]last[]] }}}<br>
   </$list>
</$wikify>

The <<getitems>> macro:

* gets all tiddlers tagged with "task"

* calculates the "value/time" number for each and divides it by 10000 to ensure that all numbers are zero-padded for proper sorting (so a value, e.g., "15" is not sorted between "1" and "2")

* prepends that calculated value in front of each task title, separated by a semi-colon (;) and surrounded by double square brackets to handle titles with spaces

The calling code:

* Uses $wikify to capture the output of <<getitems>> into a variable, "items"

* The $list widget then enlists the items and sorts them numerically

* The output then strips off the calculated number and the semi-colon separator and displays the tiddler titles in the sorted order

Let me know how it goes...

enjoy,

-e

[TonyM]

Folks,

I am all for trying a sort filter that helps if we could say sortby<key>

Where key may be {{!!fielda}}{{!!fieldb}} or any other macro concatenation it would be nice:

However I also see the value in providing a method to simply pump a data tiddler with an index of {{!!fielda}}{{!!fieldb}}

with the value of each record the tiddler title. You can then list all tiddler title in the date tidler in index order.

The issue is you need to trigger building the index before you list it. You could have a manual button "refresh index"

This is an example of why I would love to have an on-open actions field containing say <<reindex>> which contains the actions to refresh the index. Then the tiddler would list the refreshed list. This requires that the navigation actions query if there is an "on open action" on opening any tiddler.

Regards

Tony

[si]

Thank you so much Eric you have saved me yet again! I really appreciate you taking the time to help me out.

[Jeremy Ruston]

I’ve now implemented a new “subsort” filter that takes a subfilter and sorts the incoming titles by the result of running that filter on each one in turn:

https://github.com/Jermolene/TiddlyWiki5/commit/619c0752bd3c6e71d1fcdb74daa03cfe8257afe4

The OP should be achievable using a subfilter such as:

[{!!value}divide{!!time}]

With that filter in a variable called “cost” it could be used like this:

[tag[tas]subsort:number<cost>]

The new operator will be available on https://tiddlywiki.com/prelease in a few minutes (you can check on build progress here: https://travis-ci.org/github/Jermolene/TiddlyWiki5/builds)

Best wishes

Jeremy

-- 
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/37d4d40b-ce4f-44ad-b2a0-8a4217ebf502%40googlegroups.com.

[Saq Imtiaz]

Brilliant! Thanks Jeremy. I had put that on my to-do list but wasn't sure when I would get around to it.

[Mohammad]

Great feature!
Thank you Jeremy

On Saturday, April 25, 2020 at 3:02:57 PM UTC+4:30, Jeremy Ruston wrote:

I’ve now implemented a new “subsort” filter that takes a subfilter and sorts the incoming titles by the result of running that filter on each one in turn:

https://github.com/Jermolene/TiddlyWiki5/commit/619c0752bd3c6e71d1fcdb74daa03cfe8257afe4

The OP should be achievable using a subfilter such as:

[{!!value}divide{!!time}]

With that filter in a variable called “cost” it could be used like this:

[tag[tas]subsort:number<cost>]

Seems it is sortsub

Using tiddlywiki.com I made a test as below 

\define cost() [{!!value}divide{!!time}]
<$list filter="[tag[job]sortsub:number<cost>]">

</$list>

but it does not work! Please advise!

To unsubscribe from this group and stop receiving emails from it, send an email to tiddl...@googlegroups.com.

[Jeremy Ruston]

Hi Mohammad

Seems it is sortsub

It turns out there is already a “sortby” operator.

Using tiddlywiki.com I made a test as below 

\define cost() [{!!value}divide{!!time}]
<$list filter="[tag[job]sortsub:number<cost>]">

</$list>

but it does not work! Please advise!

Did you try it on the prerelease? It looks like you’d need some tiddlers tagged “job” with the right fields for it to work.

Best wishes

Jeremy.

[Mohammad]

Yes Jeremy,

I did this in Prerelease with some tiddlers having job as a tag and time and value fields. I will send an attachment within a minute.

--Mohammad

[Mohammad]

Hi again Jeremy,

 I have attached the sample tiddlers tested on https://tiddlywiki.com/prerelease/

 using the new sortsub

--Mohammad

[Jeremy Ruston]

Hi Mohammad,

Thank you, that’s helpful. I’m seeing the result Designer, Doctor, Engineer, is that not what you are seeing?

Best wishes

Jeremy

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/f1c1af1d-9575-40f5-9081-22142c4ce070%40googlegroups.com.
<test-new-sortsub.json>

[Mohammad]

Strangely NO! 

I see Engineer, Doctor, Designer. I testing online on https://tiddlywiki.com/prerelease/

On Saturday, April 25, 2020 at 5:30:51 PM UTC+4:30, Jeremy Ruston wrote:

Hi Mohammad,

Thank you, that’s helpful. I’m seeing the result Designer, Doctor, Engineer, is that not what you are seeing?

Best wishes

Jeremy

On 25 Apr 2020, at 13:58, Mohammad <mohamma...@gmail.com> wrote:

Hi again Jeremy,

 I have attached the sample tiddlers tested on https://tiddlywiki.com/prerelease/

 using the new sortsub

--Mohammad

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddl...@googlegroups.com.

[Jeremy Ruston]

Hi Mohammad

The reason we were getting different results is because I had built the index.html I was using on my Mac, whereas you were using the online version that had been built on servers run by TravisCI. Those two environments end up putting the tiddlers into the HTML file in a different order, which was the ordering we were seeing.

The problem is my fault: I gave the expression [get{!!value}divide{!!time}] without testing it, and it won’t work in this context because it relies on the current tiddler. The trouble is that this filter is run once to convert every list item into the corresponding sort key. We can only set the current tiddler to a single value for that run of the filter, so it clearly can’t address every tiddler in the input set.

The solution is to use the get operator to retrieve the field values from each item that is passed into the filter. Hence the examples in the docs such as [get[text]else[]length[]] not being written as [{!!text}length[]].

The problem here is that we need to retrieve both field values, which can’t currently be done. We can do [get[cost]divide[23]] but nothing that will divide [get[cost]] by [get[value]].

So, it turns out that at this point the subsort operator doesn’t actually meet the requirement in the OP. I’ll give it some thought.

Best wishes

Jeremy

On 25 Apr 2020, at 14:13, Mohammad <mohammad...@gmail.com> wrote:

Strangely NO! 

I see Engineer, Doctor, Designer. I testing online on https://tiddlywiki.com/prerelease/

<Slide-37.png>

On Saturday, April 25, 2020 at 5:30:51 PM UTC+4:30, Jeremy Ruston wrote:

Hi Mohammad,

Thank you, that’s helpful. I’m seeing the result Designer, Doctor, Engineer, is that not what you are seeing?

Best wishes

Jeremy

On 25 Apr 2020, at 13:58, Mohammad <mohamma...@gmail.com> wrote:

Hi again Jeremy,

 I have attached the sample tiddlers tested on https://tiddlywiki.com/prerelease/

 using the new sortsub

--Mohammad

-- 
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/f1c1af1d-9575-40f5-9081-22142c4ce070%40googlegroups.com.
<test-new-sortsub.json>

-- 
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/559e5dee-7a42-4dfd-a688-d71f8f3b1770%40googlegroups.com.
<Slide-37.png>

[Mohammad]

Jeremy,

 Many thanks for clarification! Got the point!

Cheers

Mohammad

On Saturday, April 25, 2020 at 6:03:50 PM UTC+4:30, Jeremy Ruston wrote:

Hi Mohammad

The reason we were getting different results is because I had built the index.html I was using on my Mac, whereas you were using the online version that had been built on servers run by TravisCI. Those two environments end up putting the tiddlers into the HTML file in a different order, which was the ordering we were seeing.

The problem is my fault: I gave the expression [get{!!value}divide{!!time}] without testing it, and it won’t work in this context because it relies on the current tiddler. The trouble is that this filter is run once to convert every list item into the corresponding sort key. We can only set the current tiddler to a single value for that run of the filter, so it clearly can’t address every tiddler in the input set.

The solution is to use the get operator to retrieve the field values from each item that is passed into the filter. Hence the examples in the docs such as [get[text]else[]length[]] not being written as [{!!text}length[]].

The problem here is that we need to retrieve both field values, which can’t currently be done. We can do [get[cost]divide[23]] but nothing that will divide [get[cost]] by [get[value]].

So, it turns out that at this point the subsort operator doesn’t actually meet the requirement in the OP. I’ll give it some thought.

Best wishes

Jeremy

On 25 Apr 2020, at 14:13, Mohammad <mohamma...@gmail.com> wrote:

Strangely NO! 

I see Engineer, Doctor, Designer. I testing online on https://tiddlywiki.com/prerelease/

<Slide-37.png>

On Saturday, April 25, 2020 at 5:30:51 PM UTC+4:30, Jeremy Ruston wrote:

Hi Mohammad,

Thank you, that’s helpful. I’m seeing the result Designer, Doctor, Engineer, is that not what you are seeing?

Best wishes

Jeremy

On 25 Apr 2020, at 13:58, Mohammad <mohamma...@gmail.com> wrote:

Hi again Jeremy,

 I have attached the sample tiddlers tested on https://tiddlywiki.com/prerelease/

 using the new sortsub

--Mohammad

-- 
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddl...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/f1c1af1d-9575-40f5-9081-22142c4ce070%40googlegroups.com.
<test-new-sortsub.json>

-- 
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddl...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/559e5dee-7a42-4dfd-a688-d71f8f3b1770%40googlegroups.com.
<Slide-37.png>

[Jeremy Ruston]

Hi Mohammad,

I’ve pushed an update that switches to evaluating the subfilter once for each input item, setting the currentTiddler variable for each evaluation. This makes the expression [{!!value}divide{!!cost}] work properly. It’s a bit slower, but definitely more useful.

https://github.com/Jermolene/TiddlyWiki5/commit/ad575efdcc4d03f9d09bf72defe0e8146164d34f

The update will be available on https://tiddlywiki.com/prelease in a few minutes (you can check on build progress here: https://travis-ci.org/github/Jermolene/TiddlyWiki5/builds)

Best wishes

Jeremy

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/ec65706c-bffe-44ba-aa73-be609f65c20a%40googlegroups.com.

[Mohammad]

Hi Jeremy,

 I confirm the test example now works on https://tiddlywiki.com/prerelease/

Many thanks.

Cheers

Mohammad

On Saturday, April 25, 2020 at 6:54:29 PM UTC+4:30, Jeremy Ruston wrote:

Hi Mohammad,

I’ve pushed an update that switches to evaluating the subfilter once for each input item, setting the currentTiddler variable for each evaluation. This makes the expression [{!!value}divide{!!cost}] work properly. It’s a bit slower, but definitely more useful.

https://github.com/Jermolene/TiddlyWiki5/commit/ad575efdcc4d03f9d09bf72defe0e8146164d34f

The update will be available on https://tiddlywiki.com/prelease in a few minutes (you can check on build progress here: https://travis-ci.org/github/Jermolene/TiddlyWiki5/builds)

Best wishes

Jeremy

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/ec65706c-bffe-44ba-aa73-be609f65c20a%40googlegroups.com.