Count[], sum[] etc. in nested lists?

233 views
Skip to first unread message

0 0

unread,
Sep 13, 2021, 5:25:03 AMSep 13
to TiddlyWiki
Hello!
To generate a list of overdue recurrent tiddlers I'm currently using a nested list so I can compare for each tiddler whether they haven't been interacted with in the period defined for each tiddler (frequency). Something like this:

<$list filter="[has[frequency]]">
  <$list filter="[all[current]!days:last-contact{!!frequency}]">
    <$link><$view field="title"/></$link>
  </$list>
</$list>

Now due to the second list loop handling only a single title at a time, I'm unable to extract any statistical insights out of this, such as count[] -ing all the overdue tiddlers to calculate their percentage or getting the sum[] of a field value the tiddlers share (e.g. time requirement).

Could the overdue filtering process be handled with a single list loop in any way?
Or is it perhaps possible to set a variable inside the inner list loop that would additively store the value for each item inside the inner list? Could you suggest any other way to approach this problem?

Thanks a lot
0

Charlie Veniot

unread,
Sep 13, 2021, 9:46:50 AMSep 13
to TiddlyWiki
G'day,

Just focusing on this bit of your code:

<$list filter="[all[current]!days:last-contact{!!frequency}]">
    <$link><$view field="title"/></$link>
  </$list>

Let's do this:

<$list variable="thisTiddler" filter="[all[current]!days:last-contact{!!frequency}]">

    <$link><$view field="title"/></$link>
  </$list>


Now you can use this variable to do all sorts of things inside there.  For example:

<$list variable="thisTiddler" filter="[all[current]!days:last-contact{!!frequency}]">
    <$link><$view field="title"/></$link>: <$text text={{{ [regexp<thisTiddler>count[]] }}}/>
  </$list>


Please note: I've just typed all of the above here without testing/trying the code in a TiddlyWiki.  So let's say that is just there for discussion's sake.

Charlie Veniot

unread,
Sep 13, 2021, 9:51:14 AMSep 13
to TiddlyWiki
Aside: we could merge the two <$list> filters together as a fun exercise to reduce the amount of code, but there is nothing wrong with having the two there as an explicit note to yourself of what you're thinking and how you got there.

Reducing code/steps is a silly thing to do when those steps are a visual/explicit reminder or something (i.e. a trail of breadcrumbs!)

Eric Shulman

unread,
Sep 13, 2021, 11:02:58 AMSep 13
to TiddlyWiki
On Monday, September 13, 2021 at 6:51:14 AM UTC-7 cj.v...@gmail.com wrote:
Aside: we could merge the two <$list> filters together

That wouldn't work in this case, as the inner filter uses `{!!frequency}`, which depends upon the outer filter to set the `currentTiddler` value to each tiddler that `has[frequency]`
If the two filters were merged, then `{!!frequency}` would refer to the tiddler that contains the `$list` widget, rather than each tiddler that `has[frequency]`

-e

Charlie Veniot

unread,
Sep 13, 2021, 11:09:57 AMSep 13
to TiddlyWiki
Oh yeah, oops.  Scatter-brained.

0 0

unread,
Sep 13, 2021, 11:12:45 AMSep 13
to TiddlyWiki
Thank you for your answer, I appreciate your help.
However, your suggestion does not seem to affect the underlying problem: I'm unable to get the number of tiddlers that pass the filter of second list.
I'll try to elaborate what I meant in my original post.

For example let's say the outer list alone would output 100 tiddlers. If I added count[] to this filter to get the number of tiddlers this outer list outputs I will see the number 100 instead of 100 individual titles.
Then the inner list filters out 30 tiddlers that do not satisfy the additional condition. I see 70 tiddler titles that pass the outer AND inner filter, but I'm unable to count them (except by hand).
Adding count[] to the inner list would just yield a mix of 30 zeroes (0 0 0 0...for each time the currently evaluated tiddler does not pass the inner filter) and 70 ones (1 1 1 1...whenever it does).

Now either I'd need some way to have a variable that would increase each time a tiddler passes the inner filter, or I'd need to have it all in a single-level list (which with current setup doesn't work exactly for the reason Eric describes). Or there could be a completely different approach that I'm unable to see.

0

Charlie Veniot

unread,
Sep 13, 2021, 11:18:45 AMSep 13
to TiddlyWiki
<$list filter="[has[frequency]]">
<$vars theCount={{{ [all[current]!days:last-contact{!!frequency}count[]] }}} >

  <$list filter="[all[current]!days:last-contact{!!frequency}]">
    <$link><$view field="title"/></$link>
  </$list>
</$vars>
</$list>


Charlie Veniot

unread,
Sep 13, 2021, 11:20:15 AMSep 13
to TiddlyWiki
Well, in the middle of doing something else, so not quite sure if I have the right count in the highlighted filter.

Charlie Veniot

unread,
Sep 13, 2021, 11:36:56 AMSep 13
to TiddlyWiki
This may seem silly, but a mock-up screenshot of expected output would be pretty awesome. Information with labels of what the information means.

My last sample I posted should have had the question "something like this?", because it definitely wasn't any kind of "this is the answer you're looking for.'



0 0

unread,
Sep 13, 2021, 1:25:02 PMSep 13
to TiddlyWiki
<$list filter="[has[frequency]]"> I have 10 recurring tasks
  <$list filter="[all[current]!days:last-contact{!!frequency}]"> 7 tasks haven't been completed in {{!!frequency}} days. last-contact contains the timestamp of latest completion.

    <$link><$view field="title"/></$link>
  </$list>
</$list>

As expected the output is 7 tiddler titles:
Task1
Task2
Task4
Task5
Task8
Task9
Task10

After this list I want:
You have [7] tasks.
___________________
<$vars theCount={{{ [all[current]!days:last-contact{!!frequency}count[]] }}} > does not seem to work.
"[all[current]..." contains only one title at a time which then either passes the !days filter or not. Thus after count[], <<theCount>> variable will contain a 1 or a 0.

I'm afraid there is no solution at my level of competence :/

Charlie Veniot

unread,
Sep 13, 2021, 1:58:04 PMSep 13
to TiddlyWiki
 Hi, give this a spin:

<$list filter="[has[frequency]]">
<$vars theCount={{{ [all[current]!days:last-contact{!!frequency}count[]] }}} >

  <$list filter="[all[current]!days:last-contact{!!frequency}]">
    <$link><$view field="title"/></$link>
  </$list>

You have <$text text=<<theCount>>/>
</$vars>
</$list>

Not being convinced I have my vars set right, this as an alternative:
<$vars theCount={{{ [all[current]!days:last-contact{!!frequency}] +[count[]] }}} >

0 0

unread,
Sep 13, 2021, 2:23:46 PMSep 13
to TiddlyWiki
Thanks, but still no luck (the syntax didn't matter either).
Result looks something like this (I add some arbitrary linebreaks here to make the logic clearer):
Task1 You have 1
Task2 You have 1
You have 0
Task 4 You have 1
Task 5 You have 1
You have 0
You have 0
Task 8 You have 1
Task 9 You have 1
Task 10 You have 1

Charlie Veniot

unread,
Sep 13, 2021, 2:41:22 PMSep 13
to TiddlyWiki
I'm getting thrown off by "days:last-contact{!!frequency}" or I'm not imagining the input tiddlers right, or both.

If you have the time to export tiddlers for this scenario, I can drop them into tiddlywiki.com, and figure it all out.

Or somebody else may know exactly what's going on based on the good stuff you've given us so far.



Mark S.

unread,
Sep 13, 2021, 5:18:59 PMSep 13
to TiddlyWiki
This is trickier than it looks, because of the binary nature of the output from days.

One way to handle it might be to set up a recursive routine. Recursive routines can add numbers. But it's really messy.

Another way is to put the entire nested listed set into a text field of a wikify widget. Then render and enlist the output. Sometimes there are additional complications if your output tiddlers (tasks) have spaces in them.

This is too much to pursue on my own without test data. 

Good luck!


Mark S.

unread,
Sep 13, 2021, 5:33:05 PMSep 13
to TiddlyWiki
Oh! I just remembered. You might be able to do it in one go with the new reduce operator. So, once again, some data would be helpful.

Eric Shulman

unread,
Sep 13, 2021, 6:32:15 PMSep 13
to TiddlyWiki
On Monday, September 13, 2021 at 2:18:59 PM UTC-7 Mark S. wrote:
Another way is to put the entire nested listed set into a text field of a wikify widget. Then render and enlist the output. Sometimes there are additional complications if your output tiddlers (tasks) have spaces in them.

As Mark suggests, using the $wikify widget may be the most straightforward way to handle your needs...

Try this:
```
\define getList()
<$list filter="[has[frequency]]">
   <$list filter="[all[current]!days:last-contact{!!frequency}]">
      //[//[<$link/>]//]//
   </$list>
</$list>
\end

<$wikify name="theList" text=<<getList>>>
<$vars total={{{ [has[frequency]count[]] }}}>
<$vars todo={{{ [enlist<theList>count[]] }}}>
<$vars done={{{ [<total>subtract<todo>divide<total>multiply[100]addsuffix[%]] }}}>
<$vars time={{{ [enlist<theList>get[time-needed]sum[]] }}}>
<<total>> tasks, <<percent>> are completed:
<ol><$list filter="[enlist<theList>]"><li><$link/></li></$list></ol>
Total time needed: <<time>> minutes.
</$vars>
</$vars>
</$vars>
</$vars>
</$wikify>
```

Notes:
* The `getList()` macro outputs a list of incomplete tasks (your original nested filters)
* The list items are wrapped in `[[` and `]]`, using italics syntax to separate the brackets so they will be output as literal text.  This handles tiddler titles that contain spaces.
* `$wikify` invokes `getList()` and captures the output as plain text, stored in `theList`
* Next, we calculate the some statistics: total=number of tasks,  todo=number of incomplete tasks, done=percentage complete, time=total time need to complete the remaining tasks
* Then, we display the total and percent complete
* Followed by a numbered bullet list of links to incomplete tasks
* and finally, the total time needed to complete those tasks

That about covers it.

enjoy,
-e

Eric Shulman

unread,
Sep 13, 2021, 7:09:14 PMSep 13
to TiddlyWiki
errata: I changed a variable name, but missed one line.  Change this:
<<total>> tasks, <<percent>> are completed:
to this:
<<total>> tasks, <<done>> are completed:

-e

0 0

unread,
Sep 13, 2021, 7:25:41 PMSep 13
to TiddlyWiki
Yes! Wikifying and enlisting does the trick. Thank you Mark for the initial idea and Eric for the tidy solution with all the features ready baked! This technique will be a valuable addition to my toolbelt.

0

Charlie Veniot

unread,
Sep 13, 2021, 8:40:32 PMSep 13
to TiddlyWiki
Although I figure most folk will understand Eric's nice solution, I'll throw mine here anyway, for the giggles.

If anything, you'll see clues that this old sponge of mine operates pretty much way out in left field, or the far side ...

Assuming that  the negative sign is included with the number in every frequency field ...

What we have here, folks, is a dynamically created filter, which totally gets my geek mojo going:

\define bL2() [[
\define midPart() ]!days:last-contact{
\define lastPart() !!frequency}]
\define theCount() +[count[]

! The result

<$vars thisFilter={{{ [has[frequency]sort[]addsuffix[::]] [has[frequency]sort[]addsuffix[,]] +[sort[]] +[join[]] +[split[::]] +[butlast[]] +[addprefix<bL2>] +[search-replace[,],<midPart>] +[addsuffix<lastPart>] +[join[ ]] }}}>

Count of items: <$count filter=<<thisFilter>>/>

<$list filter=<<thisFilter>>>

</$list>

</$vars>

Charlie Veniot

unread,
Sep 13, 2021, 8:44:38 PMSep 13
to TiddlyWiki
Oh yeah, forgot to attach sample related tiddlers in case anybody wants to dive into that and play.
FunkyDynamicCreationOfFilter.json

Charlie Veniot

unread,
Sep 13, 2021, 8:47:40 PMSep 13
to TiddlyWiki
Arg.  Please ignore the macro "theCount".  That was part of an initial plan that I tossed aside.

0 0

unread,
Sep 13, 2021, 9:49:15 PMSep 13
to TiddlyWiki
Thank you CJ, this solution indeed works as well! I enjoyed following along step by step the construction of the final filter and feel that your idea of "duplicating" each title at the beginning will be another useful technique that I can utilize when needed.
0

Charlie Veniot

unread,
Sep 13, 2021, 10:03:27 PMSep 13
to TiddlyWiki
Well, do love, LOVE, filtering in TiddlyWiki.

That, I think, comes from my love for SQL (in particular Oracle SQL), ever since being taught "relational algebra" at university many many many moons ago.

One would think I would be as passionate about APL, but I could not stand all of the glyphs (I have a hard time figuring out elevator open/close door buttons.)

The beauty of filtering in TiddlyWiki, just like writing SQL, is the idea of transformations from start to finish: tweak tweak tweak tweak.

Sometimes, the transformations are helped by adding more bulk.

So much fun.

BTW, I'm some glad you started this thread.  I was not aware of that {tiddler!!field} beauty in filtering.  Or saw it before and just did not clue in.  That is good stuff.  Thanks for showing that and your use case !

Reply all
Reply to author
Forward
0 new messages