Count[], sum[] etc. in nested lists?

[0 0]

Hello!

To generate a list of overdue recurrent tiddlers I'm currently using a nested list so I can compare for each tiddler whether they haven't been interacted with in the period defined for each tiddler (frequency). Something like this:

<$list filter="[has[frequency]]">
  <$list filter="[all[current]!days:last-contact{!!frequency}]">
    <$link><$view field="title"/></$link>
  </$list>
</$list>

Now due to the second list loop handling only a single title at a time, I'm unable to extract any statistical insights out of this, such as count[] -ing all the overdue tiddlers to calculate their percentage or getting the sum[] of a field value the tiddlers share (e.g. time requirement).

Could the overdue filtering process be handled with a single list loop in any way?
Or is it perhaps possible to set a variable inside the inner list loop that would additively store the value for each item inside the inner list? Could you suggest any other way to approach this problem?

Thanks a lot

0

[Charlie Veniot]

G'day,

Just focusing on this bit of your code:

<$list filter="[all[current]!days:last-contact{!!frequency}]">

    <$link><$view field="title"/></$link>

  </$list>

Let's do this:

<$list variable="thisTiddler" filter="[all[current]!days:last-contact{!!frequency}]">

    <$link><$view field="title"/></$link>
  </$list>

Now you can use this variable to do all sorts of things inside there.  For example:

<$list variable="thisTiddler" filter="[all[current]!days:last-contact{!!frequency}]">
    <$link><$view field="title"/></$link>: <$text text={{{ [regexp<thisTiddler>count[]] }}}/>
  </$list>

Please note: I've just typed all of the above here without testing/trying the code in a TiddlyWiki.  So let's say that is just there for discussion's sake.

[Charlie Veniot]

Aside: we could merge the two <$list> filters together as a fun exercise to reduce the amount of code, but there is nothing wrong with having the two there as an explicit note to yourself of what you're thinking and how you got there.

Reducing code/steps is a silly thing to do when those steps are a visual/explicit reminder or something (i.e. a trail of breadcrumbs!)

[Eric Shulman]

On Monday, September 13, 2021 at 6:51:14 AM UTC-7 cj.v...@gmail.com wrote:

Aside: we could merge the two <$list> filters together

That wouldn't work in this case, as the inner filter uses `{!!frequency}`, which depends upon the outer filter to set the `currentTiddler` value to each tiddler that `has[frequency]`

If the two filters were merged, then `{!!frequency}` would refer to the tiddler that contains the `$list` widget, rather than each tiddler that `has[frequency]`

-e

[Charlie Veniot]

Oh yeah, oops.  Scatter-brained.

[0 0]

Thank you for your answer, I appreciate your help.

However, your suggestion does not seem to affect the underlying problem: I'm unable to get the number of tiddlers that pass the filter of second list.

I'll try to elaborate what I meant in my original post.

For example let's say the outer list alone would output 100 tiddlers. If I added count[] to this filter to get the number of tiddlers this outer list outputs I will see the number 100 instead of 100 individual titles.

Then the inner list filters out 30 tiddlers that do not satisfy the additional condition. I see 70 tiddler titles that pass the outer AND inner filter, but I'm unable to count them (except by hand).

Adding count[] to the inner list would just yield a mix of 30 zeroes (0 0 0 0...for each time the currently evaluated tiddler does not pass the inner filter) and 70 ones (1 1 1 1...whenever it does).

Now either I'd need some way to have a variable that would increase each time a tiddler passes the inner filter, or I'd need to have it all in a single-level list (which with current setup doesn't work exactly for the reason Eric describes). Or there could be a completely different approach that I'm unable to see.

0

[Charlie Veniot]

<$list filter="[has[frequency]]">

<$vars theCount={{{ [all[current]!days:last-contact{!!frequency}count[]] }}} >

  <$list filter="[all[current]!days:last-contact{!!frequency}]">
    <$link><$view field="title"/></$link>
  </$list>

</$vars>
</$list>

[Charlie Veniot]

Well, in the middle of doing something else, so not quite sure if I have the right count in the highlighted filter.

[Charlie Veniot]

This may seem silly, but a mock-up screenshot of expected output would be pretty awesome. Information with labels of what the information means.

My last sample I posted should have had the question "something like this?", because it definitely wasn't any kind of "this is the answer you're looking for.'

[0 0]

<$list filter="[has[frequency]]"> I have 10 recurring tasks
  <$list filter="[all[current]!days:last-contact{!!frequency}]"> 7 tasks haven't been completed in {{!!frequency}} days. last-contact contains the timestamp of latest completion.

    <$link><$view field="title"/></$link>
  </$list>

</$list>

As expected the output is 7 tiddler titles:

Task1

Task2

Task4

Task5

Task8

Task9

Task10

After this list I want:

You have [7] tasks.

___________________

<$vars theCount={{{ [all[current]!days:last-contact{!!frequency}count[]] }}} > does not seem to work.

"[all[current]..." contains only one title at a time which then either passes the !days filter or not. Thus after count[], <<theCount>> variable will contain a 1 or a 0.

I'm afraid there is no solution at my level of competence :/

[Charlie Veniot]

 Hi, give this a spin:

<$list filter="[has[frequency]]">

<$vars theCount={{{ [all[current]!days:last-contact{!!frequency}count[]] }}} >

  <$list filter="[all[current]!days:last-contact{!!frequency}]">
    <$link><$view field="title"/></$link>
  </$list>

You have <$text text=<<theCount>>/>

</$vars>

</$list>

Not being convinced I have my vars set right, this as an alternative:

<$vars theCount={{{ [all[current]!days:last-contact{!!frequency}] +[count[]] }}} >

[0 0]

Thanks, but still no luck (the syntax didn't matter either).

Result looks something like this (I add some arbitrary linebreaks here to make the logic clearer):

Task1 You have 1

Task2 You have 1

You have 0

Task 4 You have 1

Task 5 You have 1

You have 0

You have 0

Task 8 You have 1
Task 9 You have 1

Task 10 You have 1

[Charlie Veniot]

I'm getting thrown off by "days:last-contact{!!frequency}" or I'm not imagining the input tiddlers right, or both.

If you have the time to export tiddlers for this scenario, I can drop them into tiddlywiki.com, and figure it all out.

Or somebody else may know exactly what's going on based on the good stuff you've given us so far.

[Mark S.]

This is trickier than it looks, because of the binary nature of the output from days.

One way to handle it might be to set up a recursive routine. Recursive routines can add numbers. But it's really messy.

Another way is to put the entire nested listed set into a text field of a wikify widget. Then render and enlist the output. Sometimes there are additional complications if your output tiddlers (tasks) have spaces in them.

This is too much to pursue on my own without test data. 

Good luck!

[Mark S.]

Oh! I just remembered. You might be able to do it in one go with the new reduce operator. So, once again, some data would be helpful.

[Eric Shulman]

On Monday, September 13, 2021 at 2:18:59 PM UTC-7 Mark S. wrote:

Another way is to put the entire nested listed set into a text field of a wikify widget. Then render and enlist the output. Sometimes there are additional complications if your output tiddlers (tasks) have spaces in them.

As Mark suggests, using the $wikify widget may be the most straightforward way to handle your needs...

Try this:

```

\define getList()

<$list filter="[has[frequency]]">

   <$list filter="[all[current]!days:last-contact{!!frequency}]">

      //[//[<$link/>]//]//

   </$list>

</$list>

\end

<$wikify name="theList" text=<<getList>>>

<$vars total={{{ [has[frequency]count[]] }}}>

<$vars todo={{{ [enlist<theList>count[]] }}}>

<$vars done={{{ [<total>subtract<todo>divide<total>multiply[100]addsuffix[%]] }}}>

<$vars time={{{ [enlist<theList>get[time-needed]sum[]] }}}>

<<total>> tasks, <<percent>> are completed:

<ol><$list filter="[enlist<theList>]"><li><$link/></li></$list></ol>

Total time needed: <<time>> minutes.

</$vars>

</$vars>

</$vars>

</$vars>

</$wikify>

```

Notes:

* The `getList()` macro outputs a list of incomplete tasks (your original nested filters)

* The list items are wrapped in `[[` and `]]`, using italics syntax to separate the brackets so they will be output as literal text.  This handles tiddler titles that contain spaces.

* `$wikify` invokes `getList()` and captures the output as plain text, stored in `theList`

* Next, we calculate the some statistics: total=number of tasks,  todo=number of incomplete tasks, done=percentage complete, time=total time need to complete the remaining tasks

* Then, we display the total and percent complete

* Followed by a numbered bullet list of links to incomplete tasks

* and finally, the total time needed to complete those tasks

That about covers it.

enjoy,

-e

[Eric Shulman]

errata: I changed a variable name, but missed one line.  Change this:

<<total>> tasks, <<percent>> are completed:

to this:

<<total>> tasks, <<done>> are completed:

-e

[0 0]

Yes! Wikifying and enlisting does the trick. Thank you Mark for the initial idea and Eric for the tidy solution with all the features ready baked! This technique will be a valuable addition to my toolbelt.

0

[Charlie Veniot]

Although I figure most folk will understand Eric's nice solution, I'll throw mine here anyway, for the giggles.

If anything, you'll see clues that this old sponge of mine operates pretty much way out in left field, or the far side ...

Assuming that  the negative sign is included with the number in every frequency field ...

What we have here, folks, is a dynamically created filter, which totally gets my geek mojo going:

\define bL2() [[

\define midPart() ]!days:last-contact{

\define lastPart() !!frequency}]

\define theCount() +[count[]

! The result

<$vars thisFilter={{{ [has[frequency]sort[]addsuffix[::]] [has[frequency]sort[]addsuffix[,]] +[sort[]] +[join[]] +[split[::]] +[butlast[]] +[addprefix<bL2>] +[search-replace[,],<midPart>] +[addsuffix<lastPart>] +[join[ ]] }}}>

Count of items: <$count filter=<<thisFilter>>/>

<$list filter=<<thisFilter>>>

</$list>

</$vars>

[Charlie Veniot]

Oh yeah, forgot to attach sample related tiddlers in case anybody wants to dive into that and play.

[Charlie Veniot]

Arg.  Please ignore the macro "theCount".  That was part of an initial plan that I tossed aside.

[0 0]

Thank you CJ, this solution indeed works as well! I enjoyed following along step by step the construction of the final filter and feel that your idea of "duplicating" each title at the beginning will be another useful technique that I can utilize when needed.

0

[Charlie Veniot]

Well, do love, LOVE, filtering in TiddlyWiki.

That, I think, comes from my love for SQL (in particular Oracle SQL), ever since being taught "relational algebra" at university many many many moons ago.

One would think I would be as passionate about APL, but I could not stand all of the glyphs (I have a hard time figuring out elevator open/close door buttons.)

The beauty of filtering in TiddlyWiki, just like writing SQL, is the idea of transformations from start to finish: tweak tweak tweak tweak.

Sometimes, the transformations are helped by adding more bulk.

So much fun.

BTW, I'm some glad you started this thread.  I was not aware of that {tiddler!!field} beauty in filtering.  Or saw it before and just did not clue in.  That is good stuff.  Thanks for showing that and your use case !