Count data dictionary values

[Watt]

I'm experimenting with data dictionaries and have one that records runs completed. It has dates 01 to 31 as the index and either 0 or 1 as a value, 1 indicating a run completed on that date. e.g

01:1
02:0
03:1 etc

Runs completed=2

I'd like to count those '1' values in order to get a total of runs completed, but I'm stumped. Does anybody know how to count, add or mathematically manipulate data dictionary values? I'm using the pre-release version. Thanks.

[TonyM]

By data dictionaries do you mean data tiddlers.

Ie an index and value pair.

If so you can treat the value in a data tiddler like the value in a field.

I will try and post more on this later.

I even know how to do in before prerelease.

Regards
Tony

[Watt]

Thanks TonyM, yes it's a data tiddler with index:value pairs.

01:1
02:0
03:1

04:1
05:0
etc.

Any clues on counting the '1' values much appreciated.

[TonyM]

Watt,

I have it working in an inelegant way. It only works if the values are 0 and 1 because summing all ones is also a count of all one's

I will post here without explanation other than unless I use wikify the split[ ]sum[] will not work

\define all-values()

<$list filter="[[Data]indexes[]]" variable=item>

<$view tiddler=Data index=<<item>>/>

</$list>

\end

all-values: <<all-values>>

<$wikify name=string text="<<all-values>>">

String: <<string>>

<$set name=result filter="[<string>split[]sum[]]">

 Set sum Result: <<result>><br>

</$set>

<$set name=result filter="[<string>split[]] +[count[]]">

 Set Count: <<result>><br>

</$set>

 

 <hr>

 

<$list filter="[<string>split[]sum[]]" variable=datum>

 List Result: <<datum>><br>

</$list>

<$list filter="[<string>split[]]" variable=datum>

 List: <<datum>><br>

</$list>

</$wikify>

The result of Set Count: and List: demonstrates it is still not perfect.

I am not so practiced with Data Tiddlers. 

Regards

Tony

[TonyM]

Further improvments

Note Split[ ] contains a space

\define all-values(data-tiddler) <$list filter="[[$data-tiddler$]indexes[]]" variable=item> <$view tiddler=Data index=<<item>>/></$list>

all-values: "<<all-values Data>>"<br>

<$wikify name=string text="<<all-values Data>>">
String: "<<string>>"<br>

<$set name=result filter="[<string>split[ ]sum[]]">
 Set sum Result: <<result>><br>
<

/$set>


<$set name=result filter="[<string>split[ ]count[]]">

 Set Count: <<result>><br>
</

$set>

 
 <hr>
 
<$list filter="[<string>split[ ]sum[]]" variable=datum>
 List Result: <<datum>><br>
<

/$list>


<hr>


<$list filter="[<string>split[ ]]" variable=datum>
 List Item: <<datum>><br>
</$list>


<hr>




<h3>Count exact number of items with value "1" in prefix/suffix</h3>


<$set name=result filter="[<string>split[ ]prefix[1]suffix[1]count[]]">

 Set Count: <<result>><br>
</

$set>


<$set name=result filter="[<string>split[ ]prefix[1]suffix[1]sum[]]">
 Set Sum: <<result>><br>
</$set>


<$list filter="[<string>split[ ]prefix[1]suffix[1]count[]]" variable=result>
 List Result Count: <<result>><br>
</$list>


<$list filter="[<string>split[ ]prefix[1]suffix[1]sum[]]" variable=result>
 List Result Sum: <<result>><br>
</$list>

<h3>Best Method? (with Wikify)</h3>

Use this {{{ [<string>split[ ]prefix[1]suffix[1]count[]] }}}

Or  this <$text text={{{ [<string>split[ ]prefix[1]suffix[1]count[]] }}}/>

If values only 0 or 1 use `{{{ [<string>split[ ]sum[]] }}}` giving {{{ [<string>split[ ]sum[]] }}}

</$wikify>

I believe further improvement is possible.

Regards

Tony

[Watt]

Thank you so much TonyM! Those are very useful solutions which I would never have worked out myself. I'm sure others will find them helpful too - adding and counting data tiddler values must be a common requirement but I couldn't find anything on the forum. My run stats are now reborn. "Life is like a box of chocolates".

[TonyM]

Watt,

Remember this is using sum which is in the pre-release. Be careful using this for production. Be ready to export your content and re import it when the full 5.1.20 release is available.

If you wanted it working in pre 5.1.20 you could use evans formulae plugin or the calc widget (search here or tiddlywiki.com)

Regards

Tony

[Watt]

Will do. All working on the pre-release so far. Thanks again, enjoy that snow.

[PMario]

Hi Folks,

Just had a look at the code. ... It shouldn't be that hard.

I would like to have {{{[[data]values[]sum[]]}}} ... Which should show 2 if the above content is a tiddler named: data.

I do have a new "values" operator working. I'll create a pull request today.

BE AWARE!! The code above doesn't work atm. ... It will work in 5.1.20, if Jeremy merges the pull request.

have fun!

mario

[TonyM]

Mario,

Smiley Face :)

Do you think we could also provide values with other inputs like values<varname> values{!!fieldname} values{tiddlername!!fieldname}  if it could parse this with spaces as the delimiter then numeric lists in a variable will not be deduped and can feed into our maths operators.

Perhaps values:field[fieldname] would also be valid extracting all values in all tiddlers with field name.

My only concern is the operator name "values" is a very generic word and we will have a need for more with the maths operators. 

So given the above perhaps this would be safer. [datatiddler(s)]values:data[]sum[]

[tiddler(s)]values:field[fieldname]sum[]

Regards

Tony

[TonyM]

Also

values[1 2 1 2 3]sum[]

Regards

Tony

[PMario]

On Tuesday, June 4, 2019 at 8:56:02 AM UTC+2, TonyM wrote:

Do you think we could also provide values with other inputs like values<varname> values{!!fieldname} values{tiddlername!!fieldname}  if it could parse this with spaces as the delimiter then numeric lists in a variable will not be deduped and can feed into our maths operators.

I think this is the existing "enlist" operator.

-m

[PMario]

Hi,

Working with data-tiddlers there is the "indexes" and the "getindex" operators. ... BUT they are not enough with the new math functions.

As the OP shows. ... The new "values" operator works similar to the "indexes" operator, but it lists the values instead of the indexes (keys)

Since the both outputs are sorted, I also want to have an "keyvalues" operator, which will allow us to get both elements, and use string manipulation afterwards.

-m

[PMario]

On Tuesday, June 4, 2019 at 8:57:28 AM UTC+2, TonyM wrote:

values[1 2 1 2 3]sum[]

same as indexes. ... no parameters

-m

[PMario]

uups ... missed the "deduped" :)

-m

[Watt]

Thanks for taking this further PMario! I'm only just getting to grips with data tiddlers but they seem to me to be similar to single column spreadsheets. They have an index and a single column of values, and each value 'slot' is similar to a spreadsheet's cell in just one column.
If I create 2 or more data tiddlers with the same indexes then the number of value columns available to my 'spreadsheet' expands and 'cells' across both tiddlers can be referenced by {{distinctdatatiddlername##sharedindex}}.
In an ideal world I'd like to be able to sum those values but also to be able to carry out the other maths operations that a spreadsheet allows on individual values, ranges of values, groups of values that match a criteria etc - (i.e values filtered to meet a condition, across several data tiddlers).
I hope that makes some sort of sense. Just my 2 shrimps worth if you're thinking about formulating a pull request.

[PMario]

Hi,

BE AWARE: The whole thing is still beta. 5.1.20 isn't released yet.

I did just create a PR at github: https://github.com/Jermolene/TiddlyWiki5/pull/3971

and a test version is uploaded to tiddlyspot: http://3971.tiddlyspot.com

have fun!

mario

[TonyM]

Mario

I noticed when using macros or action widgets such as newttiddler parameters of the form name=value can be automatically converted to variables containing the value. I would like to see a method to do this with multiple key value pairs exposed for users. Solutions such as yours would benefit from returning such key value pairs that they could be handled like this. Imagin if we could iterate a data tiddler and have <<key>> and <<value>> and even <<indexnumber>> returned.

We could also have data tiddlers we access primarily by index number.

Regards
Tony

[PMario]

Hi. I did have a look at those possibilities but I couldn't find a way that filter operators can return internal values to the calling widget.
-m

[TonyM]

Mario,

I think I understand what you mean but my observation of this was calling a modal with key=value pairs and at least within the modal the variables are populated. This looks like a workaround however the modal needs to define a close action and that could write the variables values elsewhere, like tiddlers or fields. Or even write a tiddler taged as a macro that defines and sets global variables.

In fact writing the content of a global macro may be a way to bypass this limitation.

I know this is a somewhat advanced idea and I may not have expressed it well but I hope you see what I mean.

Thony

[PMario]

I know what you mean. The problem is, that filter operators have no knowledge about the widget context they are called in. They are designed to have no side effects.

-m

[TonyM]

Mario,

The problem is, that filter operators have no knowledge about the widget context they are called in. They are designed to have no side effects.

In some ways this handicaps us. This possibly needs to remain the case, however we should be able to provide an alternative method.

I believe I have found a way and will open two new threads soon, since this is wandering off topic,  best in Development forum I expect?

• Processing Key Value Pairs
• Turning key=value, fieldname/value and variables into global variables

Regards

Tony

[Watt]

Thank you TonyM and PMario for developing this further. It is much appreciated. Data tiddlers have always been a mystery to me but I'm beginning to see how useful they can be and anything that makes them more accessible is great for beginners.

The documentation for creating them, appending and editing them and then manipulating and extracting their index keys and values can take a bit of understanding. Anything that simplifies the process and provides more examples and clues is a big help for the uninitiated. If you're typing in a filter on a phone keyboard you also really appreciate simplicity.

PMario if you can, could I ask you for an example of how to count the '1's as per my original post?

01: 0
02: 1
03: 1
04: 0
05: 1
06: 0

I've got it working with TonyM's method but not worked it out with your PR version. At the moment I'm more interested in manipulating values rather than keys so the many examples on your demo page help a lot, but the one example I needed wasn't there (always the case when I'm looking for examples!)

Thanks again guys, it's a valuable discussion.

[PMario]

Hi Watt,

Be aware: The "sum[]" operator will only work with TW version 5.1.20+

Try this.

<$set name="digit-pattern" value="[0][0-9]">
<$list filter="[tag[test-data]keyvalues:key,value:--><digit-pattern>sum[]]" variable=item>
<<item>><br>
</$list>
</$set>

In the above example, it uses every data tiddler tagged: test-data

<digit-pattern> is used to define the range eg: if you use [0][0-2] you'll limit it to 00, 01, 02 indices.

Without sum[] it looks like

<$list filter="[tag[xx]keyvalues:key,value:-->:usevalue[1]count[]]" variable=item>
<<item>><br>
</$list>

I did create a video series, some time ago, where I did explain everything step by step. One element of the project is manipulating data-tiddlers. ... Because I did have some trouble using it, I did create the PR now. If I would have had the new "keyvalues" operator, I think it would have been easier.

There is an edition, you can play with.

see: TiddlyWiki Script and ToDo manager HowTo

The Intro post is here: https://groups.google.com/forum/#!msg/tiddlywiki/_VQw8yIKWn8/G8TO8h6MCAAJ

have fun!

mario

[Watt]

Thanks Mario, thanks Tony - I'll play with everything and become a data tiddler ninja.