Who can I thank for Math operators?

[Mat]

Hi guys! ...in spite of my absence I do 'peek by' occasionally and I just noted that math operators are added to the prerelease!

I just want to thank whoever is behind this (Evan?), plus Jeremy for including it in core! Basic math ops is one of those things that anyone needs either sooner or later in a utility tool like TW, so: THANK YOU!

Also, anyone curious about TW should hear about this right away so it deserves a mention in HelloThere.

IMO, "basic math" was the missing piece to fulfill the "minimum of a fully general note-taking tool" - i.e: produce text, produce images, do calculations.

Wonderful!

<:-)

P.S sorry for my absence but it will continue for a while.

[Mark S.]

Interesting. Everyone knows about spreadsheets, but most people don't know about the power of databases. Once you can manipulate numbers, then you can run a garage sale, charity drive, small business, exercise logs, scientific research, sports applications ... and generate the producibles that help make sense of it all.

-- Mark

[Jeremy Ruston]

Hi Mat

Great, glad you like it! This is a new implementation that emerged from discussions about a different issue on GitHub (see https://github.com/Jermolene/TiddlyWiki5/pull/3572#issuecomment-460958193).

It is a bit different than Matt’s Calc plugin <https://github.com/mklauber/tiddly-calc> and Evan’s Formula plugin <https://github.com/EvanBalster/TiddlyWikiFormula>. It doesn’t use conventional “postfix” maths notation (ie, the familiar stuff like (2+2)/3 etc.). Instead, it implements maths operations as filter operators that work the same way as others except that they interpret their arguments as numbers and numeric results are converted to strings.

I went for this approach because it makes things really, really small and simple: by re-using all the existing filter operator infrastructure we avoid having to add or implement a new expression parser. On the other hand, it means that it isn’t much use for formulae that have been entered by users not familiar with TiddlyWiki’s filter syntax. We’ll still need plugins like those above for those applications.

I haven’t quite finished the documentation, but the list of new operators is as follows - there are new string operators like split and join as well as the maths operators:

negate - negation

abs - absolute value

ceil - smallest integer greater than or equal to a given number

floor - the largest integer less than or equal to a given number

round - the value of a number rounded to the nearest integer

trunc - the integer part of a number by removing any fractional digits

sign - the sign of a number, indicating whether the number is positive, negative or zero

add - add an operand to every number in the input list

subtract - subtract an operand from every number in the input list

multiply - multiply every number in the input list by an operand

divide - divide every number in the input list by an operand

remainder - replace each item in the input list with the remainder when dividing that number by an operand

max - replace any number in the input list larger than the operand with the operand

min - replace any number in the input list smaller than the operand with the operand

fixed - formats a number using fixed-point notation with the number of fractional digits after the decimal point given by the operand

precision - returns a string representing the number to the precision specified in the operand

exponential - returns a string representing the Number object in exponential notation with the number of digits after the decimal point specified in the operand

sum - replaces the input list with the sum of all the entries

product - replaces the input list with the product of all the entries

maxall - replaces the input list with the maximum value of all the entries

minall - replaces the input list with the minimum value of all the entries

length - replaces each item in the input list with its string length in characters

uppercase - replaces each item in the input with its uppercase equivalent

lowercase - replaces each item in the input with its lowercase equivalent

trim - trims each item in the input list

concat - concatenates all of the items in the input list into a single string

split - splits each of the items in the input list by a given separator

join - joins all of the items in the input list with a given separator

As usual, any comments and thoughts welcome,

Best wishes

Jeremy

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/1eaad72c-4e80-4999-926c-13cf7ea16ba8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[TonyM]

Jeremy,

Yet another new feature on the horizon that will add great utility to TiddlyWiki, Even better if it "uses the really small and simple: by re-using all the existing filter operator infrastructure" and I see that $set and triple curly braces 

{{{ filter }}}

that transcludes the result of the filter will be common ways of using the "math filters"

I personally love filters but this is only now that years of experience makes me very comfortable with them, 

I am concerned about new users who want to implement maths but have not yet fully grasped filter concepts, this includes variables, transclusions and references in maths filters. Could we try and make the documentation spell out how to achieve common uses? I would be happy to contribute here of course.

I am keen to ensure we document how to use maths in the following high level cases with variables, transclusions and references

• Within wikiText
  
• Within Filters (in this case interacting with title lists)
  
• Within Widgets and HTML - Parameters and Attributes
  
• Within macro definitions / macro body
  
• Within a Macro call or for a "macro parameter"
  
• and Concatenating values for use in maths formulae

Since this is new I am not yet in a position to test it, but will we be able to retrieve the maths results an use them in the above as easily as we can with Evans (= mushroom brackets =) ? Perhaps we can provide an easy method to allow "substitution" of a maths result such as $(mathresult)$ may imply.

Regards

Tony

[TonyM]

Another math Operator?

1. Could we have an "inc" or increment that adds one (or n) to its input as this is a useful and commonly used alternative to add 1 to self?.
   

Regards

Tony

On Friday, February 8, 2019 at 8:42:06 AM UTC+11, Jeremy Ruston wrote:

[Jeremy Ruston]

Hi Tony

I am concerned about new users who want to implement maths but have not yet fully grasped filter concepts, this includes variables, transclusions and references in maths filters.

Conversely, if we'd introduced another new mechanism then there would be even more to learn. TiddlyWiki's primitives might be hard to learn, but they are endlessly reused, and so the bit and pieces that one has to learn applies over and over.

Could we try and make the documentation spell out how to achieve common uses? I would be happy to contribute here of course.

Yes, the documentation is a work in progress and contributions are very welcome.

I am keen to ensure we document how to use maths in the following high level cases with variables, transclusions and references

• Within wikiText
  
• Within Filters (in this case interacting with title lists)
  
• Within Widgets and HTML - Parameters and Attributes
  
• Within macro definitions / macro body
  
• Within a Macro call or for a "macro parameter"
  
• and Concatenating values for use in maths formulae

Note that there's nothing specific about the maths operators; they are used just the same as the other filter operators.  If you know how to use filter operators then you know how to use the new maths operators.

Since this is new I am not yet in a position to test it, but will we be able to retrieve the maths results an use them in the above as easily as we can with Evans (= mushroom brackets =) ? Perhaps we can provide an easy method to allow "substitution" of a maths result such as $(mathresult)$ may imply.

The implications of my earlier explanation might not be clear. We're reusing the filtering mechanism. There is no new syntax like the mushroom brackets because we're reusing the filtering mechanism. We don't need any new method of substitution because we're reusing the filtering mechanism.

Best wishes

Jeremy

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/1cefed6d-d923-48dd-8f15-c83c55565cc1%40googlegroups.com.

[Jeremy Ruston]

Hi Tony

1. Could we have an "inc" or increment that adds one (or n) to its input as this is a useful and commonly used alternative to add 1 to self?.
   

I don't think it's worth adding [inc[]] and [dec[]] when we can use [add[1]] and [subtract[1]].

Best wishes

Jeremy.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/1f403eb9-12fe-4f61-bbc0-d275d42259aa%40googlegroups.com.

[TonyM]

Fair enough

[TonyM]

Jeremy,

So to be clear, you believe, using the triple curly braces will be sufficient to place calculated values anywhere we want?

Note that there's nothing specific about the maths operators; they are used just the same as the other filter operators.  If you know how to use filter operators then you know how to use the new maths operators.

 

What I am saying is this is not so easy for new users, learning filter operators, so I believe the documentation must provide the guidance they need. I am suggesting some explicit guidance on using filters for maths so a complete understanding of filters is not a pre-requisite for maths operations. There are a few places in TiddlyWiki where we put people in this bind,

 "they need to know more before they can start to learn the basics, but you need the basics so you can learn more".

I recognise this because I remember my own journey, but this is hard for people with a deep knowledge of TiddlyWiki to recognise.

This can be addressed in documentation, which I will contribute to, however I think it important for the development process to take this into account.

Yours, working towards world domination by TiddlyWiki

Regards, 

Tony

 

[Diego Mesa]

Hey Jeremy,

This is wonderful thank you! In the example on the docs you have to filter twice = once to get the count, then another to get an array of lengths, sum them and then divide. 

Can we also add a "mean" operator? 

Best

Diego

[Jeremy Ruston]

Hi Tony

Apologies I could have expressed myself more clearly. I don’t think one can separate out teaching people to use the maths operations without teaching them to use filters. But conversely, maybe the maths operators will make it easier to teach the filter mechanism.

At the moment, there is an overview tiddler for the maths operators:

https://tiddlywiki.com/prerelease/#Mathematics%20Operators

And now I’m gradually adding docs for each individual operator.

Best wishes

Jeremy.

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/2382c1ed-18de-44b2-8a9d-c13a016d5922%40googlegroups.com.

[Jeremy Ruston]

Hi Diego

This is wonderful thank you! In the example on the docs you have to filter twice = once to get the count, then another to get an array of lengths, sum them and then divide. 

Can we also add a "mean" operator? 

I was worried that doing so would open a hornets nest where people would want a full set of Excel-like stats functions: modes, medians, etc (which I think should be a plugin). Meanwhile, averaging makes quite a good example of a more complex calculation, and demonstrates that more complex operations can be built from the primitives provided. But having said that I’m open to arguments…

Best wishes

Jeremy

Best

Diego

On Thursday, February 7, 2019 at 6:06:59 PM UTC-6, TonyM wrote:

Jeremy,

So to be clear, you believe, using the triple curly braces will be sufficient to place calculated values anywhere we want?

Note that there's nothing specific about the maths operators; they are used just the same as the other filter operators.  If you know how to use filter operators then you know how to use the new maths operators.

 

What I am saying is this is not so easy for new users, learning filter operators, so I believe the documentation must provide the guidance they need. I am suggesting some explicit guidance on using filters for maths so a complete understanding of filters is not a pre-requisite for maths operations. There are a few places in TiddlyWiki where we put people in this bind,

 "they need to know more before they can start to learn the basics, but you need the basics so you can learn more".

I recognise this because I remember my own journey, but this is hard for people with a deep knowledge of TiddlyWiki to recognise.

This can be addressed in documentation, which I will contribute to, however I think it important for the development process to take this into account.

Yours, working towards world domination by TiddlyWiki

Regards, 

Tony

 

-- 
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/25f5e2f6-8f0f-4f8a-8c79-6a14b737f91f%40googlegroups.com.

[@TiddlyTweeter]

Jeremy

I looked at https://tiddlywiki.com/prerelease/#Mathematics%20Operators

I can already see the new maths will considerably ease making simple things like invoices and allowance trackers.

Great stuff!

Josiah

[Jeremy Ruston]

I can already see the new maths will considerably ease making simple things like invoices and allowance trackers.

Thanks Josiah.

There’s a bunch of fun possibilities that I hope to see:

* a recipe wiki that can recalculate the quantities of ingredients for different numbers of people

* multiple choice quizzes that score the answers

* graphs and charts made by dynamically rendering SVG

* tools for estimation and resource planning

* a guitar “fake book" that can transpose chords into different keys

I think there might also be a fair bit of arithmetic in RPGs; it’ll be fascinating to see what people make with it.

Best wishes

Jeremy

Great stuff!

Josiah

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/f72e38b4-d324-408d-93db-92695358aa04%40googlegroups.com.

[Jed Carty]

Wow, I haven't been keeping up with what is going on for a bit and now between this and the subfilter operator I can rework a lot of what I have made before to be much simpler.

And for things like a mean operator I think that a plugin to add more math operators would be pretty simple to put together, but I think that Jeremy is correct that if we add much beyond the basics there will be no end to requests that are just as reasonable as whatever we have already made.

Until such a plugin is created I think that putting together a wiki that has recipes for various math operations using the primitives included in the core could be useful. I am not certain but the sub filter operator may let us create macros that define the more complex operations and include them.

I just tried to make an example of mean and it isn't as simple as I hoped because we don't have a way to store the sum and the number of operands in the same filter run. I will think about this some more.

[Jed Carty]

I have run into one problem testing the math operators, they still enforce uniqueness in filtering, so if you use

<$list filter='1 1 1 1 +[sum[]]'/>

you get the answer 1.

I am looking at what would be needed to allow duplicate entries but considering how carefully designed the list widget and filtering is I am a bit concerned about the potential complexity of fixing this.

Or I am missing something obvious. I hope that is the case.

[Diego Mesa]

Hey Jeremy,

I 100% see your point and agree. 

[Jeremy Ruston]

Hi Jed

I just got caught by this too. I’ll comment more over on the GitHub issue you opened:

https://github.com/Jermolene/TiddlyWiki5/issues/3757#issuecomment-461797553

Best wishes

Jeremy

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/35153bbc-e046-4293-86db-de024ae8b9bd%40googlegroups.com.

[Diego Mesa]

Im so excited about this! I'm going to try and reimplement anwiki to only use native TW! 

[Mal]

Jeremy,

In another thread, there was some discussion about the split operator.  It exhibits some interesting behaviour when using split[] with no arguments:

1. it produces a list of single characters from the input string - this seems like a sensible edge case behaviour.
2. it does not exhibit the usual uniqueness property of filter lists - duplicates are passed through to the output.
3. The output can include single space characters - this is tricky considering lists are space delimited.

Are these behaviours by design and will continue to exist in the final release of 5.1.20 and subsequent releases?  If so, the split operator can be used for some useful string manipulation hacks.

Regards,

Mal

[Jeremy Ruston]

Hi Mal

In another thread, there was some discussion about the split operator.  It exhibits some interesting behaviour when using split[] with no arguments:

1. it produces a list of single characters from the input string - this seems like a sensible edge case behaviour.

Yes, it's exposing the way that the underlying JavaScript "split" function works, where it is a common idiom for working on the individual characters of a string.

2. it does not exhibit the usual uniqueness property of filter lists - duplicates are passed through to the output.

It's surprising but individual operators can return result lists with duplicates. The enforcement of uniqueness is performed when the result lists of multiple runs are merged. But there are also situations where the system assumes that duplicates have already been removed. For example, this returns "jeremy" as expected:

[[jeremy]split[]]  +[join[]]

But this returns "jremye":

[[jeremy]split[]] e +[join[]]

The subtlety is that the additional run consisting of the letter "e" dedupes the first of the first letter "e", but fails to "see" the second".

3. The output can include single space characters - this is tricky considering lists are space delimited.

There's less mystery here: spaces are used for delimiting lists when they are serialised into a string, but internally lists are held as arrays.

Are these behaviours by design and will continue to exist in the final release of 5.1.20 and subsequent releases?  If so, the split operator can be used for some useful string manipulation hacks.

Yes to both questions.

It's pretty clear now that the way that de-duplication is implemented was a mistake. I don't think we can fix it while retaining 100% backwards compatibility. As discussed in this ticket, I'm now of the opinion that we need to fix this:

https://github.com/Jermolene/TiddlyWiki5/issues/3757

Best wishes

Jeremy

Regards,

Mal

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/140182eb-2acf-4299-b172-c6899822d32e%40googlegroups.com.

[Mal]

That's great Jeremy. Thanks for the explanation.

Regards,

Mal