Ran into some macro development subtleties

[Hylke W. van Dijk]

Good day,

Triggered by the announcement of Tamasha I decided to dig into TW5 some more. Therefore I created a --javascript free -- plugin for easy import and display of spreadsheet data, or more generally dataframes. Along the way I ran into some subtleties / miscomprehensions of the tiddlywiki framework , which might be of interest to others. A demonstration of the dataframe plugin and  a working description of three subtleties in action is available at: https://hwvandijk.bitbucket.io/tw-dataframe/

1. splitregexp that crashes the Javascript engine of Firefox 58, but works on tiddlydesktop 0.14 (chromedriver 81)

I used a (elaborate) regular expression in splitregexp to split "ABC123" in to "123 ABC" in one go.

i.e.

regexp="([A-Z]+(?=[0-9]+))|([0-9]+(?=[A-Z]+))" 

and filter="[<ref>splitregexp<regexp>reverse[]]"

I get a Javascript error: uncaught exception: Linked List only accepts string values, not undefined

The problem has been solved by using a more straightforward regular expression in a  search-replace:regexp filter operator.

2. $x$ and <<__x__>> or similar but not identical

I had problems using macro parameters in filters. Therefor I cast them in a <$set> or <$vars> variable, either through $x$ or <<__x__>>. The former works fine unless a parameter x that is passed to the macro contains slashes.

<<mymacro one.2.three> works fine, <<mymacro one/2/three>> fails. It renders the <$vars expression> literally in the page.

3. <$set filter:"filter" variable="var"> and <$vars var={{{ filter }}}> give different results

To specify the index/columns of the dataframe  to be displayed I use a list of ranges. Meaning that a spec of "1,3-5,7" should be transformed into "1,1 3,5 7,7" such that when you feed these entries one by one into the range operator you get [1 2 3 4 5 7].

However with:

<$vars p=<<__param__>>

      regexp="^(\d+)$" >

<$set name="setref" filter="[<p>split[,]search-replace:g:regexp<regexp>,[$1,$1]]" >

<$vars varsref={{{ [<p>split[,]search-replace:g:regexp<regexp>,[$1,$1]] }}} >

The variables setref and varsref are not identical. varsref is wrong, you should use setref.

varsref only works for simple specifications, such as "7". It looks like varsref does not obey the g (global) specifier.

 

Sorry for the long mail, but hopefully someone can point out my misconceptions or file a bug if that is appropriate.

Cheers,

Hylke van Dijk

[Mat]

WOW! To say the least. (And incidentally I'm working on a distantly related thing).

Regarding your problems here are some first guesses:

>regexp="([A-Z]+(?=[0-9]+))|([0-9]+(?=[A-Z]+))" 

>and filter="[<ref>splitregexp<regexp>reverse[]]"

Maybe you need to escape the square brackets?

><<mymacro one/2/three>> fails. I

See if enclosing the argument with "..." helps

><$set filter:"filter" variable="var"> and <$vars var={{{ filter }}}> give dif

Again some syntactic tidbits (but maybe you just mistyped in this post):

<$set filter="filter" name="var">

<:-)

[Mat]

OK, so judging from your neat writeup in  Passing Macro Parameters with Slashes I'd say it is syntactical errors. This:

x=$p$

definitely rubs the wrong way. It will work for very "simple" arguments but it must be enclosed in quotes to handle arguments with space characters. Often even triple quotes (x="""$p$""" or x='''$p$'''), to be able to handle arguments with quotes in them. This is not the same with <<__p__>> which is enclosed already.

(Insufficient) references:

Macro Calls in WikiText

Macro Definitions in WikiText

<:-)

[Hylke W. van Dijk]

Thanks Mat,

ad 1

Escaping the regexp might be a solution. But how? I mean in a regular expression [a-z] is different from \[a-z\].

Unsure how to escape the regular expression. It might be a greediness / look-ahead problem.

ad 2

No luck here, tried it. If I remember correctly <<__x__>> sort of casts the variable into a tiddly. That might be part of the explanation.

ad 3

Sorry, typo in this mail.

Cheers,

Hylke

On Tuesday, March 2, 2021 at 11:58:52 PM UTC+1 Mat wrote:

[Mat]

<$set name="setref" filter="[<p>split[,]search-replace:g:regexp<regexp>,[$1,$1]]" >

Maybe I've missed some late dev but that comma is not something I'm familiar with syntactically. Ref.

(And, as noted, please triple quotes around that filter.)

<:-)

[Hylke W. van Dijk]

Spot on!

The triple quotes, which I came across, but did not understand. And which I did not try before.

-hw

[Hylke W. van Dijk]

The comma is part of the search-replace filter operator syntax

https://tiddlywiki.com/#search-replace%20Operator

-hw

[Mat]

ad 1

Escaping the regexp might be a solution. But how? I mean in a regular expression [a-z] is different from \[a-z\].

Unsure how to escape the regular expression. It might be a greediness / look-ahead problem.

OK, this is not my domain but I think maybe the \ character is ignored and so \[ is really interpreted as just a [ whereas you'd have to do \\[ for it to be interpreted as \[. Again, just a hunch. I'm sure there are others here who can answer though.

<:-)

[Mat]

The comma is part of the search-replace filter operator syntax

That was very valuable for me to learn - thanx! 

<:-)

[Hylke W. van Dijk]

Escaping the [] does the trick in the regular expression. I just gave it a try and the macro that previously crashed the JavaScript engine now works

regexp : (\[A-Z\]+(?=\[0-9\]+))|(\[0-9\]+(?=\[A-Z\]+))

remains the <$set name="setref" ...> and <$vars varsref={{{ ... }}} > unexplained. It is the <$vars varsref={{{ ... }}}> that fails.

Tried quoting the regexp, to no avail.

Tripple quoting the filter in <$set ..> has no influence.

Some how the global operator does not work in the <$var ..> setting of the variable.

-hw

[Mat]

remains the <$set name="setref" ...> and <$vars varsref={{{ ... }}} > unexplained.

I agree, I can't get it to work in your demo either. I suggest you starta a new thread with this issue only and a permalink to the demo tiddler etc.

<:-)

[Hylke W. van Dijk]

> regexp : (\[A-Z\]+(?=\[0-9\]+))|(\[0-9\]+(?=\[A-Z\]+))

Sorry, but escaping the regular expression does not work. I did not watch teh result carefully enough, I guess I was too pleased that the JavaScript did not crash.

Escaping the blockquotes ([]) does not crashes the engine, but also the regular expression does not do its work ...

[Saq Imtiaz]

Re: (1) and splitregexp, use of capture groups isn't officially supported in regular expressions in filters with the exception of the new search-replace operator. This probably needs to be better documented. 

However what is interesting here is that you report that it works in TiddlyDesktop. Was it the same TiddlyWiki file used in FF58 and in TiddlyDesktop? Or was the TiddlyDesktop file 5.1.22 or 5.1.23 pre-release?

Late in the 5.1.23 pre-release phase, filters were switched to using a linked list implementation instead of an array for performance purposes. I wonder if perhaps the new linked list implementation is less tolerant than than the old implementation for undefined values, and that there has been a regression.

Re: (2) use single, double or triple quotes around arguments. The <<__x__>> syntax was introduced precisely for those situations where getting the quoting of parameters correct might be tricky.

Re: (3)  One of the important differences between the Set and Vars widgets is that:

•  the Set widget directly supports filters and (used in the filtered item variable assignment form) assigns a title list to the variable (which can hold multiple titles).
•  However the Vars widget does not directly support filters and using a filtered attribute assignment using the triple brace syntax assigns the first title from the filter results to the variable. To be clear this isn't really down to the Vars widget but rather the way filtered attributes for widgets are evaluated, i.e. attribute={{{ filter }}} which always assigns the first title from the filter results.

Hope this helps,
Saq

On Tuesday, March 2, 2021 at 10:18:33 PM UTC+1 hwva...@gmail.com wrote:

[Hylke W. van Dijk]

Thank you Saq for the clarifications, much appreciated.

Re (1)

My sandbox tiddlywiki in tiddlydesktop 0.0.14 is based on tiddly wiki 5.1.23.  The effects I observe thus fit your intuition perfectly.

Re(2)

Verified:

   single quotes ('$p$'): work

   double quotes("$p$"): work

   triple double quotes ("""$p$"""): work

   triple single quotes ('''$p$'''): fail

Re(3)

Spot on explanation of the observed pattern.

Cheers,

-hw