Wikified macro notation suggestion

[Stobot]

I don't know if anyone else builds things like I do - with a ton of dynamic tables and branched lookups, but I use this pattern all the time:

\define macrotobuildfilter() ...

<$wikify name="macrotobuildfilterwiki" text=<<macrotobuildfilter>>>

<table>

<$list filter=<<macrotobuildfilterwiki>>>

...

</$list>

</table>

</$wikfiy>

That's just with one, but usually I have a few nested ones. In fact I'll go so far as to say that more than half of the time I use a macro, I have to wikify it before use as it's being used as a parameter for something else. I've often thought of how great it would be to have that wikification done in the same step - maybe with triple brackets like <<<wikified macro>>> ? Similarities to the {{{...}}} notation in a way.

Is this possible? I see some other threads on additional parser rules, so thought it might be a good time to bring up.

[Mat]

Stobot wrote:

I don't know if anyone else builds things like I do - with a ton of dynamic tables and branched lookups, but I use this pattern all the time:

(probably not ;-) 

I want to call your attention to that $wikify is a very expensive widget, especially if you nest them.

Often, but not always, the same result can be achieved with the $set widget and its attibute set to select=0 

I'm not qualified to answer you actual question.

<:-)

[Jeremy Ruston]

Hi Stobot

I don't know if anyone else builds things like I do - with a ton of dynamic tables and branched lookups, but I use this pattern all the time:

\define macrotobuildfilter() ...

<$wikify name="macrotobuildfilterwiki" text=<<macrotobuildfilter>>>

<table>

<$list filter=<<macrotobuildfilterwiki>>>

...

</$list>

</table>

</$wikfiy>

That's just with one, but usually I have a few nested ones. In fact I'll go so far as to say that more than half of the time I use a macro, I have to wikify it before use as it's being used as a parameter for something else.

The pattern you’re using here is a subtle one: to use the wikification process to create filter strings. As you’ve probably gathered, it can be quite brittle. For example, there can be problems if titles included in your filter strings include wikitext markup (e.g. HTTP links contain an embedded //). One tip is to prepend the \rules pragma to the start of the text to be wikified and thus restrict the wikitext rules that will be recognised.

I've often thought of how great it would be to have that wikification done in the same step - maybe with triple brackets like <<<wikified macro>>> ? Similarities to the {{{...}}} notation in a way.

Yes, it’s somewhat anomalous that there isn’t a quoted attribute syntax that returns the wikified value of the argument. Part of the reason is that as Mat points out, wikification is a relatively slow process (unlike wikifying the page we can’t do any selective refreshing, we have to process the entire text on each refresh cycle).

Is this possible? I see some other threads on additional parser rules, so thought it might be a good time to bring up.

It might be worth experimenting with it, but I’d be concerned about performance.

An alternative approach that might be a bit easier would be to create a pragma that’s a shortcut for the wikify widget. Something like:

\define-with-wikification macrotobuildfilter() ...

<table>

<$list filter=<<macrotobuildfilterwiki>>>

...

</$list>

</table>

An ordinary \define pragma generates a single <$set> widget; this new pragma would generate a <$set> widget with a nested <$wikify> widget.

https://github.com/Jermolene/TiddlyWiki5/blob/master/core/modules/parsers/wikiparser/rules/macrodef.js

Best wishes

Jeremy

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/a0aff5bb-2aa3-4bc0-8094-72cf07c550a8%40googlegroups.com.

[TonyM]

Jeremy, Stobot,

Any method that can make wikify behave better would be appreciated. 

Jeremy, can you give an example of this please

One tip is to prepend the \rules pragma to the start of the text to be wikified and thus restrict the wikitext rules that will be recognised.

Stobot, 

• I think your pattern could be of great use, pending a better method if you could publish your code pattern with some examples It would be a good contribution and others may also show alternative ways to achieve the same thing. 
• There are a few cases where construction of filters before use would get around other limitations or the need for nested filters. 
• I am also starting to configure some longer filters as global macros such as standard-fields so they can be excluded from the result of the fields[] operator.

Share Share Share

Regards

Tony

Hi Stobot

To unsubscribe from this group and stop receiving emails from it, send an email to tiddl...@googlegroups.com.

[Jeremy Ruston]

Here’s a simple example of using the wikify widget:

<$set name="text" value="This is ''bold'' and this is //italic//">

<$wikify name="html" text=<<text>> output="html">

<$text text=<<html>>/>

</$wikify>

</$set>

And here’s the same example with the addition of adding the prefix:

\define rules()

\rules except italic

\end

<$set name="text" value="This is ''bold'' and this is //italic//">

<$set name="text" value={{{ [<text>addprefix<rules>] }}}>

<$wikify name="html" text=<<text>> output="html">

<$text text=<<html>>/>

</$wikify>

</$set>

</$set>

Best wishes

Jeremy

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/d16dfbae-0bb9-403b-abf7-129918ba3ea1%40googlegroups.com.

[TonyM]

jeremy,

Thanks for your response.

The following is an outline of how I see this for a "view from a different mind", for you to keep in mind going forward.

In all your examples of using wikify you tend to use it in the body of the wikitext, just in time, wrapping its own output. I tend to run into difficulty using it any other way, such as a local or global macro, and when in your example there are "variables" in the source eg if your set included {{!!fieldname}} or $tooltip$ etc... in what is to be wikified.

It is clear we need to fill out the documentation Pragma and Wikify Widget, sometimes so people know this does not promise things it may "on the surface"

for examples Question that arise from the naive

Pragma

• When using a rule inside a define is it valid only within that macro?
• When using it at the top of the tiddler does it apply to the whole tiddler? including macros subsequently defined or referenced?
• Is using \import filter more efficient than global macros or the new view only macros tag
• Where are the rules documented? Are they a one for one match with these names $:/core/ui/ControlPanel/Parsing

I found the "undocumented "[wikiparserrules[]]" filter operator

Wikify

• With the Wikify widget is there a way for us to apply the pragma rules to the result?
• The Wikify Widget would benefit from examples and ideally through the various templates available already in the core, so people can generate and store tiddlers in alternate formats that already operate under export, save and other options.
• Storing the result of a wikify operation in the current wiki is also of substantial value (I would detail if asked)

Parsing is of substantial interest to many, so I hope eventually we can open this to users, especially to add our own markup. I and mario have being looking at this with the leading dotparagraph and trailing spacespace. Of course such markup will not be universal when tiddlers are transferred but they can be made to "fail gracefully"  when not defined.

The markup process provides the opportunity for authors and designers to enhance how they input information into a wiki, in effect allowing the development of other shorthand, custom shorthand can still be evaluated into html or text, copy and paste etc.. and exported as standard content.

Regards

Tony

On Thursday, March 12, 2020 at 4:53:02 AM UTC+11, Jeremy Ruston wrote:

Here’s a simple example of using the wikify widget:

<$set name="text" value="This is ''bold'' and this is //italic//">

<$wikify name="html" text=<<text>> output="html">

<$text text=<<html>>/>

</$wikify>

</$set>

And here’s the same example with the addition of adding the prefix:

\define rules()

\rules except italic

\end

<$set name="text" value="This is ''bold'' and this is //italic//">

<$set name="text" value={{{ [<text>addprefix<rules>] }}}>

<$wikify name="html" text=<<text>> output="html">

<$text text=<<html>>/>

</$wikify>

</$set>

</$set>

Best wishes

Jeremy

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/d16dfbae-0bb9-403b-abf7-129918ba3ea1%40googlegroups.com.

[Jeremy Ruston]

Hi Tony

The following is an outline of how I see this for a "view from a different mind", for you to keep in mind going forward.

In all your examples of using wikify you tend to use it in the body of the wikitext, just in time, wrapping its own output.

The results of the wikify widget are indeed only available within the content of the widget.

Can you show a counter example to help me understand what you mean?

I tend to run into difficulty using it any other way, such as a local or global macro, and when in your example there are "variables" in the source eg if your set included {{!!fieldname}} or $tooltip$ etc... in what is to be wikified.

You’re saying that you can’t use the wikify widget within a macro? I’m not sure what you’re getting at here. Perhaps some examples would help?

It is clear we need to fill out the documentation Pragma and Wikify Widget, sometimes so people know this does not promise things it may "on the surface"

Any help with that is welcome.

for examples Question that arise from the naive

Pragma

• When using a rule inside a define is it valid only within that macro?

Yes.

• When using it at the top of the tiddler does it apply to the whole tiddler? including macros subsequently defined or referenced?

It only applies to the direct content of the tiddler itself, and not to the content of any macros defined within it.

• Is using \import filter more efficient than global macros or the new view only macros tag

They’re all the same mechanism under the hood: the importvariables widget.

• Where are the rules documented? Are they a one for one match with these names $:/core/ui/ControlPanel/Parsing

The pragmas are documented here:

https://tiddlywiki.com/#Pragma

I found the "undocumented "[wikiparserrules[]]" filter operator

That’s a filter operator, which are nothing to do with pragmas.

Wikify

• With the Wikify widget is there a way for us to apply the pragma rules to the result?

That’s exactrly what my example in my previous post showed.

• The Wikify Widget would benefit from examples and ideally through the various templates available already in the core, so people can generate and store tiddlers in alternate formats that already operate under export, save and other options.

What do you mean by “templates” here? What’s the connection between templates and the wikify widget? Reusing the existing export filters doesn’t have anything obvious to do with the wikify widget.

• Storing the result of a wikify operation in the current wiki is also of substantial value (I would detail if asked)

As you know, changes can only be made to the wiki via action widgets, are you experiencing difficulties doing that? Again, perhaps an illustration would help.

Parsing is of substantial interest to many, so I hope eventually we can open this to users, especially to add our own markup.

Beyond your desire to add custom markup without having to use JavaScript, what else is needed to open parsing to users?

I and mario have being looking at this with the leading dotparagraph and trailing spacespace. Of course such markup will not be universal when tiddlers are transferred but they can be made to "fail gracefully"  when not defined.

The markup process provides the opportunity for authors and designers to enhance how they input information into a wiki, in effect allowing the development of other shorthand, custom shorthand can still be evaluated into html or text, copy and paste etc.. and exported as standard content.

I’ve definitely already got the message that you want to be able design custom markup :)

Best wishes

Jeremy

To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/78d04c90-9ea0-4ea2-8f55-9da55d1ff776%40googlegroups.com.

[Stobot]

Interesting - thanks.

Looking at my use-cases more thoroughly since Tony asked, they do all seem to fall into the inability to "nest functions" (functions being widgets in TW) using wikitext. My use cases fall within a couple common buckets

1. Dates: I use a lot of my wikis for project management, so I'm using things like <<dateadd>> / <$macrocall name="dateadd" basedate=... days=.../> to build a new date, then use that within a filter string

2. General lookups: Using one macro to determine a "tiddler type" based on it's tags, then using that in a filter or logs (createtiddler action widgets)

3. Fallbacks: Again going back to dynamic tables, I'll do something like have an edit-box the user can use for limiting number of results, and then have to use that number in a filter later. Since I want a default value in case the user doesn't use the edit-box, I use a <$list filter="..." emptyMessage="10">{{!!title}}</$link> pattern to get their value (or the fallback value), and then build that into the filter string.

Jeremy, it sounds like what you're saying is that I can up-front create my own version of "\define" once at a wiki level, and then use that when I know I need the results wikified - is that right? I tried to follow some of your notes on pragma, but it looked like those notes may have been to address something else. Is this possible within wikitext, or would I need to build a javascript thing? (I'm not javascript native, but could maybe adapt a similar example if you knew of one).

Also, going back to Mat's comment, I'll have to play with the "select=0" of <$set> to see how that can help me - always looking for performance gains!

[TonyM]

Jeremy,

I cat yet respond to all your questions because I remain somewhat confused but respond to a few below;

On Thursday, March 12, 2020 at 10:07:06 PM UTC+11, Jeremy Ruston wrote:

Hi Tony

The following is an outline of how I see this for a "view from a different mind", for you to keep in mind going forward.

In all your examples of using wikify you tend to use it in the body of the wikitext, just in time, wrapping its own output.

The results of the wikify widget are indeed only available within the content of the widget.

Can you show a counter example to help me understand what you mean?

I will later 

I tend to run into difficulty using it any other way, such as a local or global macro, and when in your example there are "variables" in the source eg if your set included {{!!fieldname}} or $tooltip$ etc... in what is to be wikified.

You’re saying that you can’t use the wikify widget within a macro? I’m not sure what you’re getting at here. Perhaps some examples would help?

Will show later but see buttons below 

It is clear we need to fill out the documentation Pragma and Wikify Widget, sometimes so people know this does not promise things it may "on the surface"

Any help with that is welcome.

Will do 

• Is using \import filter more efficient than global macros or the new view only macros tag

They’re all the same mechanism under the hood: the importvariables widget.

I understand this, but the other mechanisms are often applied to every tiddler, where as \import variables may apply them only when specifically named, so in some cases I imagine it more efficient?

 

• Where are the rules documented? Are they a one for one match with these names $:/core/ui/ControlPanel/Parsing

The pragmas are documented here:

https://tiddlywiki.com/#Pragma

This mentions \rules and not much more. Is there more doco somewhere? That is why I ask are the rules what we see here $:/core/ui/ControlPanel/Parsing?

\rules for adjusting the set of rules used to parse the text

No rules names, except not mentioned etc..?

I found the "undocumented "[wikiparserrules[]]" filter operator

That’s a filter operator, which are nothing to do with pragmas.

Well surely it does relate to the rules pragma, I thought this is a way to access all the defined rules, which are not documented in the \rules pragma or anywhere else.

\rules except table

Where table is in the wikiparserrules output and also found listed here Block Parse Rules

Is there an include parameter or just by naming the rule?

 

Wikify

• With the Wikify widget is there a way for us to apply the pragma rules to the result?

That’s exactly what my example in my previous post showed.

Opps Yes, Use an additional set widget to pre-pend the define, even pragma, to the text before wikifiction. 

• The Wikify Widget would benefit from examples and ideally through the various templates available already in the core, so people can generate and store tiddlers in alternate formats that already operate under export, save and other options.

What do you mean by “templates” here? What’s the connection between templates and the wikify widget? Reusing the existing export filters doesn’t have anything obvious to do with the wikify widget.

I remain awfully confused here, I think I will present another thread in time. I have tried to use the $:/core/templates tiddlers to render a tiddler and save the result as a tiddler, copy to clipboard or file.

 

• Storing the result of a wikify operation in the current wiki is also of substantial value (I would detail if asked)

As you know, changes can only be made to the wiki via action widgets, are you experiencing difficulties doing that? Again, perhaps an illustration would help.

I would be happy to make a button to do this, But I seem to be missing something. They often instead include the code unwikified. I will start a new thread when I have worked through the details.

But I would have thought the below buttons would work, when json-here returns the formated tiddler.

\define json-here()
<$wikify name=output text="""{{||$:/core/templates/json-tiddler}}""">
<$macrocall $name="copy-to-clipboard" src=<<output>>/>
</$wikify>
\end
<<json-here>>

<$button>
<$action-createtiddler $basetitle="Test"  text=<<json-here>>/>
Go1
</$button>

<$button>
<$action-createtiddler $basetitle="Test"  text={{||$:/core/templates/json-tiddler}}/>
Go2
</$button>

 

Parsing is of substantial interest to many, so I hope eventually we can open this to users, especially to add our own markup.

Beyond your desire to add custom markup without having to use JavaScript, what else is needed to open parsing to users?

I and mario have being looking at this with the leading dotparagraph and trailing spacespace. Of course such markup will not be universal when tiddlers are transferred but they can be made to "fail gracefully"  when not defined.

Good stuff, 

I just wonder if there would be a way to specify our own markup that that results in a block or inline html tag being applied at render time? Primarily to allow us to utilise other html tags that are not yet included in the wikitext standard, and without making the text harder to follow with html embedded in it. For example we need a plugin to use the html details tag, when an additional markup similar to ; and : `<dl><dt>` tags would do the same. There are plenty of special characters available.

Perhaps even a generic way to to feed block and inline content into a named html tag pair would open up html features, without needing to wait for a widget to be written. The advantage of markup is it keeps the text simple, shorthand.

Imagin with the dot paragraph Idea I could have indicated that a line was to be wrapped in `<p></p>` and we would not need to incorporate it into wikitext.

Thanks

Tony