[TW5] How force evaluation to get a resulting string?

[Mat]

It just seem so fundamental and it has popped up before... but I can't find any solution:

The stored result (regardless if using a direct macro call or via the set variable) gives the result to be a copy of the literal macro string rather than the evaluated macro result.

\define c() <$count filter="[prefix[New]]"/>

<$set name="c2" value=<<c>>>
<$button>
<$action-setfield result=<<c>> result2=<<c2>>/>
x
</$button>
</$set>

What can be done? Couldn't there be some generic macro parameter to state if the return value should be the evaluated result?

Ideas?

<:-)

[Eric Shulman]

On Sunday, February 28, 2016 at 10:18:37 AM UTC-8, Mat wrote:

It just seem so fundamental and it has popped up before... but I can't find any solution:

Your confusion stems from the fact that, despite the ability to have parameters, macros are not "functions" in the usual sense.

Macros don't "run" their content and "return the output".  A macro is simply a text substitution mechanism, and the ONLY action it performs is to replace any embedded parameters and/or variables -- the $param$ and $(variable)$ syntax -- and then return the modified content for further handling by the calling context. Other than this replacement processing, macros *never* "evaluate" (aka, "wikify") their output; that is *always* the responsibility of the context in which the macro is invoked.

The stored result (regardless if using a direct macro call or via the set variable) gives the result to be a copy of the literal macro string rather than the evaluated macro result.

\define c() <$count filter="[prefix[New]]"/>

<$set name="c2" value=<<c>>>
<$button>
<$action-setfield result=<<c>> result2=<<c2>>/>
x
</$button>
</$set>

What can be done? Couldn't there be some generic macro parameter to state if the return value should be the evaluated result?

As noted above, the "evaluation" of the macro results depends entirely on the context into which it is returned: the macro content *might* be rendered (and thus, "wikified"), or it might be used as the value of a widget parameter, or perhaps saved as-is to a tiddler field, etc.

The immediate problem here is that you are trying to use the $count widget to get a value that you want to store in a variable.  This won't work, because the $count widgets job is to simply display the total number of items in the filter result.  Thus, while it produces rendered output, it doesn't actually "return" a value that can be stored.

What you really need is a [count[]] *filter*... then, you could use the $set widget to store the results of the filter.

Here's the code for a simple "count" filter:

/*\
title: filters/count.js
type: application/javascript
module-type: filteroperator
author: EricShulman
description: Filter operator for getting the number of items in a list
\*/
(function(){
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";

exports.count = function(source,operator,options)
   { var c=0; source(function(tiddler,title) { c++; }); return [c.toString()]; };

})();

Put the above code into the text field of a tiddler, and set fields

   type="application/javascript"

and

   module-type="filteroperator"
Then save-and-reload for the new filter to be available.

Here's how you would use it:

<$set name="c" filter="[prefix[New]count[]]"
   <$button> X
      <$action-setfield result=<<c>>/>
   </$button>
</$set>

enjoy,

-e

[Mat]

Eric - thank you for a very informative and 'solving' reply! :-)

Note for anyone copying the solution; there is a missing ">" in that last code, i.e it should be;

<$set name="c" filter="[prefix[New]count[]]">


...

General thoughts on the matter:

It is depressing to realize that this basic problem is something that requires special js code and I therefore simply cannot solve myself (and this after hours and hours of trying different circumventions). So - in spite of Eric explaining that:

As noted above, the "evaluation" of the macro results depends entirely on the context into which it is returned: the macro content *might* be rendered (and thus, "wikified"), or it might be used as the value of a widget parameter, or perhaps saved as-is to a tiddler field, etc.

...AFAICT it would still be very useful with some kind of "assuming" wikifying implementations for this that evaluates straight off and presents the same ouput string as would be presented on the screen. For example the macro in the original question above is un-ambiguous, isn't it? The evaluation command could perhaps be a "macro wrap" or a maybe a parameter like;

\define eval foo() ...

or

\define(e) foo() ...

Maybe even alternative parameters to specify what kinds of content should be evaluated (parameter substitution, internal macro calls, internal widgets...)

Thoughts?

Again, thank you Eric for your reply above!!!

<:-)

[Scott Simmons (Secret-HQ)]

Hi, Mat —

Tobias's setvars plugin may be able to come to your rescue here.  With it installed you could write something like this:

\define c() <$count filter="[prefix[New]]"/>

<$setvars _count=<<c>> count="count">
<$button>
<$action-setfield result=<<count>> result2=<<count>>/>
x
</$button>
</$setvars>

Setvars uses underscore-prefixed "attributes" to capture dynamic content as literal strings of text and then sets "variables" using combinations of the specified attributes.  (You can also set variables directly if they're text strings, but I use it regularly to capture wikified output and set it as a variable to use in something that follows.)

Here, _count=<<c>> gets the wikified output of your <<c>> macrocall, and count="count" sets the variable count (which you refer to with macrocall <<count>> within the scope of the current instance of <$setvars>) to be that text string.

You could also use <$setvars> to build out the entire <$action-setfield> widget string:

\define c() <$count filter="[prefix[New]]"/>

<$setvars
  _count=<<c>>
  button-action="""\<$action-setfield result="\ count \" result2="\ count \"/>\"""
>
<$button>
<<button-action>>
x
</$button>
</$setvars>

The backslashes set off literal text in a variable declaration, so the variable button-action here is being built from literal strings and variables (well, one variable here: count).

You can get setvars here:

https://tobibeer.github.io/tw5-plugins/#setvars

It's one of those plugins I sometimes think might be worth incorporating into the core, if only because I use it for virtually everything.

[Eric Shulman]

On Sunday, February 28, 2016 at 2:31:47 PM UTC-8, Mat wrote:

Eric - thank you for a very informative and 'solving' reply! :-)

...

It is depressing to realize that this basic problem is something that requires special js code and I therefore simply cannot solve myself (and this after hours and hours of trying different circumventions).

It's not really "special js code".  The <$count> widget has much more involved js code than this tiny filter does.... and certainly less code than would be needed to extend the core macro processing to "evaluate" the macro output and extract a return value from the result.  You should think of the "count" filter as merely an add-on enhancement to the set of filters you have available.

The real problem you encountered is that the "count" filter isn't in the standard core set of filters, so you were looking for a workaround based on re-purposing existing features (i.e., the $count widget).  However, I think what you are trying to "kill a fly with a sledgehammer."  You are proposing that we radically extend the purpose and behavior of the macro mechanism on a system-wide basis, just so you can re-use an existing widget in a way that widgets aren't intended to be used.

I'm also wondering how such an extended macro processor would handle refresh events?  To "evaluate and return" macro content, the core processing would need to generate a separate, temporary parse tree for the macro content (after substitution of params and variables), extract the "output value" from the tree and then discard the parse tree before returning the extracted content.  The problem here is that without preserving the parse tree, there's no way to know that there are widgets *inside* the macro that need to be refreshed.

In my opinion, this entire approach seems like it would be a "can of worms", creating more problems than it solves.

-e

[Mark S.]

Eric, you're seeing this as someone who can whip up a special filter any time you need one. For the rest of us, how to make a filter isn't even documented. This isn't a one time problem that Mat has with his particular use case, it's a problem that comes up over and over whenever any one tries to push the boundaries even slightly.  For instance, AFAIK, there is no filter for comparison operations. Reporting on tiddlers that are newer than some arbitrary date, or that have an indexed field greater some number would be useful in many situations. I put out a request for such a filter in July 2015 and there was interest ... but no follow up. I don't want to nag anyone -- I want to be able to roll my own solutions like it was possible with your InlineJavascriptPlugin.

Thanks!
Mark

[Jed Carty]

I did make some greater than and less than filters for comparisons here. And for the original topic I ran into the same thing a while ago and made the action-storecount widget as part of my mathything plugin.

[Eric Shulman]

On Sunday, February 28, 2016 at 10:02:57 PM UTC-8, Mark S. wrote:

Eric, you're seeing this as someone who can whip up a special filter any time you need one. For the rest of us, how to make a filter isn't even documented. This isn't a one time problem that Mat has with his particular use case, it's a problem that comes up over and over whenever any one tries to push the boundaries even slightly. 

For instance, AFAIK, there is no filter for comparison operations. Reporting on tiddlers that are newer than some arbitrary date, or that have an indexed field greater some number would be useful in many situations. I put out a request for such a filter in July 2015 and there was interest ... but no follow up. I don't want to nag anyone -- I want to be able to roll my own solutions like it was possible with your InlineJavascriptPlugin.

There are two issues here:

1) What is the best course of action when the functionality you want isn't *currently* provided by the core.  Some possible responses:

   * find a workaround using different core functionality

   * open an 'issue' on github and request the missing feature

   * write your own custom javascript macro, filter, widget, etc.

2) How can people with "limited" programming skills extend TW core to fit their particular use-cases.

   * You point to my InlineJavascriptPlugin as an acceptable way to "roll your own solution" (for TWClassic only).  However, this means that you are already able to write your own javascript code.  In which case, there's VERY little difference between using something like InlineJavascriptPlugin, and writing a TW5 javascript macro.  There's just a little bit of a different 'framework' that surrounds your custom code.

The easiest way to get started is to pick any javascript macro definition in the core, clone that tiddler, and then edit the definition to replace the existing code with your own custom code.  For example, the <<now>> macro is a good one to start with:

   http://tiddlywiki.com/#%24%3A%2Fcore%2Fmodules%2Fmacros%2Fnow.js

Even without documentation, it's a very simple module, and it's fairly clear what the code is doing.

There are only four things you need to change:

1) set the title of the tiddler

2) give the macro a name:

exports.name = "MACRONAME"

3) define the names of any params

exports.params = [ {name: "PARAM1"}, {name: "PARAM2"} ];

note: if your macro uses no parameters, define an empty param array:

exports.params = [ ];

4) define the macro 'run' function, passing in the named params, which are used to generate text output that is returned for further processing depending on the calling context (e.g., rendering, use as a widget parameter, etc.)

exports.run = function(PARAM1,PARAM2) {
 ... your javascript code here ...
 return ...some computed text value...
};

That's it.  Just save-and-reload for the macro definition to take effect, and then you can write <<MACRONAME PARAM1:"foo" PARAM2:"bar">> to invoke your own custom javascript macro.  WHEE!  What Fun!

enjoy,

-e