Filter operators: Official documentation

[Mohammad Rahmani]

I was reading the some filter operator they looks like this

addsuffix Operator

3rd February 2015 at 10:02pm

Filter OperatorsString Operators

input	a selection of titles
output	the input, but with S added to the end of each title

Examples

How can I realise I should use this filter operator like below:

<$list filter="[[inputlist]addsuffix[mysuffix]]">

How should I understand it needs an input parameter which is different from input from a previous step?

In my opinion filter operators are an ESSENTIAL part of Tiddlywiki which is also the most confusing part!

My questions are:

1. DO you have better references for using and working with filters? Please share links, resources, ...

2. What is your experience using official documentation for filter operators?

3. How do you recommend to improve these documentations?

Some references to help improve the filter operators documentation

[1]. TW-Scripts

[2]. PowerSearch

[3]. Grok Tiddlywiki

[4]. Tobias Beer Filter References

[5]. S.S draft for restructuring TW documentation

[6]. Tones GitWiki

Best wishes

Mohammad

[Stobot]

Mohammad, you point to something I personally agree is a learning curve (and I think unnecessarily so) for those who use similar "formula languages" to those I'm used to. My background is more Windows/Office stuff, so Excel, VBA, M, DAX, R, AutoIT etc. and there is typically a nearly standard way of documenting the functions (widgets in our case). 

They (like your example) start with a syntax string. I think this really helps illustrate better HOW to use it. 

For example the common XLOOKUP function in Excel is documented like: 

=XLOOKUP(lookup_value, lookup_array, return_array, [if_not_found], [match_mode], [search_mode])
...and then immediately below that is a table with each of the parameters, whether they're required, what each option is and does, and what happens if it's omitted

...and then examples immediately follow (not sure why the TiddlyWiki examples are often (though not always) in separate tiddlers. 

Even just seeing in order how it's all pieced together, and then note the square bracket parameters which seems nearly universally to mean optional (at least in the languages described above). In some languages they even put the default right in the syntax line. That was Excel, but a similar pattern seems to be present in at least the other languages I use. 

So, for most widgets I'd suggest it would be easier to start with a similar syntax pattern before going into parameters. Now picking what brackets etc. to use in TiddlyWiki is a little more difficult since all of them are already actually used, but taking a crack at it... for example the <$wikify> would get a:

<$wikify name=<variable name> text=<text to wikify> [type=<ContentType (default text/vnd.tiddlywiki)>]  [mode=<parse mode (default:block)>] [output=<output type (default:text)>] >

    Text where <variable name> is available

</$wikify>

... and then the table for each of those like the documentation already has largely.

Now your example, a filter operator is slightly different, but a similar methodology could apply. Maybe include the whole list piece like you do already

<$list filter="[<input>addsuffix<suffix>]">

    The output of each input from the filter with the additional suffix

</$list>

... and then a table for input, suffix, and output. Input and output already have something, but a row for suffix could be there saying "string of text to add" or something.

Anyways, I'm sure that can be refined, but circling back to your specific questions:

1. I don't have better resources, I find that the example pages are currently where I start (I guess because I think syntax first) and then come back

2. Experience seems similar to yours. I'm often very confused until I see enough examples (especially the ones with all of the colons - though love how powerful they are)

3. Above is at least an idea of how to improve

[TiddlyTweeter]

Ciao Mohammad

It is an interesting issue. 

• I AGREE the base docs are not enough for working everything out. 

• I DISAGREE that the solution to that problem should be in the main documentation!

WHY? 

Because IF you added all possible combination examples it would inflate the main reference site, tiddlywiki.com, to be SO vast it would become unusable.

IMO, Jeremy Ruston has done a very good job on documentation. 

It is true his style is somewhat gnostic :-)

The original minimalist! :-) 

But it is also true that he has ASTUTE BREVITY.

IMO this issue should be about SECONDARY documentation.

At the moment everything focuses on tw.com. 

I'm NOT convinced that is the right approach for final practical, fully expanded, documentation.

It seems to me that we need more DIFFERENTIATION.

AND recognizing better SUPPLEMENTARY documentation that covers specific aspects of TiddlyWiki in more detail would be best.

I hope my points make sense, even if you don't agree.

TT

[Álvaro]

I agree, it is confusing especially if you don´t know/remember the filter sintax, in this case the filter steps. And how you say input is confusing if you don't know/remember that this input is the output of other filter step (or filter run). If i don't understand bad the list of titles between square bracket is a step. I like the idea the "constructor example" in a list (or in filtering transclusión), I don't know if it is better do it with styles in font or in a table. I think that also differenciate the parts of constructor, necessary vs optional.

Also i agree with Stobot. "Example constructor" will be good point. I think also it happens with macro docs (for example in "toc" macro). I working in a macro and i doubt how write the optional parameters to do understandable to everyone.

TiddlyTweeter, I agree with you in that brevety is good, but with some reminders it would be better and not much longer.

[Saq Imtiaz]

@Mohammad

<$list filter="[[inputlist]addsuffix[mysuffix]]">

How should I understand it needs an input parameter which is different from input from a previous step?

Can you please explain what you mean here? I have re-read it a few times and don't understand.

Thank you. 

[Mohammad Rahmani]

Hi Saq,

 

I mean this https://tiddlywiki.com/prerelease/#Filter%20Parameter

The reader cannot understand below logic for the 

addprefix Operator

3rd February 2015 at 9:48pm

Filter OperatorsString Operators

input	a selection of titles

output	the input, but with S added to the start of each title

you have <input> filter-ops<filter-parameter>

Example (only demo):

input: [tag[HelloThere]]

filter-op: addprefix

filter-param: student

then

<$list filter="[tag[Hellothere]addprefix[student]" ....

 

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/ed6d50a4-3a3a-43df-b439-4e78dbd20eb0n%40googlegroups.com.

[Mohammad Rahmani]

@Alvaro,

Thank you!  you explained what I mean

Best wishes

Mohammad

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/6392aca0-770a-4ccc-9d3c-42f503e68d88n%40googlegroups.com.

[Mohammad Rahmani]

On Tue, May 25, 2021 at 4:55 PM TiddlyTweeter <Tiddly...@assays.tv> wrote:

Ciao Mohammad

It is an interesting issue. 

• I AGREE the base docs are not enough for working everything out. 

• I DISAGREE that the solution to that problem should be in the main documentation!

Let me think the problem is in the main documentation. From the filter operator docs, I cannot understand what it does!

 

WHY? 

Because IF you added all possible combination examples it would inflate the main reference site, tiddlywiki.com, to be SO vast it would become unusable.

Agree! but I think the format of docs for filter ops need some more clarification! See example in my above post!

One other example

subfilter="[tag[HelloThere]] +[putfirst[]]"

Did you know, in the above example putfirst takes the last item of  [tag[HelloThere]] why? As a newbie how can I understand this?

Another example

putafter Operator

8th November 2015 at 8:48am

Filter OperatorsListops OperatorsOrder Operators

input	a list of items
suffix	an integer N, defaulting to 1
output	re-ordered list of items

putafter what? I need to know putafter needs a filter

IMO, Jeremy Ruston has done a very good job on documentation. 

You know, I am a big fan of TW!

 

It is true his style is somewhat gnostic :-)

The original minimalist! :-) 

But it is also true that he has ASTUTE BREVITY.

I think simple is the best but clear!

IMO this issue should be about SECONDARY documentation.

I think we need to help.

 

At the moment everything focuses on tw.com. 

I'm NOT convinced that is the right approach for final practical, fully expanded, documentation.

It seems to me that we need more DIFFERENTIATION.

AND recognizing better SUPPLEMENTARY documentation that covers specific aspects of TiddlyWiki in more detail would be best.

I hope my points make sense, even if you don't agree.

Sure! Thank you

TT

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/9b4500a8-cf8e-46f5-bdc4-2c8f66ff0269n%40googlegroups.com.

[Mohammad Rahmani]

Hi Stobot,

You clearly explained the problem and have suggestions for improvement!

1. I asked the question to see if the problem is only me or others have the same experiences.
2. I do not expect the core developer to correct the docs as they are very busy!
3. I think a better template for filter operator is needed in such a way newcomer can understand the syntax better
4. I agree with your suggestion. 

WE NEED TO HELP!

Best wishes

Mohammad

On Tue, May 25, 2021 at 4:05 PM Stobot <sto...@gmail.com> wrote:

Mohammad, you point to something I personally agree is a learning curve (and I think unnecessarily so) for those who use similar "formula languages" to those I'm used to. My background is more Windows/Office stuff, so Excel, VBA, M, DAX, R, AutoIT etc. and there is typically a nearly standard way of documenting the functions (widgets in our case). 

They (like your example) start with a syntax string. I think this really helps illustrate better HOW to use it. 

For example the common XLOOKUP function in Excel is documented like: 

=XLOOKUP(lookup_value, lookup_array, return_array, [if_not_found], [match_mode], [search_mode])
...and then immediately below that is a table with each of the parameters, whether they're required, what each option is and does, and what happens if it's omitted

...and then examples immediately follow (not sure why the TiddlyWiki examples are often (though not always) in separate tiddlers. 

Even just seeing in order how it's all pieced together, and then note the square bracket parameters which seems nearly universally to mean optional (at least in the languages described above). In some languages they even put the default right in the syntax line. That was Excel, but a similar pattern seems to be present in at least the other languages I use. 

So, for most widgets I'd suggest it would be easier to start with a similar syntax pattern before going into parameters. Now picking what brackets etc. to use in TiddlyWiki is a little more difficult since all of them are already actually used, but taking a crack at it... for example the <$wikify> would get a:

<$wikify name=<variable name> text=<text to wikify> [type=<ContentType (default text/vnd.tiddlywiki)>]  [mode=<parse mode (default:block)>] [output=<output type (default:text)>] >

    Text where <variable name> is available

</$wikify>

very nice!

... and then the table for each of those like the documentation already has largely.

Now your example, a filter operator is slightly different, but a similar methodology could apply. Maybe include the whole list piece like you do already

<$list filter="[<input>addsuffix<suffix>]">

    The output of each input from the filter with the additional suffix

</$list>

<$list filter="[<input>addsuffix:<suffix><filter-op-param>]">

    The output of each input from the filter with the additional suffix

</$list>

yes, we have little chars to use for syntax demonstration for example optional param, user given input, required param,...

 

... and then a table for input, suffix, and output. Input and output already have something, but a row for suffix could be there saying "string of text to add" or something.

Anyways, I'm sure that can be refined, but circling back to your specific questions:

1. I don't have better resources, I find that the example pages are currently where I start (I guess because I think syntax first) and then come back

2. Experience seems similar to yours. I'm often very confused until I see enough examples (especially the ones with all of the colons - though love how powerful they are)

3. Above is at least an idea of how to improve

very nice!

--

You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/8698e7d4-be6d-4701-8a62-62077c9628a0n%40googlegroups.com.

[si]

@Mohammad I haven't read all of this thread, but I did notice that the descriptions of filter expressions in the pre-release documentation seem to be missing important info. Eg. they should have a "purpose" and "parameter" field: https://tiddlywiki.com/#putafter%20Operator

[PMario]

The "operator" docs is a formal description. ... The (Examples) should be for the "understanding". ... The problem is, we do need more "real world" examples, + explanation. That's all.

I think everyone is free to provide "pull requests" at GitHub, or post examples in the GG, that can be transferred to the docs.

PRs are preferred, since it makes live much easier for developers.

just my thougths

mario

[Mohammad Rahmani]

Hi Mario,

Ah, I did not know that! Is there any reason for that? (missing purpose and parameter in https://tiddlywiki.com/prerelease)

On Tue, May 25, 2021 at 8:10 PM PMario <pmar...@gmail.com> wrote:

The "operator" docs is a formal description. ... The (Examples) should be for the "understanding". ... The problem is, we do need more "real world" examples, + explanation. That's all.

I think everyone is free to provide "pull requests" at GitHub, or post examples in the GG, that can be transferred to the docs.

That is true! Maybe the proposal by stobot above works here to have more clear docs for newcomers!

PRs are preferred, since it makes live much easier for developers.

just my thougths

mario

Thank you!

 

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/c463570c-fd8f-4a6e-a13a-9f1cbe0f8ba2n%40googlegroups.com.

[Jeremy Ruston]

Just to say that I agree that the widget documentation could do with improvement. It’s worth making a distinction between two different kinds of improvement:

* Documentation for individual widgets that need clarification or more examples, such as the examples Mohammad gave. These sort of improvements can often be a PR to a single file, a workflow that we’ve worked hard to optimise

* Improving the standard conventions/macros/formatting that is used for all widgets. For example, I really like Stobot’s suggestion for improving the presentation of widget syntax

The catch is that those improvements in the second category need to be applied consistently to all the existing widgets, which is a much bigger undertaking. However, the benefit is immense, and so I’d encourage anyone who is interested to volunteer; you can expect plenty of help and support.

Best wishes

Jeremy.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/CAAV1gMAdPsZ8C-p0XsZ7gNqZezTM1dw_CVYsYE1xodpFan%3Dc%2BA%40mail.gmail.com.

[Stobot]

I agree that consistency is more than half the battle Jeremy. Looking through the documentation at least the filter operators are somewhat templated (content in the boxes are fields) - which means as a shortcut we could try writing up something in fields and then getting input on formatting later. The widgets don't seem to be done in the same way, but that might be a good idea for future flexibility. 

I'm honestly intimidated by the whole Github / PR thing for making suggestions, but could try a TiddlySpot thing over time or help someone else who knows what they're doing if that's helpful. 

[Mohammad Rahmani]

@Jeremy

Many thanks for your clarification and support! I hope I can help in this regard! 

@Stobot

Your idea is great and I would like to help! It is good to prepare a small demo for one case to see how it works! 

Yes, Tiddlyhost is a good place to start! I would be happy to help you on the GitHub part!

Best wishes

Mohammad

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/9061BDD0-6705-4135-8A27-500587D169D7%40gmail.com.