IDEA: extend [trim[]] filter operator

[Eric Shulman]

Hi all,

In a discussion earlier today (https://groups.google.com/forum/#!topic/tiddlywiki/M8PFeakbp6U), there was a "problem" caused by the use of [removeprefix[...]].

The OP had assumed that this just removed the specified leading text from the items being filtered when, in fact, it also does the matching action of [prefix[...]]

However, it would be helpful if there was a filter actually did *just* remove leading (or trailing) text,

without regard to any matching.  One example of a filter that does this kind of non-matching action

is "[trim[]]" (introduced in 5.1.20), which removes leading/trailing whitespace from the input items,

but doesn't have any effect on items that don't have leading/trailing whitespace.  Thus, a filter containing this:

[[  foobar ]] [[mumble]] [[gronk snork  ]] [[  frotz]] +[trim[]]

becomes just

[[foobar]] [[mumble]] [[gronk snork]] [[frotz]]

My thought is this... we could extend the current [trim[]] syntax to only remove leading

OR trailing whitespace by adding a suffix to the operator:

[trim:prefix[]] would remove only leading whitespace

[trim:suffix[]] would remove only trailing whitespace

and enable removing of other leading/trailing text values, not just whitespace, by adding a parameter to the operator:

[trim[foo]] would remove leading/trailing text "foo"

[trim:prefix[foo]] would remove only leading text "foo"

[trim:suffix[foo]] would remove only trailing text "foo"

The parameter would also be able to handle repeating text.  Thus, you could trim a list of "zero-padded" numbers, like this:

[[123]] [[0.004]] [[0007.0]] [[000089]] [[27.00]] [[05.00]] +[trim[0]trim:suffix[.]]

would produce this result:

[[123]] [[.004]] [[7]] [[89]] [[27]] [[5]]

and something like:

[[  this is some text...  ]] +[trim[ ]trim[.]]

would produce this result

[[this is some text]]

I can think of several instances where such a filter could have saved me untold amount of

effort in handling variations on field values entered by users via an $edit-text widget

Your thoughts?

-e

[Mat]

Good idea but/and I think this request has overlapping use;
Extend removeprefix op with :suffix to make less greedy #4520

Do please comment there if you agree with it.

<:-)

[Eric Shulman]

On Tuesday, June 30, 2020 at 10:30:05 AM UTC-7, Mat wrote:

Good idea but/and I think this request has overlapping use;
Extend removeprefix op with :suffix to make less greedy #4520

That  issue is very similar to my proposal, but would mean adding ":lazy" handling to both removeprefix[] and removesuffix[], and wouldn't address the "repeating character" functionality in my proposal.

I think that extending the trim[] operator makes more sense semantically, in that it already functions as a "modify but don't remove" operation and my proposal merely extends this to allow greater specificity without changing the overall purpose.

-e

[TW Tones]

Eric,

I support your suggestion, in effect the trim default is "leading/trailing whitespace" but you can trim other values with this suggestion.

It is true more often that not removeprefix and removesuffix work for me, but they are long operators. If trim was the equivalent of remove prefix I would like it.

With the increasing use by many, of the tipple curly braces to construct parameters from filters (Variables and References) a "trim/simple" trim operator would make them much shorter and more readable.

From memory unless remove prefix removes the prefix the title will not appear in the result. Trim on the other-hand trims if so, but passes all titles on. I understand that is what Mats "less Greedy" reference is about.

The existing split and join are related functions and I wonder if what you are suggesting is just a subset of some other operators that would be useful.

What you propose can sometimes be achieved with the split operator, but without a way to use white space rules. This is because the split text is removed from the output.

I have always felt it would be nice to have a replace operator replace "string with string" which also understood whitespace so you could say replace whitespace with / to generate a tiddler title, or replace "/" with white-space to extract subtitles. The Question is would this be the way to gain the functions you suggest?

Ideally you trim proposal AND others would be implemented. It feels we have almost all we need but not everything. Workarounds are not helpful if you need a TiddlyWiki degree to design them.

Regards

Tony

[Mark S.]

If we had a regular expression splitter (e.g. PR2963), then you could trim out prefix, suffix, or even create multiple splits from one input string. This would save having to create multiple filters for manipulating strings, especially since any new string operators would have names very similar to existing operators.

[TW Tones]

Mark,

On Wednesday, July 1, 2020 at 1:07:06 PM UTC+10, Mark S. wrote:

If we had a regular expression splitter (e.g. PR2963), then you could trim out prefix, suffix, or even create multiple splits from one input string. This would save having to create 

I agree that would be desirable but maintaining the simpler ones make sense as well, match, split, join, trim etc...

Regards

Tony

[Eric Shulman]

On Tuesday, June 30, 2020 at 8:07:06 PM UTC-7, Mark S. wrote:

If we had a regular expression splitter (e.g. PR2963), then you could trim out prefix, suffix, or even create multiple splits from one input string. This would save having to create multiple filters for manipulating strings, especially since any new string operators would have names very similar to existing operators.

Even though this commit hasn't been accepted into the TWCore, I've been using it since 2017 on a private project.

In that project, I have tiddlers whose title contains a 14-digit date/time stamp like this:

$:/Category/Section/20200616071922/Name/Of/Item/Here

where "$:/Category/Section" can vary (e.g., "$:/Books/SciFi", "$:/Music/Rock", "$:/Photos/Vacation", etc.)

and the date/time stamp shows when that tiddler was added to my collection of stuff,

regardless of when the TWCore recorded that tiddler as *created* or *modified*

and I use the regexps[] filter operator like this:

\define itempattern()  [0-9]{14}$
\define datepattern() ^[0-9]{8}
\define getAllDates()  [prefix<categorysection>regexps<itempattern>regexps<datepattern>!sort[]]

This lets me create a list of items in a selected "categorysection", sorted by the date/time stamps contained in the titles

Although I'd like to see this particular PR (https://github.com/Jermolene/TiddlyWiki5/pull/2963) accepted into the core,

it seems that Jeremy isn't quite satisfied as to "whether this is the best we can do for the core".  He has suggested

that "this PR being made into a plugin, which is the best place for experimentation."

So, the follow up is: can we get this into the "Official TiddlyWiki Plugin Library"?

That would enable people who need this functionality to move forward, but also allow continued investigation

and exploration of alternative solutions for the TWCore.  If, at some future date, this solution is superseded by

a different core solution, it can be retired from the Plugin Library in favor of that new core solution.

In the meantime, the regexps[] filter can be installed manually by copying the code from here:

https://github.com/Marxsal/TiddlyWiki5/blob/regular_expression_strings/core/modules/filters/regexps.js

and creating a tiddler with field module-type = "filteroperator"

-e