[TW5] enhance regexp or new filter "pick"

[Stephan Hradek]

Hi!

I wanted to create a new filter called "pick" which is exactly like regexp, except that it "picks out" the matched parts instead of finding matching tiddlers.

Motivation behind this is: I have some tiddlers containing links to github and I wanted to pull out all these links.

As I failed to create a plugin for doing this ad-hoc (too long absence from tiddlywiki), I shamelessly enhanced regexp to do what I want.

The enhancement is backwards compatible as long as there are no capturing groups in

Pro: Not yet another filter to learn
Cons: Will break current tiddlywikis having capturing groups

Question is: What shall I do with it? Can someone show me how I can create a plugin so that I can create a "pick" filter? Shall I create a pull request for the enhanced regexp?

[PMario]

Hi Stephan,

I'm not sure, but imo this pending PR https://github.com/Jermolene/TiddlyWiki5/pull/3003  does a similar thing. ... So it would be nice to +1 the PR and eventually check your code, if the functionality could be merged, or if it really is a new operator.

There is a second PR: https://github.com/Jermolene/TiddlyWiki5/pull/2963 which may be similar. ... +1 this one too!

-m

[Stephan Hradek]

Am Montag, 12. Februar 2018 11:36:42 UTC+1 schrieb PMario:

There is a second PR: https://github.com/Jermolene/TiddlyWiki5/pull/2963 which may be similar. ... +1 this one too!

Especially that one looks pretty similar to my code, but seems to do something differently.

Haven't figured out how to completely see that code and include it in my TW to try & test it.

Example of how mine would work:

Exampletiddler contains

Auf der Mauer, auf der Lauer sitzt 'ne kleine Wanze

First example regexp:

<$list filter="[regexp:text[auf der (\w+)]]"/>

will return "Lauer" - as "auf" doesn't match "Auf"

Second example regexp:

<$list filter="[regexp:text[(?i)auf der (\w+)]]"/>

will return "Mauer" - as "auf" now matches "Auf" and we didn't use global search

Third example regexp:

<$list filter="[regexp:text[(?gi)auf der (\w+)]]"/>

will return "Mauer" and "Lauer" - as we did a case-insensitive global search

Last Example

<$list filter="[regexp:text[(?i)auf der \w+]]"/>

will return "Exampletiddler" - as we have no capturing group and so the title is returned.

[PMario]

On Monday, February 12, 2018 at 2:38:49 PM UTC+1, Stephan Hradek wrote:

Last Example

<$list filter="[regexp:text[(?i)auf der \w+]]"/>

will return "Exampletiddler" - as we have no capturing group and so the title is returned.

Is it possible to have several capturing groups?

-m

[Stephan Hradek]

Yes

But now that you ask it… Esch group will be a seperate tiddler, so the connection, that they belong to the same source tiddler, is lost. This is (was) fine for my usecase, but now that I think about it…

[Stephan Hradek]

Am Montag, 12. Februar 2018 17:24:48 UTC+1 schrieb PMario:

Is it possible to have several capturing groups?

I think I found a good solution.

1. filter "pick:field[regexp]"
   Will pick every captured group as a new tiddler. If there is no group it acts exactly like regexp
2. filter "pickgroups:field[regexp]"
   Will capture every group into one tiddler, joined with character \x00, thus making it usable by
3. widget "splitgroups"
   which will split the <<currentTiddler>> at character \x00 into a set of variables, possibly "sweetened" by some text

Examples

Exampletiddler contains

Auf der Mauer, auf der Lauer sitzt 'ne kleine Wanze

First example

<$list filter="[pick:text[auf der (\w+)]]"/>

will return "Lauer" - as "auf" doesn't match "Auf"

Second example

<$list filter="[pick:text[(?i)auf der (\w+)]]"/>

will return "Mauer" - as "auf" now matches "Auf" and we didn't use global search

Third example

<$list filter="[pick:text[(?gi)auf der (\w+)]]"/>

will return "Mauer" and "Lauer" - as we did a case-insensitive global search

Fourth example

<$list filter="[pick:text[(?i)auf der \w+]]"/>

will return "Exampletiddler" - as we have no capturing group and so the title is returned.

Fifth example

<$list filter="[pick:text[(?i)(auf der) (\w+)]]"/>

will return "Auf der", "Mauer", "auf der" and "Lauer". Each one as a seperate tiddler

Sixth example

<$list filter="[pickgroups:text[(?gi)(auf der) (\w+)]]">

<$splitgroups a="a: $1" b="b: $2">
<<a>>, <<b>>
</$splitgroups>
</$list>


will return

"a: Auf der, b: Mauer"
and
"a: auf der, b: Lauer"

[@TiddlyTweeter]

Ciao Stephen

I don't understand the mechanics. But IF what you mean is that we could extract (say) every web-address in text or fields in a whole TW to a list I'm all for it.

For example: In my use-case it would allow me to then pass that list to a link-checker.

Josiah

[Stephan Hradek]

Am Mittwoch, 14. Februar 2018 15:06:00 UTC+1 schrieb @TiddlyTweeter:

Ciao Stephen

I don't understand the mechanics. But IF what you mean is that we could extract (say) every web-address in text or fields in a whole TW to a list I'm all for it.

Yes, we could. Rought idea, using the new ":value" suffix for "each":

<$list filter="[pick[(?g)(https?://^\S+)]each:value[][">

[Mat]

[jumping in from holidays so brief]

Stephan, I believe this could be very useful! I made the Cherrypicker plugin a long time ago and I'm guessing your solution would make for a much better solution than mine to accomplish things, including the ones not yet possible with Cherrypicker like:

• text summaries - use marks for highlighting in text and get neat summaries elsewhere
• attention notifications - @bobWow!
• hashtags - #twittlywiki
• element extractor - treat(!) markup or code "identifiers" as cherry marks to get code aggregations, e.g...
  • CSS definition overviews, e.g all tc-tiddler-xx {......}
    • ...to e.g identify "last call" overwrites
  • Macro index - list all macros and show which tids define them
  • HTML elements, e.g all <table> or #id...

What would you say - would your pick operator enable these things?

<:-)

[Stephan Hradek]

Am Samstag, 24. Februar 2018 17:40:08 UTC+1 schrieb Mat:

What would you say - would your pick operator enable these things?

I'm not sure as I'm missing context and examples for all your keywords.

Anyhow: I intend to offer a plugin soon. So you can check yourself.

[Mat]

@Stephan

Any news on this? Is your plugin created? Really looking forward to it and I think it will be very useful :-)

<:-)

[Mark S.]

The code in the PR I submitted does as you specify in your examples, except that in the last example returning the title if nothing is found would be a non-sequitur, and should probably be done through some other mechanism.

Jed's code is probably better and does more, but I've attached mine for what it's worth. Drag and drop into your tw, save, reload (back-up is good too). Remember to use regexps (with an "s" on the end) and not regexp.

-- Mark

[Stephan Hradek]

In case someone is interested: https://skeeve.github.io/tw5-plugins/

[Stephan Hradek]

Am Dienstag, 3. April 2018 16:30:43 UTC+2 schrieb Mat:

@Stephan

Any news on this? Is your plugin created? Really looking forward to it and I think it will be very useful :-)

https://skeeve.github.io/tw5-plugins/

[@TiddlyTweeter]

I can't find your plugin library???

[Stephan Hradek]

As usual:

1. Drag & Drop the link into your tiddlywiki.
2. Import the library (link)
3. Go to Settings ($:/ControlPanel)
4. Click on Plugins
5. Get more plugins
6. Open my plugin library
7. install