SNR: Search and Replace in Tiddler Fields

[Mohammad]

Announcement: SNR2

Update: Mar 7th, 2019

Changes:

• new interface
• regexp flags can be selected among i, g,m but not g and m at the same time.
• minor issues fixed

Update: Mar 6th, 2019

Changes:

• regexp flags added (g, m, i)
• whole words search added
• minor issues fixed

------

Announcement: SNR2

This is a simple script, search-n-replcae (snr2) to search in tiddler contents (including all fields) for a searchValue and replace them with replaceValue.

Demo:

http://kookma.tiddlyspot.com/    (temporarily site )

It uses a JS macro and can search-n-replace whole words and also do case-Insensitive search-n-replace.

Special thanks to Jeremy Ruston for his great solution on retrieving a JS macro result in raw text form and store it in a variable.

I also highly appreciate the invaluable help of Mark S and BTC. This tool is actually a result of collaboration between all these people.

The SNR will be part of Tiddler Commander .

--Mohammad

------

Announcement: SNR1

This is a simple script, search-n-replcae (snr) to search in tiddler contents (including all fields) for a searchValue and replace them with replaceValue.

See the original post here!

https://groups.google.com/d/msg/tiddlywiki/lTYxRbGon1o/V8PVsvt1AgAJ

Demo: https://kookma.github.io/search-n-replace/

https://groups.google.com/d/msg/tiddlywiki/_OnqU7IiB9g/ukSBoPvECAAJ

--Mohammad

[PMario]

On Thursday, February 28, 2019 at 7:54:26 AM UTC+1, Mohammad wrote:

Announcement: SNR

This is a simple script, search-n-replcae (snr) to search in tiddler contents (including all fields) for a searchValue and replace them with replaceValue.

Nice work!

----------- OT ---------------------

It seems the tiddler folding mechanism has been broken with this tiddler.

If you want to make $:/tags/Macro code visible just add the following line at the end of the source code.

<pre><$view field="text"/></pre>

 It won't interfere with the code. BUT be aware, that doesn't work with tiddlers used as templates!!!!!!

have fun!
mario

PS: Kudos go to Eric Shulman

[PMario]

fixed a typo in the last post :)

[Mohammad]

Thank you Mario!

and I appreciate your comment.

Cheers

Mohammad

[Mohammad]

Mario,

 I forgot to say, I am using the below template to have macro visible

https://kookma.github.io/search-n-replace/#%24%3A%2F.mr%2Fviewtemplate%2Fshow-contents-macro-tiddler

--Mohammad

On Thursday, February 28, 2019 at 3:51:14 PM UTC+3:30, PMario wrote:

[Mohammad]

Josiah,

 I can add a simple logic as whole word option but here whole word means a word separated from other part of the text by two spaces.

The problem is, it will fail to distinguish punctuation. So, it can find " test " but not " test!".

Anyway, snr can do many search and replace jobs, but one needs to be careful like the example you gave above!

he in she, shell, shelf, ...

If TW got a Javascript replace function, the code will be much simpler, if not using current TW features the logic will be

difficult to make whole words search.

--Mohammad

On Thursday, February 28, 2019 at 6:19:57 PM UTC+3:30, @TiddlyTweeter wrote:

Mohammad

Really excellent. Its working well and I haven't been able to get it to fail. It will have huge use I am sure.

I do think an issue is UNDO.

Let me show why ...

On a balmy day SHE took advice that HE needed to walk.

If we replace "SHE" with "HE" we get ...

On a balmy day HE took advice that HE needed to walk.

But it can't easily be reversed  just switching "find" & "replace" terms as you'd get ...

On a balmy day SHE took advice that SHE needed to walk.

I think the simplest way to deal with this is that a Tiddler under s-n-r is cloned to a state tiddler before changes occur that can be RESTORED.

What do you think?

Best wishes

Josiah

Could the tiddler/s you working on be 

On Thursday, 28 February 2019 07:54:26 UTC+1, Mohammad wrote:

Announcement: SNR

This is a simple script, search-n-replcae (snr) to search in tiddler contents (including all fields) for a searchValue and replace them with replaceValue.

See the original post here!

https://groups.google.com/d/msg/tiddlywiki/lTYxRbGon1o/V8PVsvt1AgAJ

Demo: https://kookma.github.io/search-n-replace/

--Mohammad

[Mohammad]

Yes, an undo function is possible to not overwrite the original tiddlers.

But I am afraid duplication tiddlers in this way will result in huge file size.

--Mohammad

On Thursday, February 28, 2019 at 6:19:57 PM UTC+3:30, @TiddlyTweeter wrote:

Mohammad

Really excellent. Its working well and I haven't been able to get it to fail. It will have huge use I am sure.

I do think an issue is UNDO.

Let me show why ...

On a balmy day SHE took advice that HE needed to walk.

If we replace "SHE" with "HE" we get ...

On a balmy day HE took advice that HE needed to walk.

But it can't easily be reversed  just switching "find" & "replace" terms as you'd get ...

On a balmy day SHE took advice that SHE needed to walk.

I think the simplest way to deal with this is that a Tiddler under s-n-r is cloned to a state tiddler before changes occur that can be RESTORED.

What do you think?

Best wishes

Josiah

Could the tiddler/s you working on be 

On Thursday, 28 February 2019 07:54:26 UTC+1, Mohammad wrote:

Announcement: SNR

This is a simple script, search-n-replcae (snr) to search in tiddler contents (including all fields) for a searchValue and replace them with replaceValue.

See the original post here!

https://groups.google.com/d/msg/tiddlywiki/lTYxRbGon1o/V8PVsvt1AgAJ

Demo: https://kookma.github.io/search-n-replace/

--Mohammad

[Diego Mesa]

In my mind, an "undo function" is synonymous with "version control". 

[Mohammad]

Hi Diego,

 That's absolutely correct! and seems the safer solution.

--Mohammad

[Mohammad]

Josiah,

 Have a look ate TW roadmap:

https://tiddlywiki.com/static/RoadMap.html

It gives hope for the feature we like!

--Mohammad

On Thursday, February 28, 2019 at 7:24:17 PM UTC+3:30, @TiddlyTweeter wrote:

Diego Mesa wrote:

In my mind, an "undo function" is synonymous with "version control". 

I agree. Kinda. Though likely more short term than normal VC.

Mohammad wrote:

But I am afraid duplication tiddlers in this way will result in huge file size.

I agree it would if it were persistent. I'm thinking more of cloning of just a "start version" that persists only as long as you doing the replace. That when you finish and happy ts deleted?

Best wishes

Josiah

[Mohammad]

Josiah,

 With current situation, my answer is unfortunately NO!

But the other macro I have developed can find and do partial transclusion

https://kookma.github.io/find-macro/

So, a mix of these two can do the job, but needs extra programming to work together.

--Mohammad

On Thursday, February 28, 2019 at 9:00:25 PM UTC+3:30, @TiddlyTweeter wrote:

This is such a good tool Mohammad! 

I ask a final, probably, very difficult, question ...

Can we match CARRIAGE RETURNS? And can we INSERT them in replacement strings?

Here is a simple example ...

<pre>The man was on the burning deck
His light was dim
And he got bogged.</pre>

To do that you need to replace the HTML tags and remove excess carriage returns and replace with WikiText and insert carriage returns to get...

```
The man was on the burning deck
His light was dim
And he got bogged.
```

Could this be possible?

Just wondering

Best wishes

Josiah

[BurningTreeC]

This is such a good tool Mohammad! 

I ask a final, probably, very difficult, question ...

Can we match CARRIAGE RETURNS? And can we INSERT them in replacement strings?

Hi Josiah, an sorry Mohammad, it can handle carriage returns, too.

Just make the inputs where you write the text to replace and the new text with textareas ... give those edit-text widgets the tag="textarea" attribute ... then play around with new lines

[BurningTreeC]

BTC

That is good news!! 

Rather than have a text area could it be done via substitution? 

No that's too complicated I'm not even going to give the try a thought ;)

 

I'm thinking of the complexity of the interface to have a text area just for that. 

For instance could, with the example I gave, this work through the simple input fields ?...

Search ...

\r\r<pre>

Replace ...

\r```\r

Just thinking out loud

Best wishes

Josiah

[David Gifford]

Oh, sorry, I came here because I thought you wrote "ASMR" plugin...

On Thursday, February 28, 2019 at 12:54:26 AM UTC-6, Mohammad wrote:

Announcement: SNR

This is a simple script, search-n-replcae (snr) to search in tiddler contents (including all fields) for a searchValue and replace them with replaceValue.

See the original post here!

https://groups.google.com/d/msg/tiddlywiki/lTYxRbGon1o/V8PVsvt1AgAJ

Demo: https://kookma.github.io/search-n-replace/

--Mohammad

[Thomas Elmiger]

Hi Mohammad, thank you, that looks very useful!

Regarding carriage (really?) returns, Josiah and BTC,

I would prefer a textarea that looks like an input field – but with the option to expand when needed. Like so:

<$edit-text tiddler="test" field="demo" tag="textarea" rows="1" autoHeight="yes" minHeight="1.75em"/>

Simple enough?

All the best,

Thomas

[BurningTreeC]

Hi Mohammad, thank you, that looks very useful!

Regarding carriage (really?) returns, Josiah and BTC,

I would prefer a textarea that looks like an input field – but with the option to expand when needed. Like so:

<$edit-text tiddler="test" field="demo" tag="textarea" rows="1" autoHeight="yes" minHeight="1.75em"/>

Oh yes, that's better. I found it's a problem not knowing if there are newlines in a seamingly empty textarea or not 

[Mohammad]

Hello Thomas,

 Many thanks for your feedback. The problem is with snr as it uses some filter operators not simply work with with regexp expression, if so

I personally prefer to have \t, \n, ... and like that in search area!

Cheers

Mohammad

[Mohammad]

SNR2 announced.

SNR2 uses a small JS macro and can do search and replace in tiddler fields.

It is a flexible tool can perform search-n-replace in case-sensitive, case-insensitive

wholewords, ... manner

It also accept sregular expression so from this point of view is really powerful.

It can search for carriage return using \n  (asked by other users) and much more ...

See original post

https://groups.google.com/d/msg/tiddlywiki/BwgwkxaXY_o/QtibP6v7BwAJ

SNR2 will be merged into Tiddler Commander.

Best regards

--Mohammad

[Mohammad]

Many Thanks!

Yes, I checked and it seems `>` break the codes!

I will have look to see how I can fixed it.

Cheers

Mohammad

On Sunday, March 3, 2019 at 2:43:54 PM UTC+3:30, @TiddlyTweeter wrote:

Ciao Mohammad

I'll do lots of testing of SNR2 on  http://kookma.tiddlyspot.com/  and report back. It looks very good and very powerful!

The is one BUG I found already. You can easily see it. Just enter ">" in the search box and it breaks the script like this...

Best wishes

Josiah

[Mohammad]

Hi Josiah

Many thanks. I will upload a new update.
I fixed the issue. I appreciate your time for evaluating the code and welcome your write up.

Best
Mohammad

[Mohammad]

Hello Josiah,

 Thank you for your comments!

SNR2 simply use the JS replace function. So, everything is possible with JS replace theoretically should be possible with SNR2.

By the way I will investigate the case! 

--Mohammad

On Monday, March 4, 2019 at 11:06:10 PM UTC+3:30, @TiddlyTweeter wrote:

This is a footnote. Just FYI.

Regular Expressions are one of the few things I understand well. 

So IF anything I write about them seems difficult to understand please ask for clarification.

Best wishes

Josiah

[Mohammad]

Hi Josiah,

 Seems it is true, have a look at

https://www.regular-expressions.info/replacenonprint.html

--Mohammad

[Mohammad]

Yes, I got it. I am experimenting to see how resolve the issue.

The same is true for tab character \t

--Mohammad

On Tuesday, March 5, 2019 at 12:48:57 AM UTC+3:30, @TiddlyTweeter wrote:

Mohammad

The issue  is simply that we can get the MATCH well. But on REPLACE we might need to INSERT additional things that MATCH can't do.

Hope this is clear!

Josiah

[Mohammad]

Yes, It is possible!

Save and reload and it should works.

--Mohammad

On Tuesday, March 5, 2019 at 11:15:31 PM UTC+3:30, @TiddlyTweeter wrote:

Ciao Mohammad

I request that the  m  flag be enabled to be toggled ON.

In some ways it might be better if  m  mode were  the DEFAULT as its far less dangerous than  g  .

FYI, I found it quite easy to get it run using "m" flag just by changing the JS code from "g" to "m" ... like this ...

Would this be possible?

Best wishes

Josiah 

[Mohammad]

Many Thanks Josiah!

I would appreciate your time teaching us how to use regexp in Tiddlywiki and specially here with SNR2.

For the issue you mentioned, I could not find any solution and Jeremy comments did not help me to understand what I should do!

By the way, I thought it is better to choose the flag from interface, so I am adding

i: ignore case

g: find all (global match)

\b: whole words

m: multi line mode

So, user can select one or a mixed of them (of course \b is something separate here!)

I am not sure if m is useful here, but i and g are used frequently!

--Mohammad

On Wednesday, March 6, 2019 at 5:10:19 PM UTC+3:30, @TiddlyTweeter wrote:

Mohammad & all interested

I have done a lot of testing of "s-n-r2". It is very good!

DIFFERENCES FROM NORMAL REGEX

It differs from normal regular expressions in that TiddlyWiki needs to guard itself from eating itself.

 

It does this by needing the regex to be wrapped in Quote Marks.

The type of quote marks differ according to the use-case.

IF a tiddler only contains  '  then wrapping in  "  will work. And vice-versa.

IF a tiddler contains both  '   and  "  quotes then the match string is easiest enabled by wrapping it:  """regex""" .

LIMITS

1 - I have found NO WAY to safely make changes in a tiddler that contains the string  """ . Those 3 double quotes are likely protecting TW from itself but the regex can't currently cope with them.

 

That rules out safe s-n-r via regex in many macros.

 

2 -  SUBSTITUTION. Mohammad and I found out that there is currently no easy way to get Control Codes into output replaces. This is problematic as, for instance, we need to replace  <pre>  with  \n```\n . BUT in replace the "\n" is only treated as a literal.

Best wishes

Josiah

[Mohammad]

Josiah!

 I appreciate if you put these comments in few tiddlers to added as help to SNR2.

[Mohammad]

One more comment:

I am not good at regexp, but I think snr should follow what is reule/standard in Tiddlywiki, so

I think we need to change the code to act as the current regexp operator (https://tiddlywiki.com/#regexp%20Operator)

So, please let me know, if snr has any conflict with regexp or pass TW red lines!

[Mohammad]

Okay!

 I am achanging the interface and will upload a new update within an hour.

--Mohammad

On Wednesday, March 6, 2019 at 5:29:59 PM UTC+3:30, @TiddlyTweeter wrote:

M,

In particular, multi-line mode is excellent! Great news!

It allows step by step changes.

J.

[Mohammad]

Announcement: SNR2

Update: Mar 6th, 2019

Changes:

• regexp flags added (g, m, i)
• whole words search added
• minor issues fixed

This tool has amazing feature: You can search no only the text filed but also tags, caption, creator, ... and any other fields

You can even search and replace in tiddler title (in this case a new tiddler will be created!)

Please send you feedback and comments here!

------

Announcement: SNR2

This is a simple script, search-n-replcae (snr2) to search in tiddler contents (including all fields) for a searchValue and replace them with replaceValue.

Demo:

http://kookma.tiddlyspot.com/    (temporarily site )

[Jeremy Ruston]

Hi Mohammad

Sorry I missed your question in the thread. If you're trying to get a control character into a string for use in a regexp, you can use a construction like this:

<$codeblock code={{{ [[%0a]decodeuricomponent[]addprefix[a]addsuffix[b]] }}}/>

The idea is to express the character in the % notation used by URIs, and then use the decodeuricomponent filter operator to convert it into the corresponding character. In this case, $0a corresponds to the newline character (ascii code 10).

Best wishes

Jeremy.

--

Jeremy Ruston

jer...@jermolene.com

https://jermolene.com

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/4aacaf40-2a40-43f2-a5ce-37a0ad33a1dc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mohammad]

Many thanks Jeremy!

It seems I should read and process the replaceValue first and then pass it to SNR macro!

By the way, I think this way I can handle control characters.

Best wishes

Mohammad

[Mohammad]

Okay, Josiah!

I correct it!

--Mohammad

On Wednesday, March 6, 2019 at 7:59:11 PM UTC+3:30, @TiddlyTweeter wrote:

Mohammad,

Excellent!

An issue. You should not have Multi-line AND Global BOTH enabled! 

They are mutually exclusive.

Best wishes

Josiah

[Mohammad]

Update: Mar 7th, 2019

Changes:

• new interface
• regexp flags can be selected among i, g,m but not g and m at the same time.
• minor issues fixed

Demo:

http://kookma.tiddlyspot.com/    (temporarily site )

Mohammad

[Diego Mesa]

Hey Mohammad,

I have the following strange behavior:

https://media.giphy.com/media/dgarQbGE4SycN4wGJF/giphy.gif

[Mohammad]

Hello Diegi!

 I think the SNR does the job correctly!

May be Josiah can explain this in detail, but replace keep the parts in replaceValue and only add those characters word is not

available in the search value!

Here it actually change

This ---> This not

This ----> This not      }----> This not not

--Mohammad

[Mohammad]

SNR is released as a plugin.

This thread is closed here and a new thread is started to discuss the SNR plugin.

To follow the discussion from new thread have a look at

https://groups.google.com/d/msg/tiddlywiki/mkx1hZVH-I4/IeyC_GccBAAJ

I appreciate your feedback

--Mohammad

[Diego Mesa]

Hey Josiah

It was the last point you made that really clarified it for me - the difference engine visualization confused m. Thanks!

Diego

On Thursday, March 7, 2019 at 11:59:36 AM UTC-6, @TiddlyTweeter wrote:

Diego,

You are right. It does look odd. But it IS working fine. The reason why it appears odd is ...

-- you run in "multi-line" mode and that means the match string always matches to first point that matches in the field. In this case the last two letters of This".

-- because your match ="is" and replace ="is not" no difference will show for "is" as you don't change it.

-- the "walking" green insertion is due to limits in the "difference" engine that doesn't always know on repeated words quite where to display the change. 

    It is actually happening directly next to "This".

Hope this is clear!

Josiah