[vim/vim] Please add option to change search and replace behavior wrt `\n` and `\r`. (#7744)

[Chaoren Lin]

From time to time I'll need to search and replace something with a newline, and I'll do :s/<pattern>/\n/ and I'll get a bunch of ^@s. Then I'll remember: "oh, right, vim uses \r for newline in replacements". Instead of this happening every single time, I would like an option to change that behavior.

As far as I can tell, there's no such option at the moment, and there doesn't seem to be a way to intercept the :s command and just change the replacement string.

I don't want to change the default behavior. I'm sure you have your reasons for why it behaves this way in the first place. I just want something I can set in my .vimrc so I can permanently forget about this thing.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

[Gary Johnson]

On 2021-01-25, Chaoren Lin wrote:
> From time to time I'll need to search and replace something with a newline, and
> I'll do :s/<pattern>/\n/ and I'll get a bunch of ^@s. Then I'll remember: "oh,
> right, vim uses \r for newline in replacements". Instead of this happening
> every single time, I would like an option to change that behavior.
>
> As far as I can tell, there's no such option at the moment, and there doesn't
> seem to be a way to intercept the :s command and just change the replacement
> string.
>
> I don't want to change the default behavior. I'm sure you have your reasons for
> why it behaves this way in the first place. I just want something I can set in
> my .vimrc so I can permanently forget about this thing.

The thing is, if you did that, you'd break all the plugins that rely
on the current behavior. So that's out.

However, I think you could write your own command to do that. Name
it whatever you want, but of course it must begin with an upper-case
letter. If you want the :s command to invoke your command, you can
use the :cabbrev command to replace :s with your command, something
like this (adapted from my vimrc but otherwise untested):

cabbrev <expr> s ((getcmdtype() == ':' && getcmdpos() <= 2) ? 'MyS' : 's')

The details are left as an exercise for the reader, as I long ago
just got used to typing \r instead of \n in the replacement string.

Regards,
Gary

[vim-dev ML]

On 2021-01-25, Chaoren Lin wrote:

> From time to time I'll need to search and replace something with a newline, and
> I'll do :s/<pattern>/\n/ and I'll get a bunch of ^@s. Then I'll remember: "oh,
> right, vim uses \r for newline in replacements". Instead of this happening
> every single time, I would like an option to change that behavior.
>
> As far as I can tell, there's no such option at the moment, and there doesn't
> seem to be a way to intercept the :s command and just change the replacement
> string.
>
> I don't want to change the default behavior. I'm sure you have your reasons for
> why it behaves this way in the first place. I just want something I can set in
> my .vimrc so I can permanently forget about this thing.

The thing is, if you did that, you'd break all the plugins that rely
on the current behavior. So that's out.

However, I think you could write your own command to do that. Name
it whatever you want, but of course it must begin with an upper-case
letter. If you want the :s command to invoke your command, you can
use the :cabbrev command to replace :s with your command, something
like this (adapted from my vimrc but otherwise untested):

cabbrev <expr> s ((getcmdtype() == ':' && getcmdpos() <= 2) ? 'MyS' : 's')

The details are left as an exercise for the reader, as I long ago
just got used to typing \r instead of \n in the replacement string.

Regards,
Gary

[Chaoren Lin]

if you did that, you'd break all the plugins that rely on the current behavior

Do you mean because they use \r? I don't mind having both \r and \n mean the same thing in the replacement string.

Do plugins actually use \n in the replacement string with its current behavior? I don't mind not using those plugins.

People who have problems with it could choose either to not use the option or not use the plugins. Or plugins could get updated to detect this option. We already have many options that change behavior (e.g., magic) and plugins just deal with them by checking or by being careful with their regexes (e.g., \v, \m, \V, \M).

—
You are receiving this because you commented.

[Gary Johnson]

On 2021-01-25, Chaoren Lin wrote:

> if you did that, you'd break all the plugins that rely on the current
> behavior
>
> Do you mean because they use \r? I don't mind having both \r and \n mean the
> same thing in the replacement string.

That would be another way to implement the requested behavior, but
it still changes the existing behavior.

> Do plugins actually use \n in the replacement string with its current behavior?
> I don't mind not using those plugins.

I don't know.

You may not mind not using those plugins, but other people might.

> People who have problems with it could choose either to not use the option or
> not use the plugins. Or plugins could get updated to detect this option. We
> already have many options that change behavior (e.g., magic) and plugins just
> deal with them by checking or by being careful with their regexes (e.g., \v, \
> m, \V, \M).

When a plugin starts behaving oddly, it doesn't announce, "My
behavior has changed because you changed this to that". The user
just notices that something's not right. They may not notice the
problem for a while and may not associate the change in behavior to
their setting of the option. The problem then has to be
troubleshot.

The use of \r instead of \n in the replacement string is enough of
an annoyance to new users that nearly every new user, upon hearing
about an option to change the behavior, would set that option. They
would be unaware of the possible consequences and would be the
least capable of troubleshooting their configurations if a problem
arose.

Plugins often don't get updated. They are written, then maintained
for a while, then the author disappears or loses interest.

There is not a central repository of plugins, so it would not be
possible for one person to just update them all.

One of the great features of Vim is that Bram is fastidious about
maintaining backward compatibility. Vim hardly ever breaks because
of some new feature.

Consequently, little annoyances like this persist and have to
adjusted to or worked around. I still think that your best bet is
to write your own command that would do what you want. It solves
your problem and doesn't interfere with anything or anyone else.

Regards,
Gary

[vim-dev ML]

[Chaoren Lin]

Plugins are probably using the substitue() function instead of the command :substitute. It doesn't need to apply to substitute().

If that's still not good enough. If there's a way to determine whether a command was entered by hand vs from a plugin, have the option only apply when the command was manually entered. This would cause an issue with plugin writers if they expect a plugin's behavior to be the same as when entered by hand, but presumably plugin writers should know how to read the docs.

The use of \r instead of \n in the replacement string is enough of an annoyance to new users that nearly every new user, upon hearing about an option to change the behavior, would set that option.

Seems like a pretty good argument for having such an option.

—
You are receiving this because you commented.

[Chaoren Lin]

cabbrev s ((getcmdtype() == ':' && getcmdpos() <= 2) ? 'MyS' : 's')

Isn't cabbrev command line mode only? Why do you need to check getcmdtype()?

There could be an optional range in front of the command. getcmdpos() <= 2 would exclude that.

What's wrong with cabbrev s MyS?

In general I just find it super jarring when whatever I'm typing gets changed automatically from under my cursor. That's why I prefers not use imaps and cabbrevs. Since this is a single character command, it could feel different. I'll try it for a while to see if I end up hating it. I'd still prefer an option though.

—
You are receiving this because you commented.

[Chaoren Lin]

The cabbrev method breaks incsearch for :substitute. There doesn't seem to be a way to add incsearch functionality to custom commands.

—
You are receiving this because you commented.

[Chaoren Lin]

Oh wait, I can probably use call matchadd('IncSearch', {pattern}) in a CmdlineChanged autocmd.

—
You are receiving this because you commented.

[Gary Johnson]

On 2021-01-25, Chaoren Lin wrote:

> cabbrev s ((getcmdtype() == ':' && getcmdpos() <= 2) ? 'MyS' : 's')
>
> Isn't cabbrev command line mode only? Why do you need to check getcmdtype()?

Because command-line mode includes entering search patterns and
filter commands as well as ex commands. See ":help
Command-line-mode".

> There could be an optional range in front of the command. getcmdpos() <= 2
> would exclude that.

I may have been wrong about how easy this would be. None of my
applications of :cabbrev use ranges.

To get this solution to work may require getcmdline() instead of
getcmdpos() and parsing the result to see if the s is preceded by
a range. Ick.

> What's wrong with cabbrev s MyS?

The potential problem is that the s would be expanded to MyS
anywhere in the command line that a single s appears.

> In general I just find it super jarring when whatever I'm typing gets changed
> automatically from under my cursor.

Agreed. I usually use command-line abbreviations that are the
lower-case spelling of the command name so that only the first
letter changes. I would have called your function S, but that
command name is used by the vis plugin and my thought was to change
the name of your command to MyS.

> That's why I prefers not use imaps and cabbrevs. Since this is
> a single character command, it could feel different. I'll try it
> for a while to see if I end up hating it. I'd still prefer an
> option though.

I see your points. I don't think your proposed option is going to
fly for the reasons I gave before, though. The chances of
encountering a problem with a plugin are small, but they're not
zero.

To your proposal of having the option apply only to the command
line: I can't think of any option that does that. It would be
confusing (to others) if \n worked only on the command line. That's
why it would be nice if you could customize :s for just yourself
without having to write a whole lot of vimscript, or be jarred by
the abbreviation expansion.

Regards,
Gary

[vim-dev ML]

[Chaoren Lin]

Alright so now I have this monstrosity that should work exactly the same as :s. The incsearch highlighting doesn't work with ranges or custom delimiters yet, but they should be possible. When I'm done with this I should probably release it as a plugin.

function! s:Substitute(line1, line2, ...) abort
	let l:a000 = copy(a:000)
	if a:0 > 0 && a:1[0] =~ '[[:punct:]]'
		let l:args = split(a:1, a:1[0], 1)
		if len(l:args) >= 3
			let l:args[2] = substitute(l:args[2], '\\n', '\\r', 'g')
		endif
		let l:a000[0] = join(l:args, a:1[0])
	endif
	execute a:line1 . ',' . a:line2 . 'substitute' join(l:a000)
endfunction

command! -bar -range -nargs=* S call s:Substitute(<line1>, <line2>, <f-args>)
cabbrev <expr> s getcmdtype() == ':' ? 'S' : 's'

function! s:SubstituteHighlight(leave) abort
	if expand('<afile>') != ':'
		return
	endif

	if !exists('s:sub_ids')
		let s:sub_ids = {}
	else
		for [l:wid, l:mid] in items(s:sub_ids)
			if win_id2win(l:wid) != 0
				call matchdelete(l:mid, l:wid)
			endif
			unlet s:sub_ids[l:wid]
		endfor
	endif

	if a:leave
		unlet s:sub_ids
		return
	endif

	let l:cmd = getcmdline()
	if l:cmd[0] != 'S' || l:cmd[1] != '/'
		return
	endif

	let l:pattern = split(l:cmd, '/', 1)[1]

	if empty(l:pattern)
		redraw
		return
	endif

	let l:wnr = winnr()
	windo let s:sub_ids[win_getid()] = matchadd('IncSearch', l:pattern, 0, -1, {'window': winnr()})
	execute l:wnr . 'wincmd' 'w'

	redraw
endfunction

autocmd CmdlineChanged * call s:SubstituteHighlight(0)
autocmd CmdlineLeave   * call s:SubstituteHighlight(1)

—
You are receiving this because you commented.

[Christian Brabandt]

Option settings like gdefault and magic are known to break plugins, because not many are aware of those. I am therefore not in favor of adding another option that changes how \r and \n is used in searches. You might be able to use c_CTRL-_e to evaluate your command line and replace \r by \n (or the other way around).

—
You are receiving this because you commented.

[Bram Moolenaar]

Since there is an alternative, and it will help only few users, adding an option isn't going to happen. Either just learn to use \r or use a custom command. Discussing the command doesn't need to take place here, therefore closing.

—
You are receiving this because you commented.

[Bram Moolenaar]

Closed #7744.

—
You are receiving this because you commented.