:g/./d slow because clipboard used for each processed line

[Dominique Pellé]

Hi

I noticed that running a vim command such as...

:g/./d

... was slow with a large number of lines, whereas it used to
be reasonably fast. It e.g. takes ~7 seconds with "only"
5000 lines using:

$ yes | nl | head -5000 | vim - -c 'g/./d'

At first I thought it was a recent vim regression. But then
I recalled that I recently installed the clipman applet [1] in
my xfce4 desktop. I disabled that xfce4 applet and saw that
the :g/./d command in vim became faster (~ 2 sec).

I then realized that the Vim command :g/./d copies
each line that it deletes to the clipboard. And somehow
the clipman applet made copying to the clipboard slower.

In fact if I disable x11 in Vim using Vim -X, this command
becomes instantaneous:

$ yes | nl | head -5000 | vim -X - -c 'g/./d'

It's not really a vim bug, but I wonder whether we can
do something in Vim to make a command like :g/./d
faster. I don't really need this command to copy each
of the 5000 lines to the clipboard. Maybe it could be smart
enough to copy only the last line to the clipboard?
It may not work if we do CTRL-C though.

Or maybe there could be a new command modifier
`:keepclipboard {cmd}` which would run the command {cmd}
without touching the clipboard, in the same spirit as existing
commands like `:keepalt`, `:keepmarks`, `:keepjumps`,
`:keeppatterns`.

I would then be able to run`:keepclipboard g/./d` which would
presumably be much faster (instantaneous with only
~ 5000 lines).

FWIW, I use the clipman applet [1] in xfce4 for 2 reasons:
- store the history of N recently copied text
- and unify the 2 Linux clipboards: primary clipboard and
default clipboard, in a way that it is possible to paste
what is selected with the mouse. I always found
that having 2 clipboards to be more annoying than
useful and unifying them with clipman helps me.

[1] https://docs.xfce.org/panel-plugins/clipman/start

Regards
Dominique

[Nick Jensen]

The 2021-05-06 00:45, Dominique Pellé wrote:
>It e.g. takes ~7 seconds with "only" 5000 lines using:
>
>$ yes | nl | head -5000 | vim - -c 'g/./d'

On my system it takes ~7.5 seconds with 150000 lines. But if I delete to
the black hole register it works in just under a second:

$ time yes | nl | head -150000 | vim - -c ':g/./d_' -c 'q!'

Regards,
Nick

[James McCoy]

On Thu, May 06, 2021 at 12:45:21AM +0200, Dominique Pellé wrote:
> Hi
>
> I noticed that running a vim command such as...
>
> :g/./d
>
> ... was slow with a large number of lines, whereas it used to
> be reasonably fast. It e.g. takes ~7 seconds with "only"
> 5000 lines using:
>
> $ yes | nl | head -5000 | vim - -c 'g/./d'
>
> At first I thought it was a recent vim regression. But then
> I recalled that I recently installed the clipman applet [1] in
> my xfce4 desktop. I disabled that xfce4 applet and saw that
> the :g/./d command in vim became faster (~ 2 sec).
>
> I then realized that the Vim command :g/./d copies
> each line that it deletes to the clipboard. And somehow
> the clipman applet made copying to the clipboard slower.

This was supposed to be addressed by 7.4.396, which defers updating the
selection ownership until :g is done running.

Cheers,
--
James
GPG Key: 4096R/91BF BF4D 6956 BD5D F7B7 2D23 DFE6 91AE 331B A3DB

[Dominique Pellé]

Thanks, that's is a good solution!

James McCoy wrote:

> > I then realized that the Vim command :g/./d copies
> > each line that it deletes to the clipboard. And somehow
> > the clipman applet made copying to the clipboard slower.
>

> This was supposed to be addressed by 7.4.396, which
> defers updating the selection ownership until :g is done running.

Link to that patch:
https://github.com/vim/vim/commit/6b1ee34aa0236b50f675f3bbcd9bf0b7a3384f7f

updated for version 7.4.396

Problem: When 'clipboard' is "unnamed", :g/pat/d is very slow. (Praful)
Solution: Only set the clipboard after the last delete. (Christian Brabandt)

Ah, it's fast when clipboard=unnamed.
However, it's slow with clipboard=unnamed,unnamedplus
which I have in my ~/.vimrc.

Regards
Dominique

[Bram Moolenaar]

Hmm, the patch also includes code for unnamedplus, who would it not
work?

--
Google is kind of like Dr. Who's Tardis; it's weirder on the
inside than on the outside...

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///