global command slow when clipboard=unnamed

[Praful Kapadia]

I have had an annoying issue with gvim 7.4, with patches 1-307. If I open a large file (e.g. containing 200,000 lines) and use the global command to delete lines, the operation takes a very long time on Windows if clipboard has been set to unnamed. I'm assuming it's the constant copying of deleted lines to the Windows clipboard that's slowing gvim down.

I use Windows 7 64-bit. I have compiled gvim 64-bit using ming. The issue occurs on gvim 64-bit, 32-bit and, to a lesser extent, on MacVim.

On Windows, it takes several minutes to carry out the operation. During this time, Windows becomes unusable, which is poor but that's another issue.

On OS X, in MacVim, the same operation takes 30 seconds. With clipboard="", it takes two seconds.

Here are the steps to reproduce the issue:

1. Open gvim with no plugins and no vimrc.
2. :set clipboard=unnamed
3. Open a text file with about 200,000 lines.
4. Enter :g/string/d The "string" should match about 150,000 lines i.e. you want to delete lots of rows!
5. Go for a coffee break (Windows!) or wait 30 seconds (OS X)

In practice, if I issue the command on Windows, I kill the process then open the file again, this time setting clipboard="" before I issue the command.

The workaround (:set clipboard="") is fine if you remember it! It would be nice if gvim did this e.g. (pseudo-code):

old_clipboard = clipboard
try
clipboard = ""
execute global command
finally
clipboard = old_clipboard
end

One consideration for side effects: currently, if clipboard=unnamed, the only text that ends up on the system clipboard is the final deleted line, not all deleted lines. If anything, you might want all deleted text to be on the clipboard but that is not what currently happens. I suspect neither the last line nor all lines is generally required. I don't care (others might) what ends up on the clipboard and would be happy if there was no speed penalty when the command was issued!

It would be great if someone could look at this!

Thanks

Praful

[Павлов Николай Александрович]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On May 29, 2014 9:25:05 PM GMT+03:00, Praful Kapadia <praf...@gmail.com> wrote:
>I have had an annoying issue with gvim 7.4, with patches 1-307. If I
>open a large file (e.g. containing 200,000 lines) and use the global
>command to delete lines, the operation takes a very long time on
>Windows if clipboard has been set to unnamed. I'm assuming it's the
>constant copying of deleted lines to the Windows clipboard that's
>slowing gvim down.
>
>I use Windows 7 64-bit. I have compiled gvim 64-bit using ming. The
>issue occurs on gvim 64-bit, 32-bit and, to a lesser extent, on MacVim.
>
>
>On Windows, it takes several minutes to carry out the operation. During
>this time, Windows becomes unusable, which is poor but that's another
>issue.
>
>On OS X, in MacVim, the same operation takes 30 seconds. With
>clipboard="", it takes two seconds.

Do not mislead yourself. '"' character starts a comment, so

set clipboard=""

is equivalent to

set clipboard="unnamed"

which is really (after you strip comment)

set clipboard=

. There is also no need to balance quotes:

set clipboard="unnamed

means the same thing. Use

let &clipboard="..."

syntax if you want '"' to start a string and not a comment.

>
>Here are the steps to reproduce the issue:
>
>1. Open gvim with no plugins and no vimrc.
>2. :set clipboard=unnamed
>3. Open a text file with about 200,000 lines.
>4. Enter :g/string/d The "string" should match about 150,000
>lines i.e. you want to delete lots of rows!

If you do not need to *cut* lines (any Vim delete operation cuts by default) you need to use black hole register: "delete _" in place of just "delete" ("d" is the short form of it).

>5. Go for a coffee break (Windows!) or wait 30 seconds (OS X)
>
>In practice, if I issue the command on Windows, I kill the process then
>open the file again, this time setting clipboard="" before I issue the
>command.
>
>The workaround (:set clipboard="") is fine if you remember it! It would
>be nice if gvim did this e.g. (pseudo-code):
>
> old_clipboard = clipboard
> try
> clipboard = ""
> execute global command
> finally
> clipboard = old_clipboard
> end
>
>One consideration for side effects: currently, if clipboard=unnamed,
>the only text that ends up on the system clipboard is the final deleted
>line, not all deleted lines. If anything, you might want all deleted
>text to be on the clipboard but that is not what currently happens. I
>suspect neither the last line nor all lines is generally required. I
>don't care (others might) what ends up on the clipboard and would be
>happy if there was no speed penalty when the command was issued!
>
>It would be great if someone could look at this!
>
>Thanks
>
>Praful
>

>--
>--
>You received this message from the "vim_dev" maillist.
>Do not top-post! Type your reply below the text you are replying to.
>For more information, visit http://www.vim.org/maillist.php
>
>---
>You received this message because you are subscribed to the Google
>Groups "vim_dev" group.
>To unsubscribe from this group and stop receiving emails from it, send
>an email to vim_dev+u...@googlegroups.com.
>For more options, visit https://groups.google.com/d/optout.
-----BEGIN PGP SIGNATURE-----
Version: APG v1.1.1

iQI1BAEBCgAfBQJTh8dWGBxaeVggPHp5eC52aW1AZ21haWwuY29tPgAKCRCf3UKj
HhHSvm8TD/90z5dINDuy3tFr7RcSzd671AYa1yeBEEEfY4/dLtWebdl9nNPROFSk
tJ0E9X+fyAb5zbBs+tdhtTj8Zk5wEMtcYqNDJvtkIQKEQy2o+iJKXgKNKqKF11gN
YWQm0jpA5TlW7mLhSXR3AsqYXHHLKiWeRx/s+G2T8GW8GWX5/w3CbwOx88lSo7FQ
K7zs+LoF8UQoXmxZjzT3XKSwve9KRhIQB9VqjvREtmBzOGS8ZW+8GNvApMqAcbdG
y0HZ616R4gW8wl4yuDeWeNfIbAAYyOvzz6+hDrFtwNeNHnPf0os5+bBHNe/be6DP
8oml7sRr80pAsfLjbYx2nsImKNTX/+BBCmd2s+VQzjgJJFZzQ2Z6tcDhb4WKtFCU
0KLbmCHDE0fVkRr14gzPsNSPgA+bHLiYFMxwoZUjs5E2gv96VY1hYTlhDxCq4nx1
s5FWHUIAGwrjUDE5cY68jbxhmm6pNRrMzO0m2kQlwWjy02KgtC2+gNFNxqEq7U1a
LtcVIp7tuOFud0SUvA4yVhcMdsxUdF8LYe9MCzLmrWAjE6vT16OeFJCO+Q44f8LX
rJdtaDOnj5rEK+fEpQDDOQXCkGm4uli0egJ/gcd9EpwlrLoTnPDDGTGRAn3G4Bun
i3sJDX6bVSXSYo+XISaa0n5HvsxltzF4f8QbOVeRhIDHVtAbuJSzEA==
=t81i
-----END PGP SIGNATURE-----

[John Beckett]

It's a bit hard to see the excellent advice in that last post,
so here it is:

To delete (not cut) all lines matching pattern use:

:g/pattern/d_

John

[Praful Kapadia]

Point taken about quotes being comment indicators in my set clipboard="".

I completely forgot about the black hole register! Isn't vim wonderful just to have that concept!

Thank you ZyX - and John for making sure I didn't miss the black hole advice.

Praful

[Christian Brabandt]

Hi Praful!

This is a known issue for windows (see :h todo.txt and search for
:g/test/d)

Here is an experimental patch, that resets the clipboard option for :g
command, when there are many matches (>100). Note: I haven't tested it.

Best,
Christian
--

[Praful Kapadia]

On Friday, 30 May 2014 12:27:01 UTC+1, Christian Brabandt wrote:
> This is a known issue for windows (see :h todo.txt and search for
>
> :g/test/d)

Thanks Christian - and apologies to all for not seeing the todo list

>
> Here is an experimental patch, that resets the clipboard option for :g
>
> command, when there are many matches (>100). Note: I haven't tested it.
>

I may try the patch or probably wait until it enters the main code base. :g/test/d_ has already entered my muscle memory - probably because the alternative is so disruptive!

Praful

[Praful]

OK - I've tried the patch and can confirm it works.

I'm unfamiliar with what happens once I've applied a patch. Will I need to apply the patch every time I pull/update? Or will I not receive updates to my patched files?

Also, how does this patch become accepted in the official code base?

Thanks

Praful

[Christian Brabandt]

You should probably unapply the patch, before pulling again from the
mercurial repository. After updating, you can apply the patch again and
build again.

> Also, how does this patch become accepted in the official code base?

Bram decides, whether a patch gets applied or not.

Best,
Christian
--
Allmus, der
Ein gigantischer nigerianischer Baum, aus dessen harzigem Saft
sämtliche Kantinen-Marmeladen hergestellt werden.
-- Douglas Adams, John Lloyd, Sven Böttcher ("Der tiefere Sinn des Labenz")

[Gary Johnson]

On 2014-05-30, Christian Brabandt wrote:
> Hi Praful!
>
> On Do, 29 Mai 2014, Praful Kapadia wrote:
>
> > I have had an annoying issue with gvim 7.4, with patches 1-307.
> > If I open a large file (e.g. containing 200,000 lines) and use
> > the global command to delete lines, the operation takes a very
> > long time on Windows if clipboard has been set to unnamed. I'm
> > assuming it's the constant copying of deleted lines to the
> > Windows clipboard that's slowing gvim down.

[...]

> This is a known issue for windows (see :h todo.txt and search for
> :g/test/d)

I have seen a related problem, if not the same problem, running vim
7.4.283 in an xterm on Fedora 17.

While editing a file of about 50k lines, I put the cursor on a line
a few tens of lines from the bottom of the file and executed dgg.
When I then tried moving the cursor a few lines lower, it moved in
response to the first few j commands, then vim locked up for about 5
minutes.

After reading this thread, I tried two workarounds, both of which
worked to keep the cursor responsive.

1. "_dgg

2. :set clipboard&
dgg

My 'clipboard' is normally set to

unnamed,autoselect,exclude:cons\|linux

Also, while vim and the xterm are running on Fedora 17, the X server
is NoMachine (NX). The NoMachine client is running on Windows 7, so
anything going to the X clipboard is also deposited in a Windows
clipboard. I don't know what effect that has on the problem, e.g.,
whether the X client must wait for data to be copied to the Windows
clipboard.

For what it's worth, I attached gdb to vim process and found that
while stuck, a backtrace looked like this.

(gdb) bt
#0 0x0000003dbfae8ea4 in poll () from /lib64/libc.so.6
#1 0x0000003dcb22d428 in _XtWaitForSomething () from /lib64/libXt.so.6
#2 0x0000003dcb22e987 in XtAppNextEvent () from /lib64/libXt.so.6
#3 0x0000000000544a52 in xterm_update () at os_unix.c:7068
#4 0x00000000005426be in RealWaitForChar (fd=0, msec=-1, check_for_gpm=0x7fff63686104) at os_unix.c:5438
#5 0x00000000005423e9 in WaitForChar (msec=-1) at os_unix.c:5139
#6 0x000000000053e273 in mch_inchar (buf=0x89bca8 "", maxlen=64, wtime=-1, tb_change_cnt=251) at os_unix.c:450
#7 0x00000000005cf494 in ui_inchar (buf=0x89bca8 "", maxlen=64, wtime=-1, tb_change_cnt=251) at ui.c:199
#8 0x00000000004c682f in inchar (buf=0x89bca8 "", maxlen=192, wait_time=-1, tb_change_cnt=251) at getchar.c:3082
#9 0x00000000004c6473 in vgetorpeek (advance=1) at getchar.c:2857
#10 0x00000000004c47bd in vgetc () at getchar.c:1627
#11 0x00000000004c4cf1 in safe_vgetc () at getchar.c:1832
#12 0x0000000000512a82 in normal_cmd (oap=0x7fff636864e0, toplevel=1) at normal.c:638
#13 0x0000000000612681 in main_loop (cmdwin=0, noexmode=0) at main.c:1325
#14 0x00000000006120d0 in main (argc=6, argv=0x7fff636867f8) at main.c:1025

> Here is an experimental patch, that resets the clipboard option for :g
> command, when there are many matches (>100). Note: I haven't tested it.

If this is the same problem, then fixing it for :g/teset/d doesn't
address the other cases of deletion such as dgg.

Regards,
Gary

[Bram Moolenaar]

I wonder if this only applies to the :g command. Doesn't :%s have the
same problem? And perhaps joining many lines. Anything that repeatedly
puts text in the " register.

Wouldn't it be possible to postpone putting text on the clipboard until
a command has finished? It's like settint the 'clipboard' option empty
and restoring it, and when it's restored to the copy to the clipboard.
Perhaps set a flag "postponed_clipboard_set" and handle it in the main
loop. Not sure if there are exceptions (e.g., fetching the clipboard
halfway the command).


--
hundred-and-one symptoms of being an internet addict:
238. You think faxes are old-fashioned.

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Praful]

On Friday, 30 May 2014 19:31:06 UTC+1, Bram Moolenaar wrote:

> I wonder if this only applies to the :g command. Doesn't :%s have the
>
> same problem? And perhaps joining many lines. Anything that repeatedly
>
> puts text in the " register.

I just tried :%s where I substituted something on almost all 200,000 lines of a file. It did the substitutions in less than a few seconds. To make sure I wasn't using Christian's patch, I then tried :g/pattern/d and Windows froze!

I tried the :%s on my patched gvim as well and that worked fine too.

Praful

[Praful]

On Friday, 30 May 2014 19:31:06 UTC+1, Bram Moolenaar wrote:

> I wonder if this only applies to the :g command. Doesn't :%s have the
>
> same problem? And perhaps joining many lines. Anything that repeatedly
>
> puts text in the " register.
>

I should have added that joining many lines (eg 190000J) is fine too. (Clipboard was set to unnamed.)

Praful

[Bram Moolenaar]

I now realize that the :s command does not set a register. So your
problem most likely is really about setting the clipboard, not something
else.

Perhaps :g and :v are the only commands that fill a register multiple
times? Well, executing a script does, of course. And executing from a
register multiple times. But that's not done with one command.

--
"The amigos also appear to be guilty of not citing the work of others who had
gone before them. Even worse, they have a chapter about modeling time and
space without making a single reference to Star Trek!"
(Scott Ambler, reviewing the UML User Guide)

[Christian Brabandt]

Hi Gary!

Yes, it wouldn't fix it for your case. Your use case looks really awful.
Are other applications as slow? I am not sure, we could do anything for
you. Perhaps it would be possible to disable the clipboard, if it takes
too long updating it? Why are you setting your clipboard to unnamed, if
it works so slow?

Best,
Christian
--
Je mehr Licht man in die Kirchengeschichte bringt, desto dunkler
wird's.
-- Heinrich Wiesner

[Christian Brabandt]

Hi Bram!

On Sa, 31 Mai 2014, Bram Moolenaar wrote:

>
> Praful wrote:
>
> > On Friday, 30 May 2014 19:31:06 UTC+1, Bram Moolenaar wrote:
> >
> > > I wonder if this only applies to the :g command. Doesn't :%s have the
> > >
> > > same problem? And perhaps joining many lines. Anything that repeatedly
> > >
> > > puts text in the " register.
> >
> > I just tried :%s where I substituted something on almost all 200,000
> > lines of a file. It did the substitutions in less than a few seconds.
> > To make sure I wasn't using Christian's patch, I then tried
> > :g/pattern/d and Windows froze!
> >
> > I tried the :%s on my patched gvim as well and that worked fine too.
>
> I now realize that the :s command does not set a register. So your
> problem most likely is really about setting the clipboard, not something
> else.
>
> Perhaps :g and :v are the only commands that fill a register multiple
> times? Well, executing a script does, of course. And executing from a
> register multiple times. But that's not done with one command.

probably also:
:folddoopen
:folddoclosed

Mit freundlichen Grüßen
Christian
--
Abwärts Tyrann, nach oben ein Knecht; Verleumder des Menschen,
Speichellecker des Herrn - voila des Glaubens Porträt.
-- Ludwig Feuerbach

[Bram Moolenaar]

Christian wrote:

> On Sa, 31 Mai 2014, Bram Moolenaar wrote:
>
> > Praful wrote:
> >
> > > On Friday, 30 May 2014 19:31:06 UTC+1, Bram Moolenaar wrote:
> > >
> > > > I wonder if this only applies to the :g command. Doesn't :%s have the
> > > >
> > > > same problem? And perhaps joining many lines. Anything that repeatedly
> > > >
> > > > puts text in the " register.
> > >
> > > I just tried :%s where I substituted something on almost all 200,000
> > > lines of a file. It did the substitutions in less than a few seconds.
> > > To make sure I wasn't using Christian's patch, I then tried
> > > :g/pattern/d and Windows froze!
> > >
> > > I tried the :%s on my patched gvim as well and that worked fine too.
> >
> > I now realize that the :s command does not set a register. So your
> > problem most likely is really about setting the clipboard, not something
> > else.
> >
> > Perhaps :g and :v are the only commands that fill a register multiple
> > times? Well, executing a script does, of course. And executing from a
> > register multiple times. But that's not done with one command.
>
> probably also:
> :folddoopen
> :folddoclosed

Oh, yes. Then also :bufdo, :windo and friends.

--
Marriage isn't a word. It's a sentence.

[Gary Johnson]

Hi Christian,

On 2014-05-31, Christian Brabandt wrote:
> Hi Gary!
>
> On Fr, 30 Mai 2014, Gary Johnson wrote:
>
> > On 2014-05-30, Christian Brabandt wrote:
> > > Hi Praful!
> > >
> > > On Do, 29 Mai 2014, Praful Kapadia wrote:
> > >
> > > > I have had an annoying issue with gvim 7.4, with patches 1-307.
> > > > If I open a large file (e.g. containing 200,000 lines) and use
> > > > the global command to delete lines, the operation takes a very
> > > > long time on Windows if clipboard has been set to unnamed. I'm
> > > > assuming it's the constant copying of deleted lines to the
> > > > Windows clipboard that's slowing gvim down.
> >
> > [...]
> >
> > > This is a known issue for windows (see :h todo.txt and search for
> > > :g/test/d)
> >
> > I have seen a related problem, if not the same problem, running vim
> > 7.4.283 in an xterm on Fedora 17.
> >
> > While editing a file of about 50k lines, I put the cursor on a line
> > a few tens of lines from the bottom of the file and executed dgg.
> > When I then tried moving the cursor a few lines lower, it moved in
> > response to the first few j commands, then vim locked up for about 5
> > minutes.

[...]

> > > Here is an experimental patch, that resets the clipboard option for :g
> > > command, when there are many matches (>100). Note: I haven't tested it.
> >
> > If this is the same problem, then fixing it for :g/teset/d doesn't
> > address the other cases of deletion such as dgg.
>
> Yes, it wouldn't fix it for your case. Your use case looks really awful.
> Are other applications as slow? I am not sure, we could do anything for
> you. Perhaps it would be possible to disable the clipboard, if it takes
> too long updating it? Why are you setting your clipboard to unnamed, if
> it works so slow?

The speed of applications running on the Linux machine in this setup
varies. Coworkers have commented on the sluggishness of our
NoMachine configurations. One suggested a tweak to the NoMachine
settings which helped a lot. Another is using Xming instead and
says the performance is much better. I just haven't gotten around
to trying it.

Before this issue with deleting huge blocks of text, the only
speed-related problems I've had with Vim in this setup have been its
response to termresponse strings and the updating of windows in diff
mode. I haven't seen the termresponse issue in a while and the
tweak to NoMachine seems to have fixed the diff-mode refresh issue.

I hadn't had any problems with the clipboard before this attempt to
delete almost 50k lines. I seldom delete that much and usually copy
and paste no more than a couple of pages of text. Any other
slowness due to my use of the clipboard has been negligible. A
workaround of deleting such large amounts of text to the black hole
register would be OK as long as I remember to specify that register
_before_ starting the deletion.

For most of my work, the benefits of setting the clipboard to
unnamed outweigh the disadvantages. It allows me to easily copy and
paste between multiple Vim instances, some of them running on Linux
and some running on Windows, without having to prefix each operation
with "* or "+. I also run autocutsel (see
http://mutelight.org/articles/subtleties-of-the-x-clipboard) so that
anything I copy to the primary selection is automatically copied to
the clipboard and vice versa.

Regards,
Gary

[Christian Brabandt]

On Sa, 31 Mai 2014, Bram Moolenaar wrote:

> > > Perhaps :g and :v are the only commands that fill a register multiple
> > > times? Well, executing a script does, of course. And executing from a
> > > register multiple times. But that's not done with one command.

> > :folddoopen
> > :folddoclosed
> Oh, yes. Then also :bufdo, :windo and friends.

Updated patch attached.

Best,
Christian
--
In der Liebe wird der Ernst der Jungfrau bezaubern; in der Ehe, die
selber ein langer Ernst ist, möchte leichtes Scherzen und Bescherzen
der Welt besser einschlagen.
-- Jean Paul

[Christian Brabandt]

On So, 01 Jun 2014, Christian Brabandt wrote:

> On Sa, 31 Mai 2014, Bram Moolenaar wrote:
>
> > > > Perhaps :g and :v are the only commands that fill a register multiple
> > > > times? Well, executing a script does, of course. And executing from a
> > > > register multiple times. But that's not done with one command.
> > > :folddoopen
> > > :folddoclosed
> > Oh, yes. Then also :bufdo, :windo and friends.
>
> Updated patch attached.

Best,
Christian
--

Fragt der Lehrer die Esther: "Was heißt Shalom?"
Darauf Esther: "Friede."
"Und was heißt Elshalom?"
"Elfriede."

[Bram Moolenaar]

Christian wrote:

On So, 01 Jun 2014, Christian Brabandt wrote:

> On Sa, 31 Mai 2014, Bram Moolenaar wrote:
>
> > > > Perhaps :g and :v are the only commands that fill a register multiple
> > > > times? Well, executing a script does, of course. And executing from a
> > > > register multiple times. But that's not done with one command.
> > > :folddoopen
> > > :folddoclosed
> > Oh, yes. Then also :bufdo, :windo and friends.
>
> Updated patch attached.

When restoring clip_unnamed, it's not too dificult to set the clipboard
then. Would require checking that it would have been set (default
register changed). Then the remarks about "doesn't work for some
commands" can be changed to "for some commands disabled until the end".
I'm sure this avoids surprises for users.

--
hundred-and-one symptoms of being an internet addict:

249. You've forgotten what the outside looks like.

[Praful]

On Sunday, 1 June 2014 15:30:44 UTC+1, Bram Moolenaar wrote:
>
> When restoring clip_unnamed, it's not too dificult to set the clipboard
>
> then. Would require checking that it would have been set (default
>
> register changed). Then the remarks about "doesn't work for some
>
> commands" can be changed to "for some commands disabled until the end".
>
> I'm sure this avoids surprises for users.
>

I'm not sure how to test the :bufdo, :windo, etc but I'm happy to test the issue I originally reported once a patch has been finalised.

If someone posts the use cases for :bufdo, etc, I'll test those as well

Praful

[Praful]

Hi Christian

Are you considering another patch to incorporate Bram's comments? Or is this patch the candidate for inclusion in the Vim code base? If it is, I'll test it.

Thanks

Praful

[Christian Brabandt]

On So, 01 Jun 2014, Bram Moolenaar wrote:
> When restoring clip_unnamed, it's not too dificult to set the
> clipboard then. Would require checking that it would have been set
> (default register changed). Then the remarks about "doesn't work for
> some commands" can be changed to "for some commands disabled until the
> end". I'm sure this avoids surprises for users.

I am not sure how to do it.

Best,
Christian
--
Jedes menschliche Werk ist zugleich Sache und Symbol.
-- Paul Johannes Tillich

[Christian Brabandt]

Hi Praful!

On Fr, 06 Jun 2014, Praful wrote:

> Hi Christian
>
> Are you considering another patch to incorporate Bram's comments? Or is this patch the candidate for inclusion in the Vim code base? If it is, I'll test it.

I am not exactly sure how to integrate Brams comments. So I can't be of
any help currently.

Best,
Christian
--
Keine Kunst ist's, alt zu werden, es ist Kunst, es zu ertragen.
-- Johann Wolfgang von Goethe (Zahme Xenien)

[Bram Moolenaar]

Christian wrote:

> On So, 01 Jun 2014, Bram Moolenaar wrote:
> > When restoring clip_unnamed, it's not too dificult to set the
> > clipboard then. Would require checking that it would have been set
> > (default register changed). Then the remarks about "doesn't work for
> > some commands" can be changed to "for some commands disabled until the
> > end". I'm sure this avoids surprises for users.
>
> I am not sure how to do it.

This would require a flag, e.g. "did_set_selection". When resetting
clip_unnamed before doing a global operation, this flag would be reset.
Then when text is deleted or yanked, clip_own_selection() will be
called, and if the register used matches the register that would be
associated with clip_unnamed, then set the "did_set_selection" flag.
When restoring clip_unnamed, check if "did_set_selection" was set, and
if it is call clip_own_selection() to get the current unnamed register
into the clipboard.

It's a bit sketchy, there are probably a few more details to work out,
but basically it would work to avoid putting text on the clipboard many
times and still end up with the correct clipboard contents in the end.
If we can make this work properly we can avoid users having to be aware
of this and just enjoy the much faster operation.


--
GALAHAD: Camelot ...
LAUNCELOT: Camelot ...
GAWAIN: It's only a model.
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

[Christian Brabandt]

On So, 08 Jun 2014, Bram Moolenaar wrote:

> Christian wrote:
>
> > On So, 01 Jun 2014, Bram Moolenaar wrote:
> > > When restoring clip_unnamed, it's not too dificult to set the
> > > clipboard then. Would require checking that it would have been set
> > > (default register changed). Then the remarks about "doesn't work for
> > > some commands" can be changed to "for some commands disabled until the
> > > end". I'm sure this avoids surprises for users.
> >
> > I am not sure how to do it.
>
> This would require a flag, e.g. "did_set_selection". When resetting
> clip_unnamed before doing a global operation, this flag would be reset.
> Then when text is deleted or yanked, clip_own_selection() will be
> called, and if the register used matches the register that would be
> associated with clip_unnamed, then set the "did_set_selection" flag.
> When restoring clip_unnamed, check if "did_set_selection" was set, and
> if it is call clip_own_selection() to get the current unnamed register
> into the clipboard.
>
> It's a bit sketchy, there are probably a few more details to work out,
> but basically it would work to avoid putting text on the clipboard many
> times and still end up with the correct clipboard contents in the end.
> If we can make this work properly we can avoid users having to be aware
> of this and just enjoy the much faster operation.

Okay, I think I understood. Here is a patch.

Best,
Christian
--
Nicht durch Zorn, sondern durch Lachen tötet man.
-- Friedrich Wilhelm Nietzsche (Also sprach Zarathustra)

[Bram Moolenaar]

Almost. This code now uses clip_did_set_selection for two purposes. I
would reset clip_unnamed before calling global_exe(), where you set
clip_did_set_selection to FALSE. Simlarly in ex_listdo().

Oh, and move the calls to clip_own_selection() to a function, instead of
copying this code to three places. That function would then also
restore clip_unnamed.

For consistency the code to "start a sequence of global changes" would
be in one function, and that function containing clip_own_selection()
would be callend the "end" function.

So, something like:

start_global_changes()
clip_unnamed_saved = clip_unnamed;
if (clip_unnamed)
{
clip_unnamed = FALSE;
clip-did_set_selection = FALSE;
}

end_global_changes()
if (clip_unnamed_saved)
{
clip_unnamed = TRUE;
if (clip_did_set_selection)
{
if (clip_unnamed & CLIP_UNNAMED)
clip_own_selection(&clip_star);
else if (clip_unnamed & CLIP_UNNAMED_PLUS)
clip_own_selection(&clip_plus);
}
}

Still sketchy.

For :global I think we can do this always, not only when ndone is large.

--
Microsoft: "Windows NT 4.0 now has the same user-interface as Windows 95"
Windows 95: "Press CTRL-ALT-DEL to reboot"
Windows NT 4.0: "Press CTRL-ALT-DEL to login"

[Christian Brabandt]

Hi Bram!

On Do, 12 Jun 2014, Bram Moolenaar wrote:

> Almost. This code now uses clip_did_set_selection for two purposes. I
> would reset clip_unnamed before calling global_exe(), where you set
> clip_did_set_selection to FALSE. Simlarly in ex_listdo().

I wouldn't. Saves us an another global variable and makes
adjust_clip_reg() just work. And it isn't actually needed is it?

> Oh, and move the calls to clip_own_selection() to a function, instead of
> copying this code to three places. That function would then also
> restore clip_unnamed.

Ok that would work. Here is an updated patch.

Mit freundlichen Grüßen
Christian
--

Die Astrologie ist eine Form von Aberglauben, die sich anmaßt, Gott in
die Karten zu schauen.
-- Hoimar von Ditfurth

[Praful]

On Friday, 13 June 2014 15:50:50 UTC+1, Christian Brabandt wrote:
> Hi Bram!
>
>
>
> On Do, 12 Jun 2014, Bram Moolenaar wrote:
>
>
>
> > Almost. This code now uses clip_did_set_selection for two purposes. I
>
> > would reset clip_unnamed before calling global_exe(), where you set
>
> > clip_did_set_selection to FALSE. Simlarly in ex_listdo().
>
>
>
> I wouldn't. Saves us an another global variable and makes
>
> adjust_clip_reg() just work. And it isn't actually needed is it?
>
>
>
> > Oh, and move the calls to clip_own_selection() to a function, instead of
>
> > copying this code to three places. That function would then also
>
> > restore clip_unnamed.
>
>
>
> Ok that would work. Here is an updated patch.

I have tried this patch and it made no difference! It's still slow when I try g!/<pattern>/d. Using d_ was fine as expected.

This is Windows 7 x64.

Praful

[Christian Brabandt]

Sorry, typo. Please try this updated patch.

Best,
Christian
--
Die Liebe ist unsere Strafe dafür, daß wir es nicht einfach bei der
Fortpflanzung bewenden lassen.
-- Helmar Nahr

[Praful]

On Wednesday, 18 June 2014 18:02:20 UTC+1, Christian Brabandt wrote:
>
> Sorry, typo. Please try this updated patch.
>

Thanks for the quick response, Christian. Sorry, same again. I still have issue.

Praful

[Christian Brabandt]

On Mi, 18 Jun 2014, Praful wrote:

Hm, not sure, why it doesn't work...

Best,
Christian
--
Bessere Lieder müßten sie mir singen, daß ich an ihren Erlöser glauben
lerne. Erlöster müßten mir seine Jünger aussehen.
-- Friedrich Wilhelm Nietzsche (Zarathrusta II, Von den Priestern)

[Christian Brabandt]

On Mi, 18 Jun 2014, Christian Brabandt wrote:

> On Mi, 18 Jun 2014, Praful wrote:
>
> > On Wednesday, 18 June 2014 18:02:20 UTC+1, Christian Brabandt wrote:
> > >
> > > Sorry, typo. Please try this updated patch.
> > >
> >
> > Thanks for the quick response, Christian. Sorry, same again. I still have issue.
>
> Hm, not sure, why it doesn't work...

Perhaps, it's clip_gen_set_selection() that is slow?

Best,
Christian
--
Was nützt es dem Menschen, wenn er Lesen und Schreiben gelernt hat,
aber das Denken anderen überläßt?
-- Ernst R. Hauschka

[Praful]

On Wednesday, 18 June 2014 22:37:47 UTC+1, Christian Brabandt wrote:

> On Mi, 18 Jun 2014, Christian Brabandt wrote:
>
>
> Perhaps, it's clip_gen_set_selection() that is slow?
>

Sorry, still slow! Is there any debugging information I can provide to help you diagnose what's going on?

In case you've forgotten, your original solution did work even though it wasn't as generic as Bram wanted. It could be there is no generic solution and you have to settle for a specific solution.

Thanks

Praful

[Christian Brabandt]

Thanks for the feedback. I'll need to get a Windows setup and try to
reproduce there to be able to help, rather then guessing. Anybody knows
a good way to profile Vim using MSVC?

Best,
Christian
--

[Christian Brabandt]

On Do, 19 Jun 2014, Praful wrote:

Please try the attached patch. It was indeed clip_gen_set_selection()
that was slow, but the previous patch was still a little bit wrong.

Please also make sure, the clipboard still works. I have tested it
locally and instead of 15 seconds for a :g/^/y command it took 2.5
seconds (as much as with :set clipboard=), while still copying the last
yank into the clipboard.

For the records, I did profiling using ltprof. This worked quite well.

Best,
Christian
--
Computer können Dinge tun, die wir nur tun können, indem wir darüber
nachdenken.

[Praful]

Hi Christian

Excellent news: it is now working! I can confirm that the item that remains on the clipboard after issuing g/<pattern/d is the last row that was deleted.

Thank you for persevering with this issue!

Praful

[Praful]

On Thursday, 26 June 2014 14:42:29 UTC+1, Praful wrote:

>
> Hi Christian
>
>
>
> Excellent news: it is now working! I can confirm that the item that remains on the clipboard after issuing g/<pattern/d is the last row that was deleted.
>
>
>
> Thank you for persevering with this issue!
>
>
>
> Praful

I've lost track of this issue. Given that it's fixed, when will this be included in the official patches? When I now patch vim, I have to undo Christian's patch then apply the official patches then redo Christian's patch.

Thanks

Praful

[Christian Brabandt]

My guess is, its in Brams todo list and will take some time until
Bram comes to include this patch.

Best,
Christian
--
Die Arznei macht Kranke, die Mathematik traurige, die Theologie
sündhafte Menschen.
-- Martin Luther

[Daniel Drucker]

On Thursday, July 17, 2014 at 4:55:50 PM UTC-4, Christian Brabandt wrote:
> On Do, 17 Jul 2014, Praful wrote:
>
> > On Thursday, 26 June 2014 14:42:29 UTC+1, Praful wrote:
> >
> > >
> > > Hi Christian
> > >
> > >
> > >
> > > Excellent news: it is now working! I can confirm that the item that remains on the clipboard after issuing g/<pattern/d is the last row that was deleted.
> > >
> > >
> > >
> > > Thank you for persevering with this issue!
> > >
> > >
> > >
> > > Praful
> >
> > I've lost track of this issue. Given that it's fixed, when will this be included in the official patches? When I now patch vim, I have to undo Christian's patch then apply the official patches then redo Christian's patch.
>
> My guess is, its in Brams todo list and will take some time until
> Bram comes to include this patch.

It's now 2017, and it doesn't appear that this has been included yet in:

VIM - Vi IMproved 8.0 (2016 Sep 12, compiled Mar 20 2017 15:55:48)
Included patches: 1-494


Does anyone know why not?


Daniel

[Christian Brabandt]

I think this has been included long ago with 7.4.396


Best,
Christian
--
Originalität ist die Kunst, sich Bonmots zu merken und zu vergessen,
von wem sie stammen.
-- Danny Kaye (eig. Daniel David Sominski)

[Daniel Drucker]

On Monday, May 8, 2017 at 2:19:43 PM UTC-4, Christian Brabandt wrote:
> On Mo, 08 Mai 2017, Daniel Drucker wrote:
>
> > On Thursday, July 17, 2014 at 4:55:50 PM UTC-4, Christian Brabandt wrote:
> > > On Do, 17 Jul 2014, Praful wrote:
> > >
> > > > On Thursday, 26 June 2014 14:42:29 UTC+1, Praful wrote:
> > > >
> > > > >
> > > > > Hi Christian
> > > > >
> > > > >
> > > > >
> > > > > Excellent news: it is now working! I can confirm that the item that remains on the clipboard after issuing g/<pattern/d is the last row that was deleted.
> > > > >
> > > > >
> > > > >
> > > > > Thank you for persevering with this issue!
> > > > >
> > > > >
> > > > >
> > > > > Praful
> > > >
> > > > I've lost track of this issue. Given that it's fixed, when will this be included in the official patches? When I now patch vim, I have to undo Christian's patch then apply the official patches then redo Christian's patch.
> > >
> > > My guess is, its in Brams todo list and will take some time until
> > > Bram comes to include this patch.
> >
> >
> > It's now 2017, and it doesn't appear that this has been included yet in:
> >
> > VIM - Vi IMproved 8.0 (2016 Sep 12, compiled Mar 20 2017 15:55:48)
> > Included patches: 1-494
> >
> >
> > Does anyone know why not?
>
> I think this has been included long ago with 7.4.396

Hmm. I wonder what's going on in mine then? In my config:
https://gist.github.com/dmd/70aaef4072c7e8d78018cef5ecc28903

with a 200000 line file, v/mystring/d takes about 8 minutes with set clipboard=unnamed, or less than a second without.

Is there something I can do to debug why this is happening?

Daniel

[Christian Brabandt]

Try to bisect, if this is a recent regression.


Best,
Christian
--
Wußten Sie schon...
... daß auch Kettenraucher an Eisenmangel leiden können?

[Daniel Drucker]

I am getting the same result in 7.4.396:
https://gist.github.com/a2c22023308cbc80f88efb3cf22ccb00

Slow with set clipboard=unnamed, fast without.
The only thing in .vimrc is set clipboard=unnamed. Removing that makes it fast.

Now what?

Daniel

[Daniel Drucker]

Unsetting DISPLAY fixes this for me, if that's a clue.

Daniel

[Daniel Drucker]

(But, of course, then the clipboard doesn't actually WORK, so that's not really a fix.)

Daniel

[Daniel Drucker]

This is what it's spending its time doing:

0.000018 poll([{fd=3, events=POLLIN}], 1, 0) = 0 (Timeout)
0.000018 select(4, [0 3], NULL, [0], {0, 0}) = 0 (Timeout)
0.000052 recvmsg(3, 0x7fff09efec00, 0) = -1 EAGAIN (Resource temporarily unavailable)
0.000018 recvmsg(3, 0x7fff09efec00, 0) = -1 EAGAIN (Resource temporarily unavailable)

over and over, thousands upon thousands of times.

Daniel