[vim/vim] regression, slow draw speed during tags insert-mode completion (#5141)
Kedar Hirve
Describe the bug
When I use CTRL-P insert-mode completion, as I repeatedly hit CTRL-P to iterate through matching items, the cursor jumps around to the length of each completed entry but the text of the entry to be inserted doesn't show until I hit escape or the tags scan completely finishes.
I have about 2 GB total of tags files, and so tags scanning takes upwards of 5 seconds. Still, this worked perfectly in vim 8.0.776 but is basically so bad in vim 8.1.2231 that I have to disable tags scanning.
To Reproduce
Detailed steps to reproduce the behavior:
use a vimrc with only the following lines:
set completeopt=
set nocompatible
set tags={path to my tags files}
Edit a file with a lot of keywords starting with the same prefix, for example "ModuleA"
go into insert mode and type ModuleA and hit CTRL-P a bunch of times
Expected behavior
Each time I hit CTRL-P I should see the full text of the potential entry to complete and insert.
Actual behavior
The cursor moves around after every completion to show the length of the entry it would insert, but the text of the entry isn't shown.
Screenshots
In the following screenshot, I had put my cursor after "Party" in "PartyLocalChatControl" and hit CTRL-P a few times. I would expect the cursor to stick to the end of a word that it's previewing for insertion, but instead it's just jumping around in the keyword I already had there, with some entry it won't show me until I hit escape.
Environment (please complete the following information):
VIM - Vi IMproved 8.1 (2018 May 18, compiled Oct 28
2019 23:02:10)
MS-Windows 64-bit GUI version with OLE support
Included patches: 1-2231
Compiled by appveyor@APPVYR-WIN
Huge version with GUI. Features included (+) or not (-):
+acl +eval +multi_lang
+tcl/dyn
+arabic +ex_extra +mzscheme/dyn
-termguicolors
+autocmd +extra_search +netbeans_intg
+terminal
+autochdir -farsi +num64
-termresponse
+autoservername +file_in_path +ole
+textobjects
+balloon_eval +find_in_path +packages
+textprop
-balloon_eval_term +float +path_extra
-tgetent
+browse +folding +perl/dyn
+timers
++builtin_terms -footer +persistent_undo
+title
+byte_offset +gettext/dyn -postscript
+toolbar
+channel -hangul_input +printer
+user_commands
+cindent +iconv/dyn +profile
+vartabs
+clientserver +insert_expand +python/dyn
+vertsplit
+clipboard +job +python3/dyn
+virtualedit
+cmdline_compl +jumplist +quickfix
+visual
+cmdline_hist +keymap +reltime
+visualextra
+cmdline_info +lambda +rightleft
+viminfo
+comments +langmap +ruby/dyn
+vreplace
+conceal +libcall +scrollbind -vtp
+cryptv +linebreak +signs
+wildignore
+cscope +lispindent +smartindent
+wildmenu
+cursorbind +listcmds +sound
+windows
+cursorshape +localmap +spell
+writebackup
+dialog_con_gui +lua/dyn +startuptime
-xfontset
+diff +menu +statusline -xim
+digraphs +mksession -sun_workshop
+xpm_w32
+directx +modify_fname +syntax
-xterm_save
-dnd +mouse +tag_binary
-ebcdic +mouseshape -tag_old_static
+emacs_tags +multi_byte_ime/dyn -tag_any_white
OS: Windows 10 18362.1.amd64fre.19h1_release.190318-1202
Terminal: n/a, reproes in GVIM
Additional context
Again, with vim 8.0 (e.g. 8.0.776) on the same PC with exactly the same repro steps (minimal vimrc file, same file to edit, same PC, etc.) it doesn't repro.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
Christian Brabandt
> Am 29.10.2019 um 22:40 schrieb Kedar Hirve <notifi...@github.com>:
>
>
> Describe the bug
> When I use CTRL-P insert-mode completion, as I repeatedly hit CTRL-P to iterate through matching items, the cursor jumps around to the length of each completed entry but the text of the entry to be inserted doesn't show until I hit escape or the tags scan completely finishes.
>
> I have about 2 GB total of tags files, and so tags scanning takes upwards of 5 seconds. Still, this worked perfectly in vim 8.0.776 but is basically so bad in vim 8.1.2231 that I have to disable tags scanning.
>
> To Reproduce
> Detailed steps to reproduce the behavior:
>
> use a vimrc with only the following lines:
> set completeopt=
> set nocompatible
> set tags={path to my tags files}
>
> Edit a file with a lot of keywords starting with the same prefix, for example "ModuleA"
>
> go into insert mode and type ModuleA and hit CTRL-P a bunch of times
>
> Expected behavior
> Each time I hit CTRL-P I should see the full text of the potential entry to complete and insert.
>
> Actual behavior
> The cursor moves around after every completion to show the length of the entry it would insert, but the text of the entry isn't shown.
>
> Screenshots
> In the following screenshot, I had put my cursor after "Party" in "PartyLocalChatControl" and hit CTRL-P a few times. I would expect the cursor to stick to the end of a word that it's previewing for insertion, but instead it's just jumping around in the keyword I already had there, with some entry it won't show me until I hit escape.
>
>
>
Kedar Hirve
Andy Massimino
Possibly 8.1.1094: long line in tags file causes error?
@KedarHMSFT, can you try with verbose?
vim -V5 file.txt
Andy Massimino
Already verbose enough, you should see a message
Kedar Hirve
Possibly 8.1.1094: long line in tags file causes error?
@KedarHMSFT, do you see a message like that?
I've never seen a message like that. Where would you expect it to show up? In the status bar at the bottom? I only see "Scanning tags..." there.
Andy Massimino
Oh sorry, start vim like
vim -V5 foo.txt
then do the tag complete, you should see "Ignoring long line in tags file," in the bottom or in :messages
Kedar Hirve
Aha, learned something new with that verbose setting. Yes, I do see that message pop up, and :messages shows that it was logged a lot of times (over a hundred at least). I wonder what length of line constitutes "long". Maybe I can doctor up my tags file and remove those long lines.
Andy Massimino
Currently, each line in the tags file can be up to 510 chars long, including end-of-line chars.
Normally, vim uses binary search to find a tags in the tag file but reverts to slow linear search when it sees a long line. A simple workaround is to truncate your lines.
What tool is generating long tag lines? If they are legitimate, perhaps vim's limit should be raised at this point?
Kedar Hirve
But long lines themselves shouldn't be the problem, right? The code, at least from those commit descriptions, should be ignoring these lines. And things were working in an earlier version of VIM--I don't know if it was silently ignoring long tags lines or what, but it did find results in my same tags files and didn't have the UI draw problem I'm reporting.
To answer your question, I'm just using ctags. I have a very large repo (Windows source code, and that too I'm only generating tags files for a tiny portion of it). Certain identifiers and file paths themselves approach the 500 character limit. Is there something I need to pass to ctags to make it spit out shorter lines, somehow?
Here's an example that's not even as bad as it gets:
ABI::Windows::Foundation::Collections::IVectorView_impl::GetMany K:\os\public\amd64chk\onecore\external\sdk\inc\abi\windows.foundation.collections.h /^ virtual HRESULT STDMETHODCALLTYPE GetMany(In unsigned startIndex, In unsigned capacity, Out_writes_to(capacity,*actual) T_abi *value, Out unsigned *actual) = 0;$/;" p struct:ABI::Windows::Foundation::Collections::IVectorView_impl access:public signature:(In nsigned startIndex, In unsigned capacity, Out_writes_to(capacity,*actual) T_abi *value, Out unsigned *actual)
And here's one that's really quite awful (but, shockingly, I've actually written code before that calls it):
AddWebAccountForUserAsync K:\os\public\amd64chk\sdk\inc\cppwinrt\winrt\windows.security.authentication.web.provider.h /^ template auto consume_Windows_Security_Authentication_Web_Provider_IWebAccountManagerStatics3::AddWebAccountForUserAsync(Windows::System::User const& user, param::hstring const& webAccountId, param::hstring const& webAccountUserName, param::async_map_view<hstring, hstring> const& props, Windows::Security::Authentication::Web::Provider::WebAccountScope const& scope, param::hstring const& perUserWebAccountId) const$/;" f class:winrt::impl::consume_Windows_Security_Authentication_Web_Provider_IWebAccountManagerStatics3 signature:(Windows::System::User const& user, param::hstring const& webAccountId, param::hstring const& webAccountUserName, param::async_map_view<hstring, hstring> const& props, Windows::Security::Authentication::Web::Provider::WebAccountScope const& scope, param::hstring const& perUserWebAccountId) const
Kedar Hirve
An update: I trimmed out about 150,000 lines that were 500 characters or more from my tags files and now I'm not hitting the bug anymore. I guess somehow the handling of those long lines has managed to make this a problem.
Andy Massimino
I think the problem is that vim makes an assumption about tag line lengths in order to seek around the file to do binary search. Before 8.1.1094 the line length check was there but faulty, and long lines could cause failures later on or just silently not find tags properly. Previously, vim doing the wrong thing generally worked and was faster. The reason it is slow now is because it's using linear search as binary search could (in theory) get stuck.
Depending on the ctags implementation you can try --pattern-length-limit=[N] to shorten the lines. Also using --tag-relative=yes can possibly shorten the filenames. That class: name is still an issue though there may be a way to truncate/disable.
Regarding this broader issue.. the current situation is not very user friendly since there is not even a real warning message, things just go slow. Is there any harm in increasing the limit to, say, 1022 or 2046?
Bram Moolenaar
Andy Massimino wrote:
> I think the problem is that vim makes an assumption about tag line
> lengths in order to seek around the file to do binary search. Before
> 8.1.1094 the line length check was there but faulty, and long lines
> could cause failures later on or just silently not find tags properly.
> Previously, vim doing the wrong thing generally worked and was faster.
> The reason it is slow now is because it's using linear search as
> binary search could (in theory) get stuck.
>
> Depending on the ctags implementation you can try
> --pattern-length-limit=[N] to shorten the lines. Also using
> --tag-relative=yes can possibly shorten the filenames. That class:
> name is still an issue though there may be a way to truncate/disable.
>
> Regarding this broader issue.. the current situation is not very user
> friendly since there is not even a real warning message, things just
> go slow. Is there any harm in increasing the limit to, say, 1022 or
> 2046?
more of the file, which may make it slower. Best would be to
dynamically increase the size when a long line is found. That also
avoids imposing an arbitrary limit, even when it seems big enough (like
500 bytes was longer than what any tags line would be 10 years ago).
--
hundred-and-one symptoms of being an internet addict:
4. Your eyeglasses have a web site burned in on them.
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
Christian Brabandt
answer from Bram, not mirrored here:
https://groups.google.com/d/msg/vim_dev/6YIYymbY0mw/TMTTN9qAAwAJ
Christian Brabandt
On Mi, 30 Okt 2019, Bram Moolenaar wrote:
>
> Andy Massimino wrote:
>
> > I think the problem is that vim makes an assumption about tag line
> > lengths in order to seek around the file to do binary search. Before
> > 8.1.1094 the line length check was there but faulty, and long lines
> > could cause failures later on or just silently not find tags properly.
> > Previously, vim doing the wrong thing generally worked and was faster.
> > The reason it is slow now is because it's using linear search as
> > binary search could (in theory) get stuck.
> >
> > Depending on the ctags implementation you can try
> > --pattern-length-limit=[N] to shorten the lines. Also using
> > --tag-relative=yes can possibly shorten the filenames. That class:
> > name is still an issue though there may be a way to truncate/disable.
> >
> > Regarding this broader issue.. the current situation is not very user
> > friendly since there is not even a real warning message, things just
> > go slow. Is there any harm in increasing the limit to, say, 1022 or
> > 2046?
>
> It's not that simple. Increasing the size also means that Vim reads
> more of the file, which may make it slower. Best would be to
> dynamically increase the size when a long line is found. That also
> avoids imposing an arbitrary limit, even when it seems big enough (like
> 500 bytes was longer than what any tags line would be 10 years ago).
Clearly, we can increase the line length limit by 50%?
Best,
Christian
--
Schließlich, und endlich: was vermisse ich unter meinen Mitmenschen am
meisten: Wirkliche, wirkliche Phantasie.
-- Christian Morgenstern
Bram Moolenaar
It's not like we suddenly accept 90% more, right?
--
hundred-and-one symptoms of being an internet addict:
Christian Brabandt
On Do, 31 Okt 2019, Bram Moolenaar wrote:
> > But it's been 512 for as long as Vim7 exists, so at least 15 years.
> > Clearly, we can increase the line length limit by 50%?
>
> How many more tag files would we accept then?
> It's not like we suddenly accept 90% more, right?
lines and add an option that specifies the tag line length with a
default of 512. This way the user can increase the limit if he runs into
the problem and for all others it should work the same, right?
Best,
Christian
--
Nicht geniale Einseitigkeit, sondern talentvolle Mehrseitigkeit (wie
bei Stainlein) führt im Geschäftleben zu hohen Posten; jene schließt
aus.
-- Jean Paul
Dominique Pellé
The longest line was ~2100 characters, which is
way over 512. What are the consequences?
Slow search or even failure to resolve tags?
Regards
Dominique
Christian Brabandt
On Do, 31 Okt 2019, Dominique Pellé wrote:
> I just checked the tag file of a c++ project a work.
> The longest line was ~2100 characters, which is
> way over 512. What are the consequences?
> Slow search or even failure to resolve tags?
Best,
Christian
--
Portablilität ist für Leute, die keine neuen Programm schreiben können.
-- Linus Torvalds
Bram Moolenaar
Christian wrote:
> On Do, 31 Okt 2019, Bram Moolenaar wrote:
>
> > > But it's been 512 for as long as Vim7 exists, so at least 15 years.
> > > Clearly, we can increase the line length limit by 50%?
> >
> > How many more tag files would we accept then?
> > It's not like we suddenly accept 90% more, right?
>
> True. How about throwing an warning message if the tag file has too long
> lines and add an option that specifies the tag line length with a
> default of 512. This way the user can increase the limit if he runs into
> the problem and for all others it should work the same, right?
I haven't looked at the details but I guess the dynamic size should be
possible.
--
hundred-and-one symptoms of being an internet addict:
K.Takata
I think this has been fixed with 8.1.2269: dc9ef26
If not, feel free to reopen.
K.Takata
Closed #5141.