[vim/vim] quickfix performance regression on Windows (#7033)

[markonm]

Describe the bug
Interaction with quickfix window and quickfix list after patch 8.0.1781 suffered slowdown on Windows.

To Reproduce

" measureQuickFix.vim
let filename = tempname()
silent execute 'w' filename
let lines = map(repeat([filename], 10000), '{"filename": v:val}')

let start = reltime()
call setqflist(lines)
echom '  setqflist:' reltimestr(reltime(start))

let start = reltime()
silent copen
echom '  copen:    ' reltimestr(reltime(start))

let start = reltime()
call setqflist(lines)
echom '  setqflist:' reltimestr(reltime(start))

Run: gvim --clean -S measureQuickFix.vim

Output of above script:

v8.0.1780:

setqflist:   0.009004
copen:       0.009129
setqflist:   0.020638

v8.0.1781

setqflist:   0.009413
copen:       3.745755
setqflist:   3.794615

:copen and setqflist are much slower with 8.0.1781 included. Results are the same on the current master.

Expected behavior
Interaction with quickfix is fast as it was before patch 8.0.1781 was included.

Environment (please complete the following information):

• OS: Windows 10 2004

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

[Yegappan Lakshmanan]

Hi,

Can you try the attached patch?

- Yegappan

[vim-dev ML]

Hi,

On Sun, Sep 27, 2020 at 11:16 AM markonm <vim-dev...@256bit.org> wrote:

> *Describe the bug*

> Interaction with quickfix window and quickfix list after patch 8.0.1781

> <https://github.com/vim/vim/commit/a796d46f29e3cc235cc981696d7ee80faccb5000>
> suffered slowdown on Windows.
>
> *To Reproduce*

> *Expected behavior*

> Interaction with quickfix is fast as it was before patch 8.0.1781 was
> included.
>

> *Environment (please complete the following information):*
>
> - OS: Windows 10 2004

>
>
>
Can you try the attached patch?

- Yegappan

[K.Takata]

Can you try the attached patch?

The link is here: https://groups.google.com/d/msg/vim_dev/bau3jTheHMY/y2Z8d-StAQAJ

—
You are receiving this because you commented.

[markonm]

Thank you for the patch. I can see a big improvement in initial test.

v8.2.1767 with qfexpand.diff outputs:

setqflist:   0.011608
copen:       0.012365
setqflist:   0.019360

I also made a new test. This one has 46947 entries spread across 2307 files. The old one had 10000 entries and only 1 file.

" measureQuickFix2.vim
silent grep! /S "file" *
let qf = getqflist()

let start = reltime()
call setqflist(qf)
echom 'setqflist:' reltimestr(reltime(start))

let start = reltime()
silent copen
echom 'copen:    ' reltimestr(reltime(start))

let start = reltime()
call setqflist(qf)
echom 'setqflist:' reltimestr(reltime(start))

Steps used:

1. git clone --depth 1 https://github.com/vim/vim
2. cd vim
3. gvim --clean -S measureQuickFix2.vim

It produced the following output:

v8.0.1780

setqflist:   0.053862
copen:       0.031189
setqflist:   0.084630

v8.0.1781

setqflist:   0.052574
copen:      15.808420
setqflist:  16.696758

v8.2.1767

setqflist:   0.059002
copen:      16.761416
setqflist:  16.750146

v8.2.1767 with qfexpand.diff

setqflist:   0.058875
copen:       0.916748
setqflist:   0.946482

Although the patch greatly improves performance, interaction with quickfix is still a bit slower than in v8.0.1780.

—
You are receiving this because you commented.

[Yegappan Lakshmanan]

Hi,

The changes made by 1781 fixes the issue reported in

https://github.com/vim/vim/issues/2846

This change shortens the file name for each buffer. So the overhead

of doing this will increase the time it takes to populate a quickfix list

or to display the entries in the quickfix buffer (proportional to the number

of unique buffers in the quickfix list). We can try to optimize the time

it takes to shorten the buffer name. But we won't be able to get back

to the same time as before this change.

Regards,

Yegappan

 

[vim-dev ML]

Hi,

On Mon, Sep 28, 2020 at 9:03 AM markonm <vim-dev...@256bit.org> wrote:

> Thank you for the patch. I can see a big improvement in initial test.
>
> v8.2.1767 with qfexpand.diff outputs:
>
> setqflist: 0.011608
> copen: 0.012365
> setqflist: 0.019360
>
> I also made a new test. This one has 46947 entries spread across 2307
> files. The old one had 10000 entries and only 1 file.
>
> " measureQuickFix2.vim
> silent grep! /S "file" *
> let qf = getqflist()
>

> let start = reltime()
> call setqflist(qf)

> echom 'setqflist:' reltimestr(reltime(start))
>
> let start = reltime()
> silent copen
> echom 'copen: ' reltimestr(reltime(start))
>
> let start = reltime()

> call setqflist(qf)
> echom 'setqflist:' reltimestr(reltime(start))
>
> Steps used:
>

> 1. git clone --depth 1 https://github.com/vim/vim
> 2. cd vim
> 3. gvim --clean -S measureQuickFix2.vim

[Bram Moolenaar]

Yegappan wrote:

> On Sun, Sep 27, 2020 at 11:16 AM markonm <vim-dev...@256bit.org> wrote:
>

> > *Describe the bug*

> > Interaction with quickfix window and quickfix list after patch 8.0.1781

> > <https://github.com/vim/vim/commit/a796d46f29e3cc235cc981696d7ee80faccb5000>
> > suffered slowdown on Windows.
> >
> > *To Reproduce*
> >

> > *Expected behavior*

> > Interaction with quickfix is fast as it was before patch 8.0.1781 was
> > included.
> >

> > *Environment (please complete the following information):*
> >
> > - OS: Windows 10 2004

>
> Can you try the attached patch?

Since it's reported to be a nice improvement, can you make a pull
request for this?

--
The Feynman problem solving Algorithm:
1) Write down the problem
2) Think real hard
3) Write down the answer

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[vim-dev ML]

Yegappan wrote:

> On Sun, Sep 27, 2020 at 11:16 AM markonm <vim-dev...@256bit.org> wrote:
>
> > *Describe the bug*

> > Interaction with quickfix window and quickfix list after patch 8.0.1781

> > <https://github.com/vim/vim/commit/a796d46f29e3cc235cc981696d7ee80faccb5000>
> > suffered slowdown on Windows.
> >
> > *To Reproduce*
> >

> > *Expected behavior*

> > Interaction with quickfix is fast as it was before patch 8.0.1781 was
> > included.
> >

> > *Environment (please complete the following information):*
> >
> > - OS: Windows 10 2004
>
> Can you try the attached patch?

Since it's reported to be a nice improvement, can you make a pull
request for this?

--
The Feynman problem solving Algorithm:
1) Write down the problem
2) Think real hard
3) Write down the answer

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Yegappan Lakshmanan]

Hi Bram,

I have created PR #7039 for this.

https://github.com/vim/vim/pull/7039

Regards,

Yegappan

[vim-dev ML]

Hi Bram,

On Mon, Sep 28, 2020 at 12:42 PM Bram Moolenaar <Br...@moolenaar.net> wrote:

>
> Yegappan wrote:
>
> > On Sun, Sep 27, 2020 at 11:16 AM markonm <vim-dev...@256bit.org>
> wrote:
> >
> > > *Describe the bug*

> > > Interaction with quickfix window and quickfix list after patch 8.0.1781
> > > <

> https://github.com/vim/vim/commit/a796d46f29e3cc235cc981696d7ee80faccb5000
> >
> > > suffered slowdown on Windows.
> > >
> > > *To Reproduce*
> > >

> > > *Expected behavior*

> > > Interaction with quickfix is fast as it was before patch 8.0.1781 was
> > > included.
> > >

> > > *Environment (please complete the following information):*
> > >
> > > - OS: Windows 10 2004
> >
> > Can you try the attached patch?
>
> Since it's reported to be a nice improvement, can you make a pull
> request for this?
>
>
>
I have created PR #7039 for this.

https://github.com/vim/vim/pull/7039

Regards,
Yegappan

[Bram Moolenaar]

Closed #7033 via 8ec92c9.

—
You are receiving this because you commented.

[markonm]

To further eliminate the delay the following patch is needed. mch_isFullName is much slower on Windows because it calls mch_FullName.

diff --git a/src/os_mswin.c b/src/os_mswin.c
index 80009533c..429d93970 100644
--- a/src/os_mswin.c
+++ b/src/os_mswin.c
@@ -387,21 +387,10 @@ mch_FullName(
     int
 mch_isFullName(char_u *fname)
 {
-    // WinNT and later can use _MAX_PATH wide characters for a pathname, which
-    // means that the maximum pathname is _MAX_PATH * 3 bytes when 'enc' is
-    // UTF-8.
-    char szName[_MAX_PATH * 3 + 1];
-
     // A name like "d:/foo" and "//server/share" is absolute
-    if ((fname[0] && fname[1] == ':' && (fname[2] == '/' || fname[2] == '\\'))
-	    || (fname[0] == fname[1] && (fname[0] == '/' || fname[0] == '\\')))
-	return TRUE;
-
-    // A name that can't be made absolute probably isn't absolute.
-    if (mch_FullName(fname, (char_u *)szName, sizeof(szName) - 1, FALSE) == FAIL)
-	return FALSE;
-
-    return pathcmp((const char *)fname, (const char *)szName, -1) == 0;
+    return ((ASCII_ISALPHA(fname[0]) && fname[1] == ':'
+	    && (fname[2] == '/' || fname[2] == '\\'))
+	    || (fname[0] == fname[1] && (fname[0] == '/' || fname[0] == '\\')));
 }
 
 /*

Similar thing is also done in Neovim: https://github.com/neovim/neovim/blob/bfed67e00ecdf71e0c7d17b1fd802f223b42c800/src/nvim/path.c#L2294

Measured with measureQuickFix2.vim

v8.0.1780

setqflist:   0.053862
copen:       0.031189
setqflist:   0.084630

v8.0.2122

setqflist:   0.065174
copen:       0.907272
setqflist:   0.977449

v8.0.2122 with the patch

setqflist:   0.059616
copen:       0.038923
setqflist:   0.104571

Now the delay is almost completely gone.

—
You are receiving this because you commented.