Slow Python syntax highlighting

[François Ingelrest]

Hi all,

I'm not sure when this started, but for some Python files there's a
noticeable lag due to syntax highlighting.

For example, using O to open a new line can take up to a few hundreds
of ms at some places of the file while it's instantaneous at some
other places of the same file. I say it's caused by syntax
highlighting because disabling it solves the issue. I couldn't find a
pattern explaining why some parts are slower.

The file is not a big one (457 lines / 17kB), its doesn't have long
lines (max is around 250 chars) and I'm using the syntax file provided
by Vim, not a custom one. I don't use any plugins at all (related to
Python or not).

I tried using :syntime (i.e., on / off / report) but I couldn't spot a
difference between a 'fast' place and a 'slow' place. I may not have
used it properly.

I'm using a huge 8.0.19 build on a 64 bits Debian Jessie. I'm using
Vim in a Gnome terminal, I don't use GVim.

Any suggestions where to look for?

Thanks.

[Dominique Pellé]

Can you provide a file where it happens and a minimalistic
vimrc, just enough to reproduce it?

Thanks
Dominique

[François Ingelrest]

Hi Dominique,

On 1 October 2016 at 09:46, Dominique Pellé wrote:
> Can you provide a file where it happens and a minimalistic
> vimrc, just enough to reproduce it?

Here's what I came up with.

If I type o to open a new line:
- On line 87, it's instantaneous
- On line 85, there's a noticeable lag

With :syntax off, it's instantaneous in both cases. It may have
something to do with indent as well, because turning off indent in the
.vimrc solves the issue as well.

[Dominique Pellé]

I downloaded your 2 files and started vim-8.0.20 huge on
Linux x86_64, compiled with gcc -O2 in a terminal like this:

$ vim -u -u vimrc --noplugin +85 foo.py

Then I pressed O. I did not observe an obvious difference
between lines 85 and 87. I then measured it and saw a difference:

3 measurement at line 85:

$ time ./vim -u vimrc --noplugin +85 -c 'norm! O' -c 'q!' foo.py

real 0m0.365s
user 0m0.230s
sys 0m0.033s

$ time ./vim -u vimrc --noplugin +85 -c 'norm! O' -c 'q!' foo.py

real 0m0.359s
user 0m0.221s
sys 0m0.035s

$ time ./vim -u vimrc --noplugin +85 -c 'norm! O' -c 'q!' foo.py

real 0m0.357s
user 0m0.239s
sys 0m0.019s

3 measurements at line 87:

$ time ./vim -u vimrc --noplugin +87 -c 'norm! O' -c 'q!' foo.py

real 0m0.267s
user 0m0.153s
sys 0m0.012s

$ time ./vim -u vimrc --noplugin +87 -c 'norm! O' -c 'q!' foo.py

real 0m0.269s
user 0m0.137s
sys 0m0.028s

$ time ./vim -u vimrc --noplugin +87 -c 'norm! O' -c 'q!' foo.py

real 0m0.267s
user 0m0.144s
sys 0m0.021s

OK, command O is measurably slower at line 85, but from your
email, it seemed like you observed a much longer delay than what
I see.

Interestingly, trying the old vim-7.4.52 (that comes with Ubuntu-14.04)
I see that it's much faster and takes about the same amount of time at line
at line 85 or 87:

$ time /usr/bin/vim -u vimrc --noplugin +85 -c 'norm! O' -c 'q!' foo.py

real 0m0.185s
user 0m0.059s
sys 0m0.027s

$ time /usr/bin/vim -u vimrc --noplugin +87 -c 'norm! O' -c 'q!' foo.py

real 0m0.176s
user 0m0.056s
sys 0m0.017s

I did not have the time yet to perform a bisection yet to see when
performance degraded, but I'll try later.

Can you also try to measure it automatically as I did? Just to make
sure that what I measure is what you also observe.

Regards
Dominique

[François Ingelrest]

On 2 October 2016 at 22:29, Dominique Pellé wrote:
> Can you also try to measure it automatically as I did? Just to make
> sure that what I measure is what you also observe.

This is what I got with line 85 (the slow one):

% time vim -u ./vimrc --noplugin +85 -c 'norm! o' -c 'q!' foo.py
0.17s user 0.00s system 63% cpu 0.277 total

% time vim -u ./vimrc --noplugin +85 -c 'norm! o' -c 'q!' foo.py
0.14s user 0.03s system 63% cpu 0.272 total

% time vim -u ./vimrc --noplugin +85 -c 'norm! o' -c 'q!' foo.py
0.16s user 0.02s system 62% cpu 0.275 total


And this is for line 87 (the fast one):

% time vim -u ./vimrc --noplugin +87 -c 'norm! o' -c 'q!' foo.py
0.05s user 0.01s system 98% cpu 0.057 total

% time vim -u ./vimrc --noplugin +87 -c 'norm! o' -c 'q!' foo.py
0.05s user 0.00s system 32% cpu 0.149 total

% time vim -u ./vimrc --noplugin +87 -c 'norm! o' -c 'q!' foo.py
0.04s user 0.00s system 30% cpu 0.146 total


Visually I can tell that the second line executes faster than the
first one. Also note that I used 'o' and not 'O'. With 'O', line 87
takes more time but is still faster than line 85. There's not much
difference for line 85.

[Christian Brabandt]

Hi François!

I haven't checked yet, but could that possibly be a duplicate of
https://github.com/vim/vim/issues/1098

Are you using python indent plugin and does it work better, if you
disable the indent plugin?

Best,
Christian
--
Letzte Worte von General Custer:
"Wo kommen diese verdammten Indianer her?"

[François Ingelrest]

Hi Christian,

On 3 October 2016 at 10:30, Christian Brabandt wrote:
> I haven't checked yet, but could that possibly be a duplicate of
> https://github.com/vim/vim/issues/1098
>
> Are you using python indent plugin and does it work better, if you
> disable the indent plugin?

Yes indenting is enabled in the vimrc, and disabling it solves the
issue as well (disabling syntax highlighting or indenting).

As for the github report it may be the same problem that causes the
issue, but in my test case there's no complex string on the 'slow'
line.