long line with syntax highlighting crashes vim w/ 100% CPU

[John Whitley]

### Repro steps

1. From this gist[1], download the files `killvimrc` and `longline.rb`.
2. Run vim with `vim -u killvimrc longline.rb`.

### Actual results

`vim` locks up at 100% CPU utilization and must be killed with `kill -9 ...`

### Expected results

File loads and is editable.

### Notes

Reproduced with vim 7.4.488 and 7.4.540 (hg HEAD at time of writing).

This issue appears to be tied to the vim ruby syntax file. Removing the line `syntax on` from `killvimrc` allows the file to load, as does renaming `longline.rb` to `longline.txt`. As expected, loading the file as a .txt file and running `:set ft=ruby` will trigger the lockup.

Bugreport.txt for the 7.4.488 run is in a separate gist at [2].

Also reported as neovim/neovim#1161 [3].

This appears to be related to earlier threads on long lines in the presence of syntax highlighting, and VIM 7.4 patches 497, 519, 526, 527. [4][5][6]

[1] https://gist.github.com/jwhitley/080d53d3e6a83ce8fe61

[2] https://gist.github.com/jwhitley/2ecd483b0cb3cca0657c

[3] https://github.com/neovim/neovim/issues/1611

[4] https://groups.google.com/forum/#!msg/vim_dev/Ge7eVpTuyx0/iYKt_rZB1v0J

[5] https://groups.google.com/forum/#!msg/vim_dev/JjFYb7j1WBw/YMd3h6VLS90J

[6] https://code.google.com/p/vim/issues/detail?id=274

-- John

[Bram Moolenaar]

John Whitley wrote:

> ### Repro steps
>
> 1. From this gist[1], download the files `killvimrc` and `longline.rb`.
> 2. Run vim with `vim -u killvimrc longline.rb`.
>
> ### Actual results
>

> `vim` locks up at 100% CPU utilization and must be killed with `kill -9 ...=

> `
>
> ### Expected results
>
> File loads and is editable.
>
> ### Notes
>
> Reproduced with vim 7.4.488 and 7.4.540 (hg HEAD at time of writing).
>

> This issue appears to be tied to the vim ruby syntax file. Removing the li=
> ne `syntax on` from `killvimrc` allows the file to load, as does renaming `=
> longline.rb` to `longline.txt`. As expected, loading the file as a .txt fi=
> le and running `:set ft=3Druby` will trigger the lockup.

>
> Bugreport.txt for the 7.4.488 run is in a separate gist at [2].
>
> Also reported as neovim/neovim#1161 [3].
>

> This appears to be related to earlier threads on long lines in the presence=

> of syntax highlighting, and VIM 7.4 patches 497, 519, 526, 527. [4][5][6]
>
> [1] https://gist.github.com/jwhitley/080d53d3e6a83ce8fe61
>
> [2] https://gist.github.com/jwhitley/2ecd483b0cb3cca0657c
>
> [3] https://github.com/neovim/neovim/issues/1611
>
> [4] https://groups.google.com/forum/#!msg/vim_dev/Ge7eVpTuyx0/iYKt_rZB1v0J
>
> [5] https://groups.google.com/forum/#!msg/vim_dev/JjFYb7j1WBw/YMd3h6VLS90J
>

> [6] https://code.google.com/p/vim/issues/detail?id=3D274

It would be good if we implement the time limit for the NFA engine.

--
Hanson's Treatment of Time:
There are never enough hours in a day, but always too
many days before Saturday.

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Ben Fritz]

On Thursday, December 4, 2014 3:51:05 PM UTC-6, John Whitley wrote:
> ### Repro steps
>
> 1. From this gist[1], download the files `killvimrc` and `longline.rb`.
> 2. Run vim with `vim -u killvimrc longline.rb`.
>
> ### Actual results
>
> `vim` locks up at 100% CPU utilization and must be killed with `kill -9 ...`
>
> ### Expected results
>
> File loads and is editable.
>
> ### Notes
>
> Reproduced with vim 7.4.488 and 7.4.540 (hg HEAD at time of writing).
>
> This issue appears to be tied to the vim ruby syntax file. Removing the line `syntax on` from `killvimrc` allows the file to load, as does renaming `longline.rb` to `longline.txt`. As expected, loading the file as a .txt file and running `:set ft=ruby` will trigger the lockup.
>

https://groups.google.com/forum/#!topic/vim_dev/tTyRMSakpSI

From this discussion, Ruby highlight was especially slow after the introduction of the new regex engine, due to frequent use of zero-width look-behind assertions \@<! and \@<= without using the character-limiting features added to address the problem of their slowness in the new engine (:help /\@<= , :help /\@<! , scroll down to the part about limiting by bytes).

Looking at https://github.com/vim-ruby/vim-ruby/blob/master/syntax/ruby.vim this issue was never fully addressed, only for one of the occurrences.

Perhaps a new issue should be created in the bug tracker on that github to use the byte limits wherever possible.

[Daniel Hahler]

Am Montag, 8. Dezember 2014 17:26:32 UTC+1 schrieb Ben Fritz:

> https://groups.google.com/forum/#!topic/vim_dev/tTyRMSakpSI
>
> From this discussion, Ruby highlight was especially slow after the introduction of the new regex engine, due to frequent use of zero-width look-behind assertions \@<! and \@<= without using the character-limiting features added to address the problem of their slowness in the new engine (:help /\@<= , :help /\@<! , scroll down to the part about limiting by bytes).
>
> Looking at https://github.com/vim-ruby/vim-ruby/blob/master/syntax/ruby.vim this issue was never fully addressed, only for one of the occurrences.
>
> Perhaps a new issue should be created in the bug tracker on that github to use the byte limits wherever possible.

Done: https://github.com/vim-ruby/vim-ruby/issues/243


Thanks,
Daniel.