[vim/vim] E363 when opening json file containing long base64 lines (#4885)

[kkew3]

Describe the bug
E363 error occurs when opening json file containing long lines.

To Reproduce

1. Download the attached example file (base64.txt: 128K), and rename it to base64.json)
2. Run vim --clean base64.json)
3. The error occurs (see below for complete error)

The error message:

E363: pattern uses more memory than 'maxmempattern'
Press ENTER or type command to continue
'redrawtime' exceeded, syntax highlighting disabled
Press ENTER or type command to continue

Expected behavior
No error, since 128K is not large memory consumption, and other text editor, say, Sublime text, has no problem with it.

Environment (please complete the following information):

• Vim version: VIM - Vi IMproved 8.1 (2018 May 18, compiled Jul 28 2019 01:07:30); Included patches: 1-1722
• OS: macOS 10.14.6
• Terminal: iTerm2

** Additional Context **
The attached base64.txt contains shuffled base64 image but it shouldn't affect the problem describe above.

base64.txt

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub

[Tony Mechelynck]

1. What is 'regexpengine' set to? If set to 0, does it change anything if you set it to 1?
2. By default, 'maxmempattern' allocates 1 MB (i.e. 1000 kB) to the search engine. Does it help if you increase the setting (in kilobytes)?

Best regards,
Tony.

[Arseny Nasokin]

Default value of regexpengine is 0. Setting it to 1 and/or setting maxmempattern to 10000 doesn't help.

The problem is somewhere in highlighting. if I turn it off, everything is fast and smooth.

[Tony Mechelynck]

Aha! Well, highlighting is normally the responsibility of the syntax plugin, and the problem with that is that there is currently no maintainer for $VIMRUNTIME/syntax/json.vim.

Do you think you can find where the problem lies, and fix it?

Best regards,
Tony.

[Arseny Nasokin]

I think this is the same problem as any syntax file in vim has. Vim has the great job with telling a user "I'll run highlighting on a in-memory buffer. It's not the best one, but it's fast. I think, the same should be with very long lines.

e.g. I have a SQL file with very long lines. It's guaranteed, that I'll have problem with highlighting. If I chop lines to shorter ones, like 200-500 chars at most, I'll have no problem with syntax even I have a 500mb file

[Christian Brabandt]

well, as the warning indicates, please adjust your redrawtime and maxmempattern option values.

I feel like handling base64 inside a runtime script is nasty enough, that I wouldn't consider it a bug. However you might also contact the syntax script perhaps he has an idea how to further improve the syntax script.

[Christian Brabandt]

Closed #4885.

[Tony Mechelynck]

well, as the warning indicates, please adjust your redrawtime and maxmempattern option values.

I feel like handling base64 inside a runtime script is nasty enough, that I wouldn't consider it a bug. However you might also contact the syntax script perhaps he has an idea how to further improve the syntax script.

As I said earlier, the json syntax script is currently unmaintained:

" Vim syntax file
" Language:	JSON
" Maintainer:	vacancy
" Previous Maintainer:	Eli Parra <e...@elzr.com>
" Last Change:	2019 Jul 08
" Version:      0.12

One might try to contact the listed "previous maintainer" but he may or may not answer.

Best regards,
Tony.

[Christian Brabandt]

ah, I saw last change header and somehow thought the previous maintainer was the actual maintainer.