[vim/vim] Vim9: crash hard to reproduce (#7733)
lacygoill
I have a crash for which I can't get a minimal reproducible example.
I can't get a backtrace because Vim doesn't seem to crash when I compile it with the debugging symbols. I can't get a valgrind log, because valgrind doesn't run on my Ubuntu 16.04; I need to run it in a VM; but it's not easy to reproduce on a different machine; I've tried, it doesn't crash, and the log is useless (it doesn't contain error).
However, I have an asan log:
==20692==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x00000062627e bp 0x7ffdc813e300 sp 0x7ffdc813e2c0 T0)
#0 0x62627d in leave_block /home/user/Vcs/vim/src/ex_eval.c:943
#1 0x63168b in rewind_conditionals /home/user/Vcs/vim/src/ex_eval.c:2392
#2 0x5e909e in do_cmdline /home/user/Vcs/vim/src/ex_docmd.c:1244
#3 0x4273ac in apply_autocmds_group /home/user/Vcs/vim/src/autocmd.c:2109
#4 0x425ae3 in apply_autocmds /home/user/Vcs/vim/src/autocmd.c:1621
#5 0x631b42 in trigger_cmd_autocmd /home/user/Vcs/vim/src/ex_getln.c:73
#6 0x6401a9 in getcmdline_int /home/user/Vcs/vim/src/ex_getln.c:2427
#7 0x63ccd6 in getcmdline /home/user/Vcs/vim/src/ex_getln.c:1553
#8 0x641f19 in getexline /home/user/Vcs/vim/src/ex_getln.c:2722
#9 0x5e7053 in do_cmdline /home/user/Vcs/vim/src/ex_docmd.c:886
#10 0x81a33f in nv_colon /home/user/Vcs/vim/src/normal.c:3381
#11 0x8093ec in normal_cmd /home/user/Vcs/vim/src/normal.c:1098
#12 0xd83f3e in main_loop /home/user/Vcs/vim/src/main.c:1475
#13 0xd8257d in vim_main2 /home/user/Vcs/vim/src/main.c:865
#14 0xd8192b in main /home/user/Vcs/vim/src/main.c:412
#15 0x7f49b966283f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2083f)
#16 0x410178 in _start (/home/user/Vcs/vim/src/vim+0x410178)
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/user/Vcs/vim/src/ex_eval.c:943 leave_block
==20692==ABORTING
I have this written in my vimrc:
set rtp=/home/user/.vim,/home/user/vim-bug/vim-fugitive,/home/user/vim-bug/vim-cmdline
This in ~/.vim/plugin/fugitive.vim:
au CmdlineLeave : if 0
\ | eval 0
\ | endif
This in ~/.vim/plugged/vim-fugitive/plugin/fugitive.vim:
au BufReadPost * "
This in ~/.vim/plugged/vim-cmdline/plugin/cmdline.vim:
vim9
au CmdlineLeave : timer_start(0, () => execute('au SafeState * ++once #'))
And I start Vim with this shell command:
vim --cmd 'filetype on'
Then, I press : to enter the command-line, maybe keep pressing it for a second or two, and press Escape to leave the command-line. Sometimes, Vim crashes, but not all the time.
I tagged the report with Vim9 because it seems that the CmdlineLeave autocmd needs to be installed in a Vim9 script for the crash to have any chance to occur.
I realize the report probably doesn't contain enough info to fix the issue, but since I spent several hours trying to get a MWE, I thought it was still worth reporting what I found. If there's nothing actionable, feel free to close the issue.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.
lacygoill
I forgot to mention that when Vim is started with the previous shell command, it gives these errors:
Error detected while processing /home/user/.vim/plugin/fugitive.vim:
line 2:
E10: \ should be followed by /, ? or &
line 3:
E10: \ should be followed by /, ? or &
Not sure why. I ignore them by pressing Enter.
lacygoill
Also, I've tried to group all the commands in a single file, but without success. It seems they really need to be in different scripts, possibly in different contexts (legacy vs Vim9).
Bram Moolenaar
> Also, I've tried to group all the commands in a single file, but
> without success. It seems they really need to be in different
> scripts, possibly in different contexts (legacy vs Vim9).
variables that were declared inside a block. So yes, the script matters
very much. Look out for any "var" inside a block.
> I forgot to mention that when Vim is started with the previous shell
> command, it gives these errors:
>
> Error detected while processing /home/user/.vim/plugin/fugitive.vim:
> line 2:
> E10: \ should be followed by /, ? or &
> line 3:
> E10: \ should be followed by /, ? or &
>
> Not sure why. I ignore them by pressing Enter.
mode. 'cpo' should be set to the Vim value.
> This in `~/.vim/plugged/vim-cmdline/plugin/cmdline.vim`:
>
> vim9
> au CmdlineLeave : timer_start(0, () => execute('au SafeState * ++once #'))
remove this?
I don't see another hint, thus I hope you can find out more.
--
hundred-and-one symptoms of being an internet addict:
214. Your MCI "Circle of Friends" are all Hayes-compatible.
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
lacygoill
I can't get a valgrind log, because valgrind doesn't run on my Ubuntu 16.04; I need to run it in a VM; but it's not easy to reproduce on a different machine; I've tried, it doesn't crash, and the log is useless (it doesn't contain any error).
For some reason, on my Ubuntu 16.04, when this block is sourced:
au CmdlineLeave : if 0
\ | eval 0
\ | endif
'cpo' has this value:
aAbBcCdDeEfFgHiIjJkKlLmMnoOpPqrRsStuvwWxXyZ$!%*-+<>;
Notice that it contains the C flag.
But on my Ubuntu 18.04 in a VM, right after the set rtp... line is sourced, 'cpo' is reset with this value:
aABceFs
Notice that this time, the C flag is absent. I don't know why the difference, but that explains why I couldn't reproduce the issue with valgrind. So, I've added this line in the buggy vimrc, right after set rtp...:
set cpo=aAbBcCdDeEfFgHiIjJkKlLmMnoOpPqrRsStuvwWxXyZ$!%*-+<>;
Now, I have a useful valgrind log:
==8879== Memcheck, a memory error detector
==8879== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==8879== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==8879== Command: ./src/vim --cmd filetype\ on
==8879== Parent PID: 15300
==8879==
==8879== Conditional jump or move depends on uninitialised value(s)
==8879== at 0x1AF8EA: leave_block (ex_eval.c:935)
==8879== by 0x1B17BF: rewind_conditionals (ex_eval.c:2392)
==8879== by 0x1AAE90: do_cmdline (ex_docmd.c:1244)
==8879== by 0x140391: apply_autocmds_group (autocmd.c:2109)
==8879== by 0x140F63: apply_autocmds (autocmd.c:1621)
==8879== by 0x1B5E09: trigger_cmd_autocmd (ex_getln.c:73)
==8879== by 0x1B5E09: getcmdline_int (ex_getln.c:2427)
==8879== by 0x1AACC8: do_cmdline (ex_docmd.c:886)
==8879== by 0x222079: nv_colon (normal.c:3381)
==8879== by 0x227911: normal_cmd (normal.c:1098)
==8879== by 0x34AD06: main_loop (main.c:1475)
==8879== by 0x34BD6A: vim_main2 (main.c:865)
==8879== by 0x13A8CC: main (main.c:412)
==8879== Uninitialised value was created by a stack allocation
==8879== at 0x1A9E63: do_cmdline (ex_docmd.c:632)
==8879==
==8879==
==8879== HEAP SUMMARY:
==8879== in use at exit: 153,436 bytes in 1,202 blocks
==8879== total heap usage: 4,147 allocs, 2,945 frees, 1,449,877 bytes allocated
==8879==
==8879== LEAK SUMMARY:
==8879== definitely lost: 0 bytes in 0 blocks
==8879== indirectly lost: 0 bytes in 0 blocks
==8879== possibly lost: 0 bytes in 0 blocks
==8879== still reachable: 153,436 bytes in 1,202 blocks
==8879== suppressed: 0 bytes in 0 blocks
==8879== Reachable blocks (those to which a pointer was found) are not shown.
==8879== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==8879==
==8879== For counts of detected and suppressed errors, rerun with: -v
==8879== ERROR SUMMARY: 2 errors from 1 contexts (suppressed: 0 from 0)
Notice that, just like the asan log, it mentions the leave_block() function in ex_eval.c:
https://github.com/vim/vim/blob/c05fe07529f10fc32a2086cb44a708520c8d14ce/src/ex_eval.c#L926
Also, it reports 2 errors from 1 context. Apparently, there is an "uninitialised value created by a stack allocation", somewhere in the do_cmdline() function in ex_docmd.c:
https://github.com/vim/vim/blob/c05fe07529f10fc32a2086cb44a708520c8d14ce/src/ex_docmd.c#L627
If I run valgrind with the -v flag, the report contains much more info. Let me know if this would be useful.
lacygoill
And here is a minimal working example:
vim -Nu NONE -S <(cat <<'EOF'
set cpo+=C
au CmdlineLeave : if 0
\ | eval 0
\ | endif
def Func()
au CmdlineLeave : timer_start(0, () => execute('au SafeState * ++once #'))
enddef
call Func()
EOF
)
lacygoill
And here is a minimal working example:
This one doesn't make Vim crash, but it's enough for valgrind to report errors.
lacygoill
Sorry, I forgot that we had to enter and leave the command-line. Updated MWE:
vim -Nu NONE -S <(cat <<'EOF'
set cpo+=C
au CmdlineLeave : if 0
\ | eval 0
\ | endif
def Func()
au CmdlineLeave : timer_start(0, () => execute('au SafeState * ++once #'))
enddef
call Func()
call feedkeys(':', 'nxt')
EOF
)
lacygoill
Also, if we remove call Func(), the report suggests that there might be a memory leak. This time, there are 3 errors from 3 contexts:
==9208== Memcheck, a memory error detector
==9208== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==9208== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==9208== Command: ./src/vim -Nu NONE -S /dev/fd/63
==9208== Parent PID: 15300
==9208==
==9208==
==9208== HEAP SUMMARY:
==9208== in use at exit: 147,996 bytes in 1,137 blocks
==9208== total heap usage: 3,279 allocs, 2,142 frees, 885,994 bytes allocated
==9208==
==9208== 24 bytes in 1 blocks are possibly lost in loss record 52 of 146
==9208== at 0x4C2FA3F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==9208== by 0x4C31D84: realloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==9208== by 0x213D78: ga_grow_inner (misc2.c:2114)
==9208== by 0x2F6BAC: define_function (userfunc.c:3630)
==9208== by 0x1AD07B: do_one_cmd (ex_docmd.c:2588)
==9208== by 0x1AD07B: do_cmdline (ex_docmd.c:1003)
==9208== by 0x28D93A: do_source (scriptfile.c:1401)
==9208== by 0x28E732: cmd_source (scriptfile.c:971)
==9208== by 0x1AD07B: do_one_cmd (ex_docmd.c:2588)
==9208== by 0x1AD07B: do_cmdline (ex_docmd.c:1003)
==9208== by 0x34BF1F: exe_commands (main.c:3056)
==9208== by 0x34BF1F: vim_main2 (main.c:760)
==9208== by 0x13A8CC: main (main.c:412)
==9208==
==9208== 83 bytes in 1 blocks are possibly lost in loss record 76 of 146
==9208== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==9208== by 0x2124B0: lalloc (misc2.c:925)
==9208== by 0x212F3E: alloc (misc2.c:828)
==9208== by 0x212F3E: vim_strsave (misc2.c:1281)
==9208== by 0x2F6BBC: define_function (userfunc.c:3636)
==9208== by 0x1AD07B: do_one_cmd (ex_docmd.c:2588)
==9208== by 0x1AD07B: do_cmdline (ex_docmd.c:1003)
==9208== by 0x28D93A: do_source (scriptfile.c:1401)
==9208== by 0x28E732: cmd_source (scriptfile.c:971)
==9208== by 0x1AD07B: do_one_cmd (ex_docmd.c:2588)
==9208== by 0x1AD07B: do_cmdline (ex_docmd.c:1003)
==9208== by 0x34BF1F: exe_commands (main.c:3056)
==9208== by 0x34BF1F: vim_main2 (main.c:760)
==9208== by 0x13A8CC: main (main.c:412)
==9208==
==9208== 389 bytes in 1 blocks are possibly lost in loss record 106 of 146
==9208== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==9208== by 0x2124B0: lalloc (misc2.c:925)
==9208== by 0x212E1D: alloc_clear (misc2.c:852)
==9208== by 0x2F7C45: define_function (userfunc.c:3790)
==9208== by 0x1AD07B: do_one_cmd (ex_docmd.c:2588)
==9208== by 0x1AD07B: do_cmdline (ex_docmd.c:1003)
==9208== by 0x28D93A: do_source (scriptfile.c:1401)
==9208== by 0x28E732: cmd_source (scriptfile.c:971)
==9208== by 0x1AD07B: do_one_cmd (ex_docmd.c:2588)
==9208== by 0x1AD07B: do_cmdline (ex_docmd.c:1003)
==9208== by 0x34BF1F: exe_commands (main.c:3056)
==9208== by 0x34BF1F: vim_main2 (main.c:760)
==9208== by 0x13A8CC: main (main.c:412)
==9208==
==9208== LEAK SUMMARY:
==9208== definitely lost: 0 bytes in 0 blocks
==9208== indirectly lost: 0 bytes in 0 blocks
==9208== possibly lost: 496 bytes in 3 blocks
==9208== still reachable: 147,500 bytes in 1,134 blocks
==9208== suppressed: 0 bytes in 0 blocks
==9208== Reachable blocks (those to which a pointer was found) are not shown.
==9208== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==9208==
==9208== For counts of detected and suppressed errors, rerun with: -v
==9208== ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 0 from 0)
Bram Moolenaar
> > I can't get a valgrind log, because valgrind doesn't run on my
> > Ubuntu 16.04; I need to run it in a VM; but it's not easy to
> > reproduce on a different machine; I've tried, it doesn't crash, and
>
> For some reason, on my Ubuntu 16.04, when this block is sourced:
>
> \ | eval 0
> \ | endif
>
clearing it, just above this line:
cstack.cs_idx = -1;
With CLEAR_FIELD. Might as well remove some lines setting fields to
zero then:
#ifdef FEAT_EVAL
CLEAR_FIELD(cstack);
cstack.cs_idx = -1;
ga_init2(&lines_ga, (int)sizeof(wcmd_T), 10);
It might not actually help for the crash, but the warning for
uninitialised value should be gone. Perhaps it crashes more reliably?
You can also, instead of adding CLEAR_FIELD(), clear more of the
individual fields. That should pinpoint what field in cstack the
warning is for. I guess it's either cs_flags[] or cs_script_var_len[].
--
You can tune a file system, but you can't tuna fish
-- man tunefs
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
lacygoill
This suggests that something in "cstack" was not initialized. Try
clearing it, just above this line:cstack.cs_idx = -1;
With CLEAR_FIELD. Might as well remove some lines setting fields to
zero then:#ifdef FEAT_EVAL
CLEAR_FIELD(cstack);
cstack.cs_idx = -1;
ga_init2(&lines_ga, (int)sizeof(wcmd_T), 10);It might not actually help for the crash, but the warning for
uninitialised value should be gone. Perhaps it crashes more reliably?
Indeed, after applying this patch:
diff --git a/src/ex_docmd.c b/src/ex_docmd.c index 3dec65e88..8b9db6812 100644 --- a/src/ex_docmd.c +++ b/src/ex_docmd.c @@ -696,11 +696,8 @@ do_cmdline( ++call_depth; #ifdef FEAT_EVAL + CLEAR_FIELD(cstack); cstack.cs_idx = -1; - cstack.cs_looplevel = 0; - cstack.cs_trylevel = 0; - cstack.cs_emsg_silent_list = NULL; - cstack.cs_lflags = 0; ga_init2(&lines_ga, (int)sizeof(wcmd_T), 10); real_cookie = getline_cookie(fgetline, cookie);
Valgrind doesn't report any error anymore. However, it doesn't make Vim crash more frequently. On the contrary, it doesn't crash anymore.
You can also, instead of adding CLEAR_FIELD(), clear more of the individual fields.
I don't know what to write to clear an individual field. I'll try to make more tests later.
Bram Moolenaar
> I don't know what to write to clear an individual field. I'll try to
> make more tests later.
CLEAR_FIELD(cstack.cs_flags);
CLEAR_FIELD(cstack.cs_pending);
CLEAR_FIELD(cstack.cs_pend.csp_rv);
CLEAR_FIELD(cstack.cs_forinfo);
CLEAR_FIELD(cstack.cs_line);
CLEAR_FIELD(cstack.cs_block_id);
CLEAR_FIELD(cstack.cs_script_var_len); // most likely culprit
And remove the line:
CLEAR_FIELD(cstack)
--
Eagles may soar, but weasels don't get sucked into jet engines.
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
—
lacygoill
I've tested each field, and I can confirm that the problematic one was indeed cs_script_var_len, because only this patch prevents Vim from crashing:
diff --git a/src/ex_docmd.c b/src/ex_docmd.c index 3dec65e88..599992785 100644 --- a/src/ex_docmd.c +++ b/src/ex_docmd.c @@ -696,6 +696,7 @@ do_cmdline( ++call_depth; #ifdef FEAT_EVAL + CLEAR_FIELD(cstack.cs_script_var_len); cstack.cs_idx = -1; cstack.cs_looplevel = 0; cstack.cs_trylevel = 0;
As for a test, it could be based on this command:
vim -Nu NONE -S <(cat <<'EOF'
set cpo+=C
au CmdlineLeave : if 0
\ | eval 0
\ | endif
def Func()
au CmdlineLeave : timer_start(0, () => execute('au SafeState * ++once #'))
enddef
call Func()
call feedkeys(':', 'nxt')
EOF
)
On 8.2.2409, this command causes valgrind to report errors, and sometimes it makes Vim crash. For some reason, the second CmdlineLeave autocmd needs to be installed from the Vim9 context for a crash to be possible, hence the :def function. But it's not necessary for valgrind to report errors.
Bram Moolenaar
> I've tested each field, and I can confirm that the problematic one was indeed `cs_script_var_len`, because only this patch prevents Vim from crashing:
> ```diff
> diff --git a/src/ex_docmd.c b/src/ex_docmd.c
> index 3dec65e88..599992785 100644
> --- a/src/ex_docmd.c
> +++ b/src/ex_docmd.c
> @@ -696,6 +696,7 @@ do_cmdline(
> ++call_depth;
>
> #ifdef FEAT_EVAL
> + CLEAR_FIELD(cstack.cs_script_var_len);
> cstack.cs_idx = -1;
> cstack.cs_looplevel = 0;
> cstack.cs_trylevel = 0;
> ```
> As for a test, it could be based on this command:
>
> vim -Nu NONE -S <(cat <<'EOF'
> set cpo+=C
> \ | eval 0
> \ | endif
> au CmdlineLeave : timer_start(0, () => execute('au SafeState * ++once #'))
> call Func()
> call feedkeys(':', 'nxt')
> EOF
> )
>
> On 8.2.2409, this command causes valgrind to report errors, and
> `CmdlineLeave` autocmd needs to be installed from the Vim9 context for
> a crash to be possible, hence the `:def` function. But it's not
> necessary for valgrind to report errors.
I wonder if there is still a hidden problem. We no longer get a warning
for an uninitialized variable, but it might still be using a value that
wasn't set properly.
Please try, instead of adding CLEAR_FIELD(), to change the enter_block()
function:
static void
enter_block(cstack_T *cstack)
{
++cstack->cs_idx;
if (in_vim9script())
{
scriptitem_T *si = SCRIPT_ITEM(current_sctx.sc_sid);
cstack->cs_script_var_len[cstack->cs_idx] = si->sn_var_vals.ga_len;
cstack->cs_block_id[cstack->cs_idx] = ++si->sn_last_block_id;
si->sn_current_block_id = si->sn_last_block_id;
}
else
{
cstack->cs_script_var_len[cstack->cs_idx] = 0;
cstack->cs_block_id[cstack->cs_idx] = 0;
}
}
Does this also avoid the valgrind error? Then it might be that
in_vim9script() does not always return the right value for the context.
--
hundred-and-one symptoms of being an internet addict:
for directory assistance.
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
—
lacygoill
Does this also avoid the valgrind error?
Yes, after applying this patch on Vim 8.2.2409, valgrind no longer reports any error:
diff --git a/src/ex_eval.c b/src/ex_eval.c index 94338fcdb..484569cf0 100644 --- a/src/ex_eval.c +++ b/src/ex_eval.c @@ -920,6 +920,11 @@ enter_block(cstack_T *cstack) cstack->cs_block_id[cstack->cs_idx] = ++si->sn_last_block_id; si->sn_current_block_id = si->sn_last_block_id; } + else + { + cstack->cs_script_var_len[cstack->cs_idx] = 0; + cstack->cs_block_id[cstack->cs_idx] = 0; + } } static void
lacygoill
Closed #7733.