test_vim9_expr.vim crashes Vim on MacOS with bad instruction

[Yegappan Lakshmanan]

Hi,

When I run the test_vim9_expr.vim test on MacOS with Vim 8.2.2029),

I see that VIm crashes with a bad instruction exception.

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)

I see this crash only when building Vim with debug symbols (-g -O0).

If I build Vim without the debug symbols, I don't see this problem. 

I also don't see this crash with the ASAN and UBSAN builds.

Is this a bug in the compiler? I am using the following Clang compiler on Catalina:

Apple clang version 12.0.0 (clang-1200.0.32.21)
Target: x86_64-apple-darwin19.6.0

The complete traceback is below:

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
  * frame #0: 0x00007fff6af56fca libsystem_c.dylib`__chk_fail_overflow.cold.1 + 16
    frame #1: 0x00007fff6af54214 libsystem_c.dylib`__chk_fail_overflow + 9
    frame #2: 0x00007fff6af543f0 libsystem_c.dylib`__strcpy_chk + 83
    frame #3: 0x000000010029287b vim`add_vim9_script_var(di=0x0000000100606550, tv=0x00007ffeefbf9178, type=0x0000000000000000) at vim9script.c:604:2
    frame #4: 0x00000001000826a7 vim`set_var_const(name="FuncRef", type=0x0000000000000000, tv_arg=0x00007ffeefbf9178, copy=0, flags=0) at evalvars.c:3203:6
    frame #5: 0x00000001000606c1 vim`set_var_lval(lp=0x00007ffeefbf8f28, endp="", rettv=0x00007ffeefbf9178, copy=0, flags=0, op="=") at eval.c:1339:6
    frame #6: 0x000000010007e93b vim`ex_let_one(arg="FuncRef", tv=0x00007ffeefbf9178, copy=0, flags=0, endchars="=", op="=") at evalvars.c:1473:3
    frame #7: 0x000000010007ddc5 vim`ex_let_vars(arg_start="FuncRef", tv=0x00007ffeefbf9178, copy=0, semicolon=0, var_count=0, flags=0, op="=") at evalvars.c:922:6
    frame #8: 0x000000010007d52e vim`ex_let(eap=0x00007ffeefbf9368) at evalvars.c:888:12
    frame #9: 0x000000010007cdad vim`ex_var(eap=0x00007ffeefbf9368) at evalvars.c:699:5
    frame #10: 0x0000000100097602 vim`do_one_cmd(cmdlinep=0x00007ffeefbf9610, sourcing=1, cstack=0x00007ffeefbf9648, fgetline=(vim`getsourceline at scriptfile.c:1746), cookie=0x00007ffeefbf9d60) at ex_docmd.c:2576:2
    frame #11: 0x000000010009451b vim`do_cmdline(cmdline="vim9script", fgetline=(vim`getsourceline at scriptfile.c:1746), cookie=0x00007ffeefbf9d60, flags=7) at ex_docmd.c:993:17
    frame #12: 0x00000001001d6ac5 vim`do_source(fname="Xdef", check_other=0, is_vimrc=0, ret_sid=0x0000000000000000) at scriptfile.c:1436:5
    frame #13: 0x00000001001d5e7f vim`cmd_source(fname="Xdef", eap=0x00007ffeefbf9fc8) at scriptfile.c:971:14
    frame #14: 0x00000001001d5d9c vim`ex_source(eap=0x00007ffeefbf9fc8) at scriptfile.c:997:2
    frame #15: 0x0000000100097602 vim`do_one_cmd(cmdlinep=0x00007ffeefbfa270, sourcing=1, cstack=0x00007ffeefbfa2a8, fgetline=0x0000000000000000, cookie=0x0000000000000000) at ex_docmd.c:2576:2
    frame #16: 0x000000010009451b vim`do_cmdline(cmdline="  so Xdef", fgetline=0x0000000000000000, cookie=0x0000000000000000, flags=11) at ex_docmd.c:993:17
    frame #17: 0x00000001000953e4 vim`do_cmdline_cmd(cmd="  so Xdef") at ex_docmd.c:594:12
    frame #18: 0x0000000100287bf3 vim`call_def_function(ufunc=0x000000010182f760, argc_arg=0, argv=0x00007ffeefbfbbe0, partial=0x0000000000000000, rettv=0x00007ffeefbfbe30) at vim9execute.c:1090:7
    frame #19: 0x000000010026c0c2 vim`call_user_func(fp=0x000000010182f760, argcount=0, argvars=0x00007ffeefbfbbe0, rettv=0x00007ffeefbfbe30, funcexe=0x00007ffeefbfbd78, selfdict=0x0000000000000000) at userfunc.c:1480:2
    frame #20: 0x000000010026be44 vim`call_user_func_check(fp=0x000000010182f760, argcount=0, argvars=0x00007ffeefbfbbe0, rettv=0x00007ffeefbfbe30, funcexe=0x00007ffeefbfbd78, selfdict=0x0000000000000000) at userfunc.c:1871:2
    frame #21: 0x000000010026ae66 vim`call_func(funcname="Test_epxr7_funcref", len=-1, rettv=0x00007ffeefbfbe30, argcount_in=0, argvars_in=0x00007ffeefbfbbe0, funcexe=0x00007ffeefbfbd78) at userfunc.c:2329:11
    frame #22: 0x000000010026a670 vim`get_func_tv(name="Test_epxr7_funcref", len=-1, rettv=0x00007ffeefbfbe30, arg=0x00007ffeefbfbe60, evalarg=0x00007ffeefbfbdb8, funcexe=0x00007ffeefbfbd78) at userfunc.c:779:8
    frame #23: 0x0000000100272c48 vim`ex_call(eap=0x00007ffeefbfc018) at userfunc.c:4218:6
    frame #24: 0x0000000100097602 vim`do_one_cmd(cmdlinep=0x00007ffeefbfc2c0, sourcing=1, cstack=0x00007ffeefbfc2f8, fgetline=(vim`get_func_line at userfunc.c:4401), cookie=0x0000000102824400) at ex_docmd.c:2576:2
    frame #25: 0x000000010009451b vim`do_cmdline(cmdline="call Test_epxr7_funcref()", fgetline=(vim`get_func_line at userfunc.c:4401), cookie=0x0000000102824400, flags=3) at ex_docmd.c:993:17
    frame #26: 0x0000000100067245 vim`ex_execute(eap=0x00007ffeefbfcb98) at eval.c:5935:6
    frame #27: 0x0000000100097602 vim`do_one_cmd(cmdlinep=0x00007ffeefbfce40, sourcing=1, cstack=0x00007ffeefbfce78, fgetline=(vim`get_func_line at userfunc.c:4401), cookie=0x0000000102824400) at ex_docmd.c:2576:2
    frame #28: 0x000000010009451b vim`do_cmdline(cmdline=0x0000000000000000, fgetline=(vim`get_func_line at userfunc.c:4401), cookie=0x0000000102824400, flags=7) at ex_docmd.c:993:17
    frame #29: 0x000000010026ce1a vim`call_user_func(fp=0x0000000101806e50, argcount=1, argvars=0x00007ffeefbfde60, rettv=0x00007ffeefbfe0b0, funcexe=0x00007ffeefbfdff8, selfdict=0x0000000000000000) at userfunc.c:1731:2
    frame #30: 0x000000010026be44 vim`call_user_func_check(fp=0x0000000101806e50, argcount=1, argvars=0x00007ffeefbfde60, rettv=0x00007ffeefbfe0b0, funcexe=0x00007ffeefbfdff8, selfdict=0x0000000000000000) at userfunc.c:1871:2
    frame #31: 0x000000010026ae66 vim`call_func(funcname="RunTheTest", len=-1, rettv=0x00007ffeefbfe0b0, argcount_in=1, argvars_in=0x00007ffeefbfde60, funcexe=0x00007ffeefbfdff8) at userfunc.c:2329:11
    frame #32: 0x000000010026a670 vim`get_func_tv(name="RunTheTest", len=-1, rettv=0x00007ffeefbfe0b0, arg=0x00007ffeefbfe0e0, evalarg=0x00007ffeefbfe038, funcexe=0x00007ffeefbfdff8) at userfunc.c:779:8
    frame #33: 0x0000000100272c48 vim`ex_call(eap=0x00007ffeefbfe298) at userfunc.c:4218:6
    frame #34: 0x0000000100097602 vim`do_one_cmd(cmdlinep=0x00007ffeefbfe540, sourcing=1, cstack=0x00007ffeefbfe578, fgetline=(vim`get_loop_line at ex_docmd.c:1442), cookie=0x00007ffeefbfe468) at ex_docmd.c:2576:2
    frame #35: 0x000000010009451b vim`do_cmdline(cmdline="\" This script is sourced while editing the .vim file with the tests.", fgetline=(vim`getsourceline at scriptfile.c:1746), cookie=0x00007ffeefbfec90, flags=7) at ex_docmd.c:993:17
    frame #36: 0x00000001001d6ac5 vim`do_source(fname="runtest.vim", check_other=0, is_vimrc=0, ret_sid=0x0000000000000000) at scriptfile.c:1436:5
    frame #37: 0x00000001001d5e7f vim`cmd_source(fname="runtest.vim", eap=0x00007ffeefbfeef8) at scriptfile.c:971:14
    frame #38: 0x00000001001d5d9c vim`ex_source(eap=0x00007ffeefbfeef8) at scriptfile.c:997:2
    frame #39: 0x0000000100097602 vim`do_one_cmd(cmdlinep=0x00007ffeefbff1a0, sourcing=1, cstack=0x00007ffeefbff1d8, fgetline=0x0000000000000000, cookie=0x0000000000000000) at ex_docmd.c:2576:2
    frame #40: 0x000000010009451b vim`do_cmdline(cmdline="so runtest.vim", fgetline=0x0000000000000000, cookie=0x0000000000000000, flags=11) at ex_docmd.c:993:17
    frame #41: 0x00000001000953e4 vim`do_cmdline_cmd(cmd="so runtest.vim") at ex_docmd.c:594:12
    frame #42: 0x00000001002eb621 vim`exe_commands(parmp=0x0000000100353dc0) at main.c:3046:2
    frame #43: 0x00000001002ea672 vim`vim_main2 at main.c:763:2
    frame #44: 0x00000001002e7f73 vim`main(argc=11, argv=0x00007ffeefbff8e8) at main.c:412:12
    frame #45: 0x00007fff6ae87cc9 libdyld.dylib`start + 1
    frame #46: 0x00007fff6ae87cc9 libdyld.dylib`start + 1

- Yegappan

[Bram Moolenaar]

Yegappan wrote:

> When I run the test_vim9_expr.vim test on MacOS with Vim 8.2.2029),
> I see that VIm crashes with a bad instruction exception.
>
> * thread #1, queue = 'com.apple.main-thread', stop reason =
> EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
>
> I see this crash only when building Vim with debug symbols (-g -O0).
> If I build Vim without the debug symbols, I don't see this problem.
> I also don't see this crash with the ASAN and UBSAN builds.
>
> Is this a bug in the compiler? I am using the following Clang compiler on
> Catalina:
>
> Apple clang version 12.0.0 (clang-1200.0.32.21)
> Target: x86_64-apple-darwin19.6.0
>
> The complete traceback is below:
>
> * thread #1, queue = 'com.apple.main-thread', stop reason =
> EXC_BAD_INSTRUCTION (code=EXC_I386_INVOP, subcode=0x0)
> * frame #0: 0x00007fff6af56fca
> libsystem_c.dylib`__chk_fail_overflow.cold.1 + 16
> frame #1: 0x00007fff6af54214 libsystem_c.dylib`__chk_fail_overflow + 9
> frame #2: 0x00007fff6af543f0 libsystem_c.dylib`__strcpy_chk + 83
> frame #3: 0x000000010029287b
> vim`add_vim9_script_var(di=0x0000000100606550, tv=0x00007ffeefbf9178,
> type=0x0000000000000000) at vim9script.c:604:2

This is:

STRCPY(&newsav->sav_key, di->di_key);

And "sav_key" is a one byte array, that is actually longer. This looks
like the FORTIFY_SOURCE problem. Configure only adds this for GCC 4 and
higher, perhaps we also need it for clang?

You could try adding this to the clang command:
-U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1


--
Microsoft's definition of a boolean: TRUE, FALSE, MAYBE
"Embrace and extend"...?

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Yegappan Lakshmanan]

Hi Bram,

With FORTIFY_SOURCE, I am not able to reproduce the problem.

Regards,

Yegappan

[Bram Moolenaar]

Good to know that's the cause of the problem.

Can you please try to adjust the configure script for this?
Currently that part is only used for gcc. I wonder since what version of
clang these arguments are supported.
This is around line 4437 in src/configure.ac. You'll need to run "make
autoconf" to try it out.

--
ARTHUR: Who are you?
TALL KNIGHT: We are the Knights Who Say "Ni"!
BEDEVERE: No! Not the Knights Who Say "Ni"!
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD