[vim/vim] Motif gui_mch_dialog: first button XmStringCreate failure leaves butcount==0 then indexes buttons[-1] (Issue #20820)

1 view

Skip to first unread message

Shixin Tan

unread,

Jul 23, 2026, 5:43:29 AM (yesterday) Jul 23

to vim/vim, Subscribed

tsx543 created an issue (vim/vim#20820)

Steps to reproduce

Build Vim with Motif GUI and dialog support (FEAT_GUI_MOTIF / FEAT_GUI_DIALOG), e.g. a Motif-enabled gvim.
Open a Motif dialog that has at least one button (any confirm/message dialog that goes through gui_mch_dialog() in src/gui_motif.c).
Force XmStringCreate() to fail for the first button label (out-of-memory / Motif string allocation failure) while later setup still continues.
Observe that the button-creation loop breaks with butcount == 0, then the default-button clamp still runs and uses buttons[dfltbutton - 1].
Relevant code in current master (src/gui_motif.c):

for (butcount = 0; *p; ++butcount)
{
    ...
    label = XmStringCreate(_((char *)p), STRING_TAG);
    if (label == NULL)
        break;
    buttons[butcount] = XtVaCreateManagedWidget(...);
    ...
}
if (dfltbutton < 1)
    dfltbutton = 1;
if (dfltbutton > butcount)
    dfltbutton = butcount;          /* becomes 0 when butcount==0 */
XtVaSetValues(dialogform,
        XmNdefaultButton, buttons[dfltbutton - 1], NULL);  /* buttons[-1] */

There is no butcount == 0 guard before the buttons[dfltbutton - 1] access.

Expected behaviour

If no buttons were successfully created (butcount == 0), gui_mch_dialog() should abort cleanly (destroy the dialog / return an error) and must not index buttons[-1] or pass an invalid widget pointer to Motif/XtVaSetValues.

Version of Vim

9.2.0838 (git master @ 27cfd1c)

Environment

Operating system: Linux / Unix with Motif GUI
Terminal: n/a (GUI Motif build)
Value of $TERM: n/a
Shell: n/a

Logs and stack traces

—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications, keep track of coding agent tasks and review pull requests on the go with GitHub Mobile for iOS and Android. Download it today!
You are receiving this because you are subscribed to this thread.

h_east

unread,

Jul 23, 2026, 7:42:18 AM (yesterday) Jul 23

to vim/vim, Subscribed

h-east left a comment (vim/vim#20820)

The report is technically correct: if XmStringCreate() fails for the first
button, the loop breaks with butcount == 0, dfltbutton is clamped to 0, and
buttons[dfltbutton - 1] reads buttons[-1].

Note this only occurs on allocation failure and only with the (legacy) Motif
GUI, so there is no script-reproducible case and no regression test is possible.
The guard below matches the existing error path a few lines down (the
XmStringCreateLtoR NULL check): free buttons and return -1 rather than
destroying the dialog.

--- a/src/gui_motif.c
+++ b/src/gui_motif.c
@@ -2777,6 +2777,12 @@ gui_mch_dialog(
 	    NULL);
     }
 
+    if (butcount == 0)
+    {
+	// No button could be created (XmStringCreate() failed: out of memory).
+	vim_free(buttons);
+	return -1;
+    }

     if (dfltbutton < 1)
 	dfltbutton = 1;
     if (dfltbutton > butcount)

Please do not post AI-generated (LLM) content as-is.
You must verify it yourself before posting.