gvim :sh ^U broken by patch 8.2.0851

[Gary Johnson]

I seldom use gvim on Linux and even less often use :sh from gvim, so
I didn't notice this until recently when projects and my workflow
changed.

When :sh is used in gvim to launch a shell, the backspace and ^U no
longer work. Instead of backspacing or clearing the line, each just
puts gibberish in the line. I discovered that each was broken by
a different commit, so I'm reporting them separately.

Steps to reproduce

1. $ vim -g -N -u NONE -i NONE
2. :sh
3. Type a bunch of printing characters, it doesn't seem to matter
what. For example:

$ hello

4. Type ^U (Ctrl-U).
5. The line changes to something I can't reproduce here. It's
something like this:

$ hello |00| |00| u
|9B| |9B|

where the 'u' has an umlaut and each box occupies two spaces on
the one line. That line is immediately followed by these two
(so a newline must have been inserted, too):

Uhello\udc9b\udcfc: command not found
$ U

Hitting Enter at this point results in this:

U: command not found

Expected behavior

The 'hello' should be erased and the cursor moved to the start
of the now-empty line.

Version of Vim

9.0.651

Environment

Operating system: Ubuntu 20.04.5
Terminal: Not applicable, but gvim launched from an xterm
Value of $TERM:
In the original xterm: xterm-256color
In the :sh shell: dumb
Shell: bash
Desktop: xfce4
Behavior is the same in GNOME on Ubuntu 22.04.

Last output from 'git bisect':

$ git bisect good
f4ae6b245a54f11dd967d06b80f30e5abf55fb82 is the first bad commit
commit f4ae6b245a54f11dd967d06b80f30e5abf55fb82
Author: Bram Moolenaar <Br...@vim.org>
Date: Sat May 30 19:52:46 2020 +0200

patch 8.2.0851: can't distinguish <M-a> from accented "a" in the GUI

Problem: Can't distinguish <M-a> from accented "a" in the GUI.
Solution: Use another way to make mapping <C-bslash> work. (closes #6163)

src/getchar.c | 20 +++++++++++++++-----
src/gui.c | 4 ++--
src/gui_gtk_x11.c | 44 +++++++++-----------------------------------
src/version.c | 2 ++
4 files changed, 28 insertions(+), 42 deletions(-)

I should have mentioned this in my report of the backspace bug, but
at every 'git bisect' step, vim was build with these commands in an
environment that should not have affected the build (i.e., no
vim-affecting environment variables):

$ make distclean
$ make -j6

Regards,
Gary

[Gary Johnson]

So I did a little bit of poking at this with git and gdb and
discovered that at patch v8.2.0851, a block of code was moved from
gui_gtk_x11.c:keyval_to_string() to
getchar.c:merge_modifyOtherKeys(). In a working version of the
code, e.g. v8.2.0850, that block coverts the key code for ^U to the
value of ^U. In a non-working version of the code, e.g. v8.2.0851,
where that block has been moved, that block is not executed when the
user types ^U in the shell, so the conversion is not done and shell
receives something else.

I don't understand the overall flow of keycode processing well
enough to know where best to put that block, but it shouldn't have
been moved from where it was. Maybe put it in both places, or
create a small function to encapsulate that block and call it in
both places?

Regards,
Gary

[Bram Moolenaar]

There are several paths that typed keys follow, and the current way is
good for when Vim consumes the keys. However, in this case the keys are
going to the shell. There is a special loop in mch_call_shell_fork()
that fetches input and writes it to the external process. The external
process doesn't understand Vim's special codes, they need to be
converted to regular typed text. This is only partly done by
term_replace_bs_del_keycode(). Handling modifier codes there should
make it work.

I still wonder why it took so long for someone to uncover this problem.
And what other things might still be wrong...

--
hundred-and-one symptoms of being an internet addict:
269. You wonder how you can make your dustbin produce Sesame Street's
Oscar's the Garbage Monster song when you empty it.

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Gary Johnson]

On 2022-10-04, Bram Moolenaar wrote:
> Gary Johnson wrote:
>
> > > I seldom use gvim on Linux and even less often use :sh from gvim, so
> > > I didn't notice this until recently when projects and my workflow
> > > changed.
> > >
> > > When :sh is used in gvim to launch a shell, the backspace and ^U no
> > > longer work. Instead of backspacing or clearing the line, each just
> > > puts gibberish in the line. I discovered that each was broken by
> > > a different commit, so I'm reporting them separately.

[...]

> I still wonder why it took so long for someone to uncover this problem.
> And what other things might still be wrong...

Well, the arrow keys are kinda funky, too. Executing :sh and
pressing the up-arrow, then Enter produces this, where the boxes are
on one line and two characters wide.

$ |00| |00| |00| k
|9B| |9B| |9B|
\udc9bku: command not found

I tried looking at the arrow-key keycodes in gvim, but whether
I prefixed the key with ^V or ^K, all I got was the key name, e.g.,
<Up>.

The other arrow keys behave similarly. The characters on the
command line look the same, but the last letter of the command that
bash sees are different.

Key Command
----------- --------
up-arrow \udc9bku
down-arrow \udc9bkd
left-arrow \udc9bkl
right-arrow \udc9bkr

The shifted arrow keys give similar results but also appear
differently on the command line.

Regards,
Gary

[Bram Moolenaar]

I believe it has always been like this. The $TERM in the shell is set
to "dumb", which means it doesn't support any special keys.

There have been ideas to simulate a more clever terminal, perhaps vt52.
But using a terminal window is a much better alternative.

--
PRINCE: He's come to rescue me, father.
LAUNCELOT: (embarrassed) Well, let's not jump to conclusions ...
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

[Gary Johnson]

That sounds reasonable. I just tried it in the two oldest gvims
I have lying around and they behaved the same. I seldom use :sh in
gvim, and then just for quick checks, so a dumb terminal is fine.

Regards,
Gary