Patch 9.0.0980 causes tiny font to be used after ":set bg&"

[Gary Johnson]

After updating to the latest Vim, 9.0.0984, and starting Vim in
a terminal without a file specified, the command line font was
extremely tiny. (I'd estimate the font size to be 4 points.) If
a file was opened from the command line, its font in the buffer was
also tiny. Subsequent commands typed in the command line were of
the expected font. The font in the buffer remained tiny until ^L
was typed.

Bisecting my vimrc and my usual color scheme plugin revealed the
command causing the tiny font to be

:set bg&

and bisecting the git commits showed the bad commit to be
733a69b29f0b0c3d2ddca463a41bdd912379bc5e, tag v9.0.0980.

Steps to reproduce

1. vim -N -u NONE -i NONE --cmd 'set bg&'
2. Type a colon (:) and note that it is in a very tiny font.
3. Continue typing the ex command to edit some file, e.g.,
":e feature.h". Note that the file is displayed in the buffer
in the same tiny font.

Expected behavior

I expect Vim to use the same 11-pt font it normally uses, as set in
the terminal's preferences.

Version of Vim

9.0.0980 and 9.0.0984

Environment

Operating system: Cygwin version 3.3.6-1 (latest as of 2022-12-01)
on Windows 10 Pro version 22H2
Terminal: mintty version 3.6.2-1 (also latest as of 2022-12-01)
Value of $TERM: xterm-256color
Shell: bash version 4.4.12-3

Additional Context

During the git bisect, each version of Vim was built with these
commands:

$ make distclean
$ make

No environment variables were set to affect the build.

Regards,
Gary

[Bram Moolenaar]

This doesn't look right. Doesn't mintty have a termcap/terminfo entry
and you can set $TERM accordingly?

Since $TERM includes "xterm" Vim assumes the terminal behaves like an
xterm and will enable modifyOtherKeys level 2. This should be harmless
if the terminal advertises to be working like xterm but doesn't actually
support it.

I assume that mintty can't handle the escape sequences that do work for
xterm. To find out which one please use a log:

vim -N -u NONE -i NONE --log logfile --cmd 'set bg&'

Around where you type the edit command you should be able to find some
"raw key input:" and "raw terminal output:" lines that hopefully provide
more information.

You can also change the 'keyprotocol' option, e.g. make it empty, and
see what effect that has.

--
hundred-and-one symptoms of being an internet addict:
197. Your desk collapses under the weight of your computer peripherals.

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Gary Johnson]

Thanks very much for the reply.

It turns out that it does and you can. I've always left it set to
xterm-256color for the usual reason that "everything" knows how to
talk to an xterm-256color terminal.

I set TERM to mintty and the problem disappears.

> Since $TERM includes "xterm" Vim assumes the terminal behaves like an
> xterm and will enable modifyOtherKeys level 2. This should be harmless
> if the terminal advertises to be working like xterm but doesn't actually
> support it.
>
> I assume that mintty can't handle the escape sequences that do work for
> xterm. To find out which one please use a log:
>
> vim -N -u NONE -i NONE --log logfile --cmd 'set bg&'
>
> Around where you type the edit command you should be able to find some
> "raw key input:" and "raw terminal output:" lines that hopefully provide
> more information.

I did that with TERM=xterm-256color and TERM=mintty and used vimdiff
to compare the two logfiles. I found the offending escape sequence,
but I don't know what it means.

When TERM=xterm-256color, at the end of the "raw terminal output:"
line that prints the introductory message is the sequence

^[[?4m

I looked in the xterm source, in the file ctlseqs.txt, but could not
find that sequence, i.e., CSI ? ... m.

When I send that sequence from Vim to mintty with

:call echoraw("\e[?4m")

characters on the command line are printed in the tiny font.

I did a similar comparison between the logfiles with and without "-c
'set bg&'" argument. When that argument was _not_ present, the
logfile contained additional "raw terminal output:" lines that
presumably reset the font.

> You can also change the 'keyprotocol' option, e.g. make it empty, and
> see what effect that has.

Adding the argument

--cmd 'set keyprotocol='

or

-c 'set keyprotocol='

had no effect, whether set before or after bg&: the font on the
command line was always tiny.

So the correct solution is to set TERM=mintty. I'll do that and see
if anything else breaks, although I don't use that computer much for
my current work, so I won't notice anything for a while.

I would still like to know what that escape sequence is supposed to
do. If mintty's response seems to be a bug, then I'll report it to
the author.

Regards,
Gary

[Christopher Plewright]

ESC[4m is "set underline mode"

[Gary Johnson]

On 2022-12-02, Christopher Plewright wrote:
>
> ESC[4m is "set underline mode"

Thanks, but the sequence I'm seeing is ESC[?4m.

Regards,
Gary

[Bram Moolenaar]

Gary Johnson wrote:

[...]

> > I assume that mintty can't handle the escape sequences that do work for
> > xterm. To find out which one please use a log:
> >
> > vim -N -u NONE -i NONE --log logfile --cmd 'set bg&'
> >
> > Around where you type the edit command you should be able to find some
> > "raw key input:" and "raw terminal output:" lines that hopefully provide
> > more information.
>
> I did that with TERM=xterm-256color and TERM=mintty and used vimdiff
> to compare the two logfiles. I found the offending escape sequence,
> but I don't know what it means.
>
> When TERM=xterm-256color, at the end of the "raw terminal output:"
> line that prints the introductory message is the sequence
>
> ^[[?4m
>
> I looked in the xterm source, in the file ctlseqs.txt, but could not
> find that sequence, i.e., CSI ? ... m.

This is a new escape sequence that xterm version 377 supports. It is
used to request the current modifyOtherKeys state. It was supposed to
not do anything for existing terminals. It is documented here:
https://invisible-island.net/xterm/ctlseqs/ctlseqs.html
Search for "XTQMODKEYS".

I looked at what appears to be the page that explains escape sequences
for Mintty: https://github.com/mintty/mintty/wiki/CtrlSeqs
I cannot find this escape sequence there. Does this happen
accidentally? You could ask the mintty project about it.

> > You can also change the 'keyprotocol' option, e.g. make it empty, and
> > see what effect that has.
>
> Adding the argument
>
> --cmd 'set keyprotocol='
>
> or
>
> -c 'set keyprotocol='
>
> had no effect, whether set before or after bg&: the font on the
> command line was always tiny.

Right, even when 'keyprotocol' is empty the "xterm" builtin terminal
entries are still setup to use modifyOtherKeys. It looks like it is now
OK to remove them and let the default 'keyprotocol' value do its work.

> So the correct solution is to set TERM=mintty. I'll do that and see
> if anything else breaks, although I don't use that computer much for
> my current work, so I won't notice anything for a while.

Yes, setting $TERM to "mintty" should work best. However, I suspect
many users use some name starting with "xterm" and run into the same
problem as you.

> I would still like to know what that escape sequence is supposed to
> do. If mintty's response seems to be a bug, then I'll report it to
> the author.

It would certainly be good to know if mintty intentionally uses this CSI
code for something. It is unexpected, perhaps even a bug.

--
msdn.microsoft.com:
ERROR_SUCCESS 0 (0x0) The operation completed successfully.
I have always suspected that for Microsoft success is an error.

[Christopher Plewright]

> ESC[4m is "set underline mode"

>Thanks, but the sequence I'm seeing is ESC[?4m.
>
>Regards,
>Gary

Ah, OK, thought I recognised it, but was mistaken. I recall years ago that mintty had an issue moving text up to make space for underline. Thought it might be related, but it seems not.

[Gary Johnson]

On 2022-12-02, Bram Moolenaar wrote:

> Gary Johnson wrote:
>
> [...]
>
> > > I assume that mintty can't handle the escape sequences that do work for
> > > xterm. To find out which one please use a log:
> > >
> > > vim -N -u NONE -i NONE --log logfile --cmd 'set bg&'
> > >
> > > Around where you type the edit command you should be able to find some
> > > "raw key input:" and "raw terminal output:" lines that hopefully provide
> > > more information.
> >
> > I did that with TERM=xterm-256color and TERM=mintty and used vimdiff
> > to compare the two logfiles. I found the offending escape sequence,
> > but I don't know what it means.
> >
> > When TERM=xterm-256color, at the end of the "raw terminal output:"
> > line that prints the introductory message is the sequence
> >
> > ^[[?4m
> >
> > I looked in the xterm source, in the file ctlseqs.txt, but could not
> > find that sequence, i.e., CSI ? ... m.
>
> This is a new escape sequence that xterm version 377 supports. It is
> used to request the current modifyOtherKeys state. It was supposed to
> not do anything for existing terminals. It is documented here:
> https://invisible-island.net/xterm/ctlseqs/ctlseqs.html
> Search for "XTQMODKEYS".

Ah. I didn't think about it being a new escape sequence so I looked
in the source code I had on hand, which was for version 370. Thanks
for the link.

> I looked at what appears to be the page that explains escape sequences
> for Mintty: https://github.com/mintty/mintty/wiki/CtrlSeqs
> I cannot find this escape sequence there. Does this happen
> accidentally? You could ask the mintty project about it.

I didn't see it there, either.

[...]

> > I would still like to know what that escape sequence is supposed to
> > do. If mintty's response seems to be a bug, then I'll report it to
> > the author.
>
> It would certainly be good to know if mintty intentionally uses this CSI
> code for something. It is unexpected, perhaps even a bug.

I've submitted a bug report to the mintty project, issue #1189.

Regards,
Gary

[Gary Johnson]

On 2022-12-02, Gary Johnson wrote:

> I've submitted a bug report to the mintty project, issue #1189.

This doesn't seem to have been a bug in mintty, but mintty resolved
the conflict with xterm anyway in version 3.6.3, which is now the
latest version in Cygwin.

Regards,
Gary