[vim/vim] vim incorrectly sets t_ut even when terminfo does not contain bce (Issue #11716)

Kovid Goyal

unread,

Dec 17, 2022, 5:51:55 AM12/17/22

to vim/vim, Subscribed

Steps to reproduce

TERM=xterm-kitty vim --clean
:set t_ut?
t_ut=y
infocmp -1 xterm-kitty | grep bce

In step 4 we see that xterm-kitty does not define bce in its terminfo. Now try this with a terminfo that does define bce such as TERM=xterm and you once again find t_ut=y showing that vim is ignoring the terminfo database when setting t_ut

This causes rendering artifacts in kitty when using color schemes that define a background color, reference: https://sw.kovidgoyal.net/kitty/faq/#using-a-color-theme-with-a-background-color-does-not-work-well-in-vim

Expected behaviour

t_ut should not be set when terminfo does not contain bce

Version of Vim

9.0.1046

Environment

Arch Linux, kitty 0.26.5, TERM=xterm-kitty, zsh

Logs and stack traces

No response

—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.

Bram Moolenaar

unread,

Dec 17, 2022, 7:26:47 AM12/17/22

to vim/vim, Subscribed

When $TERM is set to a value starting with "xterm" then vim assumes the terminal is xterm-compatible.
If it is not, then set $TERM to another value. This has a long history, many users depend on it.

Try setting $TERM to "kitty", at least my system has a terminfo entry for it.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Kovid Goyal

unread,

Dec 17, 2022, 7:49:58 AM12/17/22

to vim/vim, Subscribed

On Sat, Dec 17, 2022 at 04:26:25AM -0800, Bram Moolenaar wrote:
> When $TERM is set to a value starting with "xterm" then vim assumes the terminal is xterm-compatible.
> If it is not, then set $TERM to another value. This has a long history, many users depend on it.
>
> Try setting $TERM to "kitty", at least my system has a terminfo entry for it.

The official TERM value for kitty is xterm-kitty that is not going to
change. Since you use terminfo for many things, I suggest using it for
this as well. Or create an exception for xterm0kitty if for some reason
you dont want to use terminfo.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Bram Moolenaar

unread,

Dec 17, 2022, 11:17:50 AM12/17/22

to vim/vim, Subscribed

> On Sat, Dec 17, 2022 at 04:26:25AM -0800, Bram Moolenaar wrote:
> > When $TERM is set to a value starting with "xterm" then vim assumes
> > the terminal is xterm-compatible.
> > If it is not, then set $TERM to another value. This has a long
> > history, many users depend on it.
> >
> > Try setting $TERM to "kitty", at least my system has a terminfo
> > entry for it.
>
> The official TERM value for kitty is xterm-kitty that is not going to
> change.

I really wonder why you put "xterm" in the official name even though the
terminal is apparently not xterm compatible.

> Since you use terminfo for many things, I suggest using it for this as
> well. Or create an exception for xterm0kitty if for some reason
> you dont want to use terminfo.

Anything starting with "xterm" will use the builtin xterm termcap.
If that doesn't work for you, then don't prefix "xterm" to $TERM.
This Vim behavior is not going to change, lots of users depend on this.

Looks like we are stuck here.

--
Futility Factor: No experiment is ever a complete failure - it can always
serve as a negative example.

/// Bram Moolenaar -- ***@***.*** -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Kovid Goyal

unread,

Dec 17, 2022, 7:34:15 PM12/17/22

to vim/vim, Subscribed

On Sat, Dec 17, 2022 at 08:17:28AM -0800, Bram Moolenaar wrote:
> I really wonder why you put "xterm" in the official name even though the
> terminal is apparently not xterm compatible.

No terminal other than xterm that has xterm in its TERM is actually
xterm compatible. And vim was one of the many reasons I chose
xterm-kitty. For example,

TERM=kitty vim --clean
:set ttymouse?
ttymouse=

shows ttymouse is unset which means mouse interaction wont work out of
the box.

TERM=xterm-kitty vim --clean
:set ttymouse?
ttymouse=sgr

which means mouse interaction will work.

If you are going to perpetuate this nonsense of detecting terminal
capabilities by doing string comparison on TERM at least be consistent
about it.

>
> > Since you use terminfo for many things, I suggest using it for this as
> > well. Or create an exception for xterm0kitty if for some reason
> > you dont want to use terminfo.
>
> Anything starting with "xterm" will use the builtin xterm termcap.
> If that doesn't work for you, then don't prefix "xterm" to $TERM.
> This Vim behavior is not going to change, lots of users depend on this.

I would love to see some references for which users depend on it and
how.

>
> Looks like we are stuck here.

Bottomline, a terminfo capability exists for this, you are deliberately
choosing not to use it. Using the following algorithm will break exactly
nothing for any of your existing users and unbreak the out-of-the-box
experience for people using vim in kitty with a color scheme that sets a
background color.

1. If TERM=xterm* use the builtin xterm termcap
2. Read the actual terminfo
3. If bce is not set in that terminfo, unset t_ut

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Bram Moolenaar

unread,

Dec 18, 2022, 7:32:34 AM12/18/22

to vim/vim, Subscribed

> On Sat, Dec 17, 2022 at 08:17:28AM -0800, Bram Moolenaar wrote:
> > I really wonder why you put "xterm" in the official name even though the
> > terminal is apparently not xterm compatible.
>
> No terminal other than xterm that has xterm in its TERM is actually
> xterm compatible. And vim was one of the many reasons I chose
> xterm-kitty. For example,
>
> TERM=kitty vim --clean
> :set ttymouse?
> ttymouse=
>
> shows ttymouse is unset which means mouse interaction wont work out of
> the box.
>
> TERM=xterm-kitty vim --clean
> :set ttymouse?
> ttymouse=sgr
>
> which means mouse interaction will work.

So, you say that you prefixed "xterm" to make the mouse work with Vim
"out of the box", and then you start complaining that other things don't
work because of this prefix? I say you need to go back to the original
problem. I don't recall ever seeing a patch to make 'ttymouse' set to
"sgr" for kitty.

The real solution would be to have the type of mouse support be part of
terminfo/termcap. That is where it belongs. Unfortunately, I'm not
aware of anyone working on that. Again unfortunately, this detection
had to be hard-coded in Vim, which requires updating every time a new
terminal pops up (which happens surprisingly often). Or the user
manually setting 'ttymouse', which isn't good either.

> If you are going to perpetuate this nonsense of detecting terminal
> capabilities by doing string comparison on TERM at least be consistent
> about it.

That is offensive. And an indication you don't understand how this
works.

> > Looks like we are stuck here.
>
> Bottomline, a terminfo capability exists for this, you are deliberately
> choosing not to use it.

Eh? I thought your complaint was that I *am* using the terminfo
capability. But for another terminal, which is because you prefix
"xterm" to the name. If $TERM is set to "kitty" it appears to work
fine, as far as I can see.

> Using the following algorithm will break exactly
> nothing for any of your existing users and unbreak the out-of-the-box
> experience for people using vim in kitty with a color scheme that sets a
> background color.
>
> 1. If TERM=xterm* use the builtin xterm termcap
> 2. Read the actual terminfo
> 3. If bce is not set in that terminfo, unset t_ut

No, that breaks all things that prefixing "xterm" to $TERM was fixing.

I propose you stop making suggestions to change Vim and make kitty work
properly with how Vim works. Some changes done for other terminals,
such as the 'ttymouse' detection, are in line with what is already been
done, so that is OK, would be good to see a patch for that. But do not
expect behavior to change.

--
I still remember when I gave up Smoking, Drinking and Sex. It was the
most *horrifying* hour of my life!

/// Bram Moolenaar -- ***@***.*** -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

—

Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Kovid Goyal

unread,

Dec 18, 2022, 9:27:37 AM12/18/22

to vim/vim, Subscribed

On Sun, Dec 18, 2022 at 04:32:12AM -0800, Bram Moolenaar wrote:
>
> > On Sat, Dec 17, 2022 at 08:17:28AM -0800, Bram Moolenaar wrote:
> > > I really wonder why you put "xterm" in the official name even though the
> > > terminal is apparently not xterm compatible.
> >
> > No terminal other than xterm that has xterm in its TERM is actually
> > xterm compatible. And vim was one of the many reasons I chose
> > xterm-kitty. For example,
> >
> > TERM=kitty vim --clean
> > :set ttymouse?
> > ttymouse=
> >
> > shows ttymouse is unset which means mouse interaction wont work out of
> > the box.
> >
> > TERM=xterm-kitty vim --clean
> > :set ttymouse?
> > ttymouse=sgr
> >
> > which means mouse interaction will work.
>
> So, you say that you prefixed "xterm" to make the mouse work with Vim
> "out of the box", and then you start complaining that other things don't
> work because of this prefix? I say you need to go back to the original

No, I said it was one of the many things that motivated this. Sadly you
are not the only person that does this xterm* nonsense. Which is why I was
forced to use xterm-kitty.

> problem. I don't recall ever seeing a patch to make 'ttymouse' set to
> "sgr" for kitty.
>
> The real solution would be to have the type of mouse support be part of
> terminfo/termcap. That is where it belongs. Unfortunately, I'm not
> aware of anyone working on that. Again unfortunately, this detection
> had to be hard-coded in Vim, which requires updating every time a new
> terminal pops up (which happens surprisingly often). Or the user
> manually setting 'ttymouse', which isn't good either.

Shrug. If I were you I would turn on mouse support by default. And
hardcode a blacklist of terminal emulators that break with it on. That
list will be extremely small and grow only smaller with time.

>
>
> > If you are going to perpetuate this nonsense of detecting terminal
> > capabilities by doing string comparison on TERM at least be consistent
> > about it.
>
> That is offensive. And an indication you don't understand how this
> works.

I dont see what is offensive about calling nonsense, nonsense. But sure,
I am sorry to have offended you. Feel free to continue hard coding
things to your hearts content.

>
> > > Looks like we are stuck here.
> >
> > Bottomline, a terminfo capability exists for this, you are deliberately
> > choosing not to use it.
>
> Eh? I thought your complaint was that I *am* using the terminfo
> capability. But for another terminal, which is because you prefix
> "xterm" to the name. If $TERM is set to "kitty" it appears to work
> fine, as far as I can see.

No, you aren't using the terminfo capability, you are using a hardcoded
value. The terminfo capability comes from the actual terminfo database,
of the terminal vim is running on, not your source code.

>
> > Using the following algorithm will break exactly
> > nothing for any of your existing users and unbreak the out-of-the-box
> > experience for people using vim in kitty with a color scheme that sets a
> > background color.
> >
> > 1. If TERM=xterm* use the builtin xterm termcap
> > 2. Read the actual terminfo
> > 3. If bce is not set in that terminfo, unset t_ut
>
> No, that breaks all things that prefixing "xterm" to $TERM was fixing.

Fine, so hardcode an exception for xterm-kitty to the code path the
enables bce. I dont think you can claim that will break anything. And it
fits with your philosophy of hardcoding things.

>
>
> I propose you stop making suggestions to change Vim and make kitty work
> properly with how Vim works. Some changes done for other terminals,
> such as the 'ttymouse' detection, are in line with what is already been
> done, so that is OK, would be good to see a patch for that. But do not
> expect behavior to change.

Sorry, I am not going to fix your editor for you. I have quite enough on
my plate as it is. I have made you aware of the problem, what you chose
to do about it is your concern. Good luck.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Bram Moolenaar

unread,

Dec 18, 2022, 1:21:46 PM12/18/22

to vim/vim, Subscribed

Should be fixed by patch 9.0.1073

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Bram Moolenaar

unread,

Dec 18, 2022, 1:21:47 PM12/18/22

to vim/vim, Subscribed

Closed #11716 as completed.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Shane-XB-Qian

unread,

Dec 18, 2022, 10:01:44 PM12/18/22

to vim/vim, Subscribed

@brammool @kovidgoyal

kitty: 0.21.2
vim --clean


E558: Terminal entry not found in terminfo
'kitty' not known. Available builtin terminals are:
    builtin_ansi
    builtin_vt320
    builtin_vt52
    builtin_xterm
    builtin_iris-ansi
    builtin_pcansi
    builtin_win32
    builtin_amiga
    builtin_dumb
    builtin_debug
defaulting to 'ansi'

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Kovid Goyal

unread,

Dec 18, 2022, 11:59:26 PM12/18/22

to vim/vim, Subscribed

"kitty" is not a valid TERM for kitty, always use "xterm-kitty"

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Shane-XB-Qian

unread,

Dec 19, 2022, 6:41:59 AM12/19/22

to vim/vim, Subscribed

v9.0.1073 removed the xterm- prefix.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Kovid Goyal

unread,

Dec 19, 2022, 7:00:23 AM12/19/22

to vim/vim, Subscribed

Yeah that will break lots of things in lots of places. There is no kitty
terminfo. It's just a made up one added by Dickey to some version of
ncurses that doesn't even track the actual kitty terminfo. Not to
mention it wont be available on remote machines when using the ssh
kitten since it only installs xterm-kitty.

The correct solution is to hardcode the value of t_ut alone for
xterm-kitty.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Kovid Goyal

unread,

Dec 19, 2022, 7:04:07 AM12/19/22

to vim/vim, Subscribed

Oh and just for posterity, there is not a single terminal emulator that depends on vim hardcoding a terminfo definition for xterm*. vim could just use the actual terminfo that came with the TERM variable and all those terminal emulators would work just fine. Indeed the whole purpose of terminfo is to provide a database of terminal properties precisely so applications dont have to hardcode them.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Bram Moolenaar

unread,

Dec 19, 2022, 7:21:51 AM12/19/22

to vim/vim, Subscribed

> "kitty" is not a valid TERM for kitty, always use "xterm-kitty"

My system, Ubuntu 22.04, does have a "kitty" terminfo/termcap entry.

Using the "xterm-" prefix is also wrong, since the terminal is not
compatible with xterm, that is that this issue is about.

I suggest you work towards making the name "kitty" work, everything else
looks like a workaround rather than a solution.

--
CUSTOMER: Well, can you hang around a couple of minutes? He won't be
long.
MORTICIAN: Naaah, I got to go on to Robinson's -- they've lost nine today.
CUSTOMER: Well, when is your next round?
MORTICIAN: Thursday.
DEAD PERSON: I think I'll go for a walk.
The Quest for the Holy Grail (Monty Python)

/// Bram Moolenaar -- ***@***.*** -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

—

Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Kovid Goyal

unread,

Dec 19, 2022, 7:27:02 AM12/19/22

to vim/vim, Subscribed

I suggest you revert 9.0.1073 unless you want to get a flood of bug reports. Even colorschemes that use rgb colors look wrong with that patch.

I am perfectly fine with you not fixing this bug. I filed it in a moment of unfounded optimism. I will just continue to direct vim users that experience it to the FAQ which has been around for years.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Bram Moolenaar

unread,

Dec 19, 2022, 8:27:53 AM12/19/22

to vim/vim, Subscribed

> Oh and just for posterity, there is not a single terminal emulator
> that depends on vim hardcoding a terminfo definition for xterm*. vim
> could just use the actual terminfo that came with the TERM variable
> and all those terminal emulators would work just fine. Indeed the
> whole purpose of terminfo is to provide a database of terminal
> properties precisely so applications dont have to hardcode them.

When I stopped using the builtin xterm termcap entries for Kitty several
things stopped working. So you are wrong, this builtin xterm termcap is
essential and lots of things will break without it. I suggest you
actually try it out before making bold statements that are wrong.

If you have a look at what actually happens, you will see that this
builtin xterm termcap has entries that don't even exist in terminfo.

The purpose for terminfo you state is right, but terminfo hasn't met
this purpose for a very long time. There is a big difference between
how things are supposed to work and how they actually work. And users
have to deal with the last.

--
[clop clop]
MORTICIAN: Who's that then?
CUSTOMER: I don't know.
MORTICIAN: Must be a king.
CUSTOMER: Why?
MORTICIAN: He hasn't got shit all over him.

The Quest for the Holy Grail (Monty Python)

/// Bram Moolenaar -- ***@***.*** -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

—

Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Shane-XB-Qian

unread,

Dec 19, 2022, 8:51:19 AM12/19/22

to vim/vim, Subscribed

1. Kovid:

> Not to mention it wont be available on remote machines when using the ssh kitten since it only installs xterm-kitty.

2. Bram:

> My system, Ubuntu 22.04, does have a "kitty" terminfo/termcap entry.

3. Kovid:

> Even colorschemes that use rgb colors look wrong with that patch.

4. Shane:
UTF issue, some wide-chars (e.g when copy/paste from local to ssh remote) lost chars, or displayed wrong (local or remote).

Though irrelevant but above 4 looks were the *most freq* faced problem when using some terminal (or with Gun-screen or with Vim or at local / ssh remote) per experience, just Not always OUT-OF-BOX!

--
shane.xb.qian

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Kovid Goyal

unread,

Dec 19, 2022, 9:18:07 AM12/19/22

to vim/vim, Subscribed

On Mon, Dec 19, 2022 at 05:27:29AM -0800, Bram Moolenaar wrote:
>
> > Oh and just for posterity, there is not a single terminal emulator
> > that depends on vim hardcoding a terminfo definition for xterm*. vim
> > could just use the actual terminfo that came with the TERM variable
> > and all those terminal emulators would work just fine. Indeed the
> > whole purpose of terminfo is to provide a database of terminal
> > properties precisely so applications dont have to hardcode them.
>
> When I stopped using the builtin xterm termcap entries for Kitty several
> things stopped working. So you are wrong, this builtin xterm termcap is
> essential and lots of things will break without it. I suggest you
> actually try it out before making bold statements that are wrong.

We are just talking past each other. I am saying use terminfo for things
that are *in terminfo*. Things that are not in terminfo obviously need
some other technique for detection. My statement is, that if you use
terminfo for properties actually present in terminfo, nothing will
break. So in your scenario, you read the terminfo for the terminal you
are running in, then, you fill in those properties in what you have read
that arent actually present in terminfo. I stand by my "bold claim" that
doing so will break nothing.

In this particular case t_ut aka bce is actually in terminfo. All the
xterm* terminfo databases define it, except for kitty's. Because kitty
does not implement it. Thankfully, Dickey added a property to terminfo
because there indeed are terminals that choose not to implement it.
And so kitty uses that property to inform applications of the fact that
it is one of those terminals that does not support it. It would be nice
if vim would respect kitty's terminfo. If you dont want to do that,
fine, that's your choice, but I again strongly urge you not to change
xterm-kitty to kitty, that will have *far* worse effects than the
problem you are trying to solve.

> If you have a look at what actually happens, you will see that this
> builtin xterm termcap has entries that don't even exist in terminfo.

Yes, I know, I talked about one earlier, ttymouse. I have nothing
against you hardcoding those, or doing anything you like about them. I
am simply saying dont hardcode bce since it actually exists in terminfo.
More generally dont hardcode properties that are well defined in
terminfo. But if that is a bridge too far, that's fine. Just dont
hardcode bce. Or if even that is a bridge too far, then just special
case t_ut for xterm-kitty, so your users dont have to do it themselves
in ~/.vimrc. If even that is a bridge too far, then feel free to do
nothing, but please, do NOT change xterm-kitty to kitty.

>
> The purpose for terminfo you state is right, but terminfo hasn't met
> this purpose for a very long time. There is a big difference between
> how things are supposed to work and how they actually work. And users
> have to deal with the last.

I opened this bug report because this issue causes users to have to
workaround it in vimrc. I was naively hoping that since they are your
users too, you would like to make their lives a little easier. In the
current situation what often happens is a user uses a color theme with a
background color and it causes rendering issues in kitty+vim. But not in
kitty+neovim. Or not in other terminal+vim. And they get confused. And
they report a bug either here or in kitty's bug tracker. I was hoping
that you would be willing to fix that, by doing the IMO reasonable thing
of using terminfo for bce, since terminfo in fact defines bce.

But if you dont want to do that, OK. Your current fix is worse than the
original issue, so please revert it and feel free to ignore this bug
report. I am heartily sorry I opened it.

—
Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Bram Moolenaar

unread,

Dec 19, 2022, 11:50:42 AM12/19/22

to vim/vim, Subscribed

Kovid Goyal wrote:

[...]

I do not quite get the idea of what you are proposing.
I suggest you create a PR so we can see how you would make this work.

--
WOMAN: Dennis, there's some lovely filth down here. Oh -- how d'you do?
ARTHUR: How do you do, good lady. I am Arthur, King of the Britons.
Who's castle is that?
WOMAN: King of the who?

The Quest for the Holy Grail (Monty Python)

/// Bram Moolenaar -- ***@***.*** -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

—

Reply to this email directly, view it on GitHub.

You are receiving this because you are subscribed to this thread.

Kovid Goyal

unread,

Dec 19, 2022, 11:57:11 AM12/19/22

to vim/vim, Subscribed

I am proposing you revert 9.0.1073 for the moment. If I have some time in the future I will put together a PR for you.