Same number different colors for ctermbg and ctermfg

[Enan Ajmain]

Hi,

For the value 7, ctermfg and ctermbg gives different colors. 'ctermfg=7'
gives non-bold white, which is expected. But 'ctermbg=7' gives bold
Black color. I'm assuming there is a if-conditional somewhere that's
causing this distinction. If so, where is the conditional in the source
code? If not, why is this happening?

P.S. I'm on Windows. I have a Linux laptop but haven't checked if this
behavior is reproducible on that OS.

--
Enan

[Tony Mechelynck]

Try to explicitly remove bold, as follows:
:hi User1 cterm=NONE ctermbg=red ctermfg=7 gui=NONE guibg=#FF0000 guifg=#777777
:hi User2 cterm=NONE ctermbg=7 ctermfg=red gui=NONE guibg=#777777 guifg=#FF0000
(Of course in order to see something we cannot set both ctermbg=7 and
ctermfg=7 on the same highlight group).
IIUC a Vim instance compiled with no GUI should ignore the gui= guibg=
guifg= settings but I don't like to define a highlight group with one
of cterm and gui colors and not the other.
You may replace User1 and User2 by whatever suits you, but if the
groups you set are already defined you may need (I'm not sure) to
clear them first with ":hi clear <name>"; then set the highlight for
two regions with text to the two highlight groups so defined.
Does the problem disappear this way, with cterm=NONE prefixed to the
ctermbg and ctermfg settings?

Best regards,
Tony.

[Enan Ajmain]

On Thu, 4 May 2023 12:38:52 +0200
Tony Mechelynck <antoine.m...@gmail.com> wrote:
> [...]

> (Of course in order to see something we cannot set both ctermbg=7 and
> ctermfg=7 on the same highlight group).

See the attached image. I agree with you: ctermbg and ctermfg, if set
to the same numeric value, should show the same color and basically be
an invisible block. But that's not true when the color value is 7.
That's the issue I was describing.

I may have misled you by using the term "bold black." Probably should
have used Bright Black. In any case, the 'cterm=' value is of no
consequence here (and it's set to NONE already).


--
Enan

[Bram Moolenaar]

Color support depends on the terminal. Background and foreground
settings are often different, especially with older terminals and a low
number of colors supported. Sometimes the "bold" flag changes the color
between a darker and lighter color. You can find some information at
":help highlight-ctermfg".

--
Engineers are always delighted to share wisdom, even in areas in which they
have no experience whatsoever. Their logic provides them with inherent
insight into any field of expertise. This can be a problem when dealing with
the illogical people who believe that knowledge can only be derived through
experience.
(Scott Adams - The Dilbert principle)

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// \\\
\\\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[Enan Ajmain]

On Thu, 04 May 2023 15:48:43 +0100
Bram Moolenaar <Br...@moolenaar.net> wrote:
> > For the value 7, ctermfg and ctermbg gives different colors. 'ctermfg=7'
> > gives non-bold white, which is expected. But 'ctermbg=7' gives bold
> > Black color. I'm assuming there is a if-conditional somewhere that's
> > causing this distinction. If so, where is the conditional in the source
> > code? If not, why is this happening?
>

> Color support depends on the terminal. Background and foreground
> settings are often different, especially with older terminals and a low
> number of colors supported. Sometimes the "bold" flag changes the color
> between a darker and lighter color. You can find some information at
> ":help highlight-ctermfg".

I shouldn't have said "bold black." That caused unnecessary confusion.
The 'cterm=' value is NONE. I was just trying to say "darker" shade of
black.

> > P.S. I'm on Windows. I have a Linux laptop but haven't checked if this
> > behavior is reproducible on that OS.

I checked on my Linux laptop. These are the behaviors:

highlight | Linux | Windows
------------+---------------+-------------
ctermfg=0 | Black | Black
ctermfg=7 | White | Black
ctermfg=8 | Bright black | Bright black
ctermfg=15 | Bright white | Bright white

I run Windows Terminal on Windows and Suckless Terminal on Linux. The
colors in my WT are set in a JSON file, with 16 keys. But the colors in
my ST is a C array of strings, and the indices match with Vim's behavior
of 'ctermfg=<index>', i.e., 'ctermfg=0' is black and 'ctermfg=8' brblack
(that's how it's named in the comments in ST's source code [1]).

The fun part is: I ssh'd into my Linux laptop from my Windows laptop.
Meaning, I ran Vim on the Linux machine but in WT. There, 'ctermfg=7'
still showed White, unlike the Vim on native Windows. Seems like WT
does show 'ctermfg=7' as white, but the 'builtin_win32' term isn't
supplying WT with proper whatchamacallit. (I'm probably wrong. This was
just a guess.)

[1]: https://github.com/3N4N/st/blob/5dc3385bfce0e25d04942d2e0d9af8f9ba94d70b/config.h#L96-L114

--
Enan

[Bram Moolenaar]

> > > For the value 7, ctermfg and ctermbg gives different colors. 'ctermfg=7'
> > > gives non-bold white, which is expected. But 'ctermbg=7' gives bold
> > > Black color. I'm assuming there is a if-conditional somewhere that's
> > > causing this distinction. If so, where is the conditional in the source
> > > code? If not, why is this happening?
> >
> > Color support depends on the terminal. Background and foreground
> > settings are often different, especially with older terminals and a low
> > number of colors supported. Sometimes the "bold" flag changes the color
> > between a darker and lighter color. You can find some information at
> > ":help highlight-ctermfg".
>
> I shouldn't have said "bold black." That caused unnecessary confusion.
> The 'cterm=' value is NONE. I was just trying to say "darker" shade of
> black.
>
> > > P.S. I'm on Windows. I have a Linux laptop but haven't checked if this
> > > behavior is reproducible on that OS.
>
> I checked on my Linux laptop. These are the behaviors:
>
> highlight | Linux | Windows
> ------------+---------------+-------------
> ctermfg=0 | Black | Black
> ctermfg=7 | White | Black
> ctermfg=8 | Bright black | Bright black
> ctermfg=15 | Bright white | Bright white

So what is the number of colors, set with the t_Co option in each case?

> I run Windows Terminal on Windows and Suckless Terminal on Linux. The
> colors in my WT are set in a JSON file, with 16 keys.

Not sure what this means. Do you manually specify colors for the
Windows Terminal? Obviously Vim cannot know what these colors are.

For "ctermfg-7" the color should probably "grey", neither white or
black. "Bright white" is just white, nothing is brighter than that.
"Bright black" is some kind of grey.

> But the colors in
> my ST is a C array of strings, and the indices match with Vim's behavior
> of 'ctermfg=<index>', i.e., 'ctermfg=0' is black and 'ctermfg=8' brblack
> (that's how it's named in the comments in ST's source code [1]).

I don't understand any of this. If this is about a terminal with 8 or
16 colors, better follow the standard colors. Vim will not be able to
handle any manually configured colors.

> The fun part is: I ssh'd into my Linux laptop from my Windows laptop.

What do you set $TERM to then? I doubt a Linux system has any specific
terminfo/termcap entry for that. I don't even know what ssh command
would be able to make a connection from a Windows Terminal. I've used
Putty (with varying success) and the Secure Shell built into Chrome,
which works quite well.

> Meaning, I ran Vim on the Linux machine but in WT. There, 'ctermfg=7'
> still showed White, unlike the Vim on native Windows. Seems like WT
> does show 'ctermfg=7' as white, but the 'builtin_win32' term isn't
> supplying WT with proper whatchamacallit. (I'm probably wrong. This was
> just a guess.)

The 'builtin_win32" entry predates Windows Terminal by many years, it
was made for the Windows console, and not for use with ssh. I don't
know what best to use for Windows Terminal, it has been under
development for quite a long time. Not sure if it can be considered
"stable" now. And I have no idea what to expect from an ssh command
running in Windows Terminal. What would it set $TERM to? What ssh
command is even available for this?

--
For society, it's probably a good thing that engineers value function over
appearance. For example, you wouldn't want engineers to build nuclear power
plants that only _look_ like they would keep all the radiation inside.

[Enan Ajmain]

I'm not an expert on terminal emulators, so this may be incorrect, but
as far as I know all modern terminal emulators let the user define 16
colors, even if the terminal emulator supports true color (256 colors).

And as for Vim handling manually configured colors, note 'ctermfg=1' is
red . If I change the value of 'Red' in Suckless Terminal config to a
hex value of something green, 'ctermfg=1' shows green. I mean, this is
the expected behavior, right? Vim gives the terminal emulator a color
code; the terminal emulator decides how to render that, according to its
config.

> > The fun part is: I ssh'd into my Linux laptop from my Windows
> > laptop.
>
> What do you set $TERM to then? I doubt a Linux system has any
> specific terminfo/termcap entry for that. I don't even know what ssh
> command would be able to make a connection from a Windows Terminal.
> I've used Putty (with varying success) and the Secure Shell built
> into Chrome, which works quite well.

I ssh'd with MSYS's ssh binary. The $TERM value is undefined in Windows
Terminal. ':set term?' in Vim-in-Linux-over-SSH-on-Windows shows
'xterm-256colors'. Don't know where that's being defined.

> > Meaning, I ran Vim on the Linux machine but in WT. There,
> > 'ctermfg=7' still showed White, unlike the Vim on native Windows.
> > Seems like WT does show 'ctermfg=7' as white, but the
> > 'builtin_win32' term isn't supplying WT with proper
> > whatchamacallit. (I'm probably wrong. This was just a guess.)
>
> The 'builtin_win32" entry predates Windows Terminal by many years, it
> was made for the Windows console, and not for use with ssh. I don't
> know what best to use for Windows Terminal, it has been under
> development for quite a long time. Not sure if it can be considered
> "stable" now. And I have no idea what to expect from an ssh command
> running in Windows Terminal. What would it set $TERM to? What ssh
> command is even available for this?

I think I covered this above. And anyway, I was not intending to dive
into terminal stuff. I thought 'ctermfg=7' was a case specially handled
in Vim source, basically something like this:

if (do_ctermfg && arg == 7) {
arg = 0; // forcefully use black color
}

This ^ is what I was expecting. If true, I was gonna comment it out.
Seems like this isn't the case. No problem. Thanks.


--
Enan

[Bram Moolenaar]

Usually when a terminal supports more than 16 colors, those first 16
colors may be very well different from terminals that only support 16
colors. True color is much more than 256 colors. 256 color support was
an intermediate step, and most follow the colors that xterm uses (but
not every one).

> And as for Vim handling manually configured colors, note 'ctermfg=1' is
> red . If I change the value of 'Red' in Suckless Terminal config to a
> hex value of something green, 'ctermfg=1' shows green. I mean, this is
> the expected behavior, right? Vim gives the terminal emulator a color
> code; the terminal emulator decides how to render that, according to its
> config.

Although technically correct, this is also very confusing. Better keep
the terminal at showing the default colors (e.g. the ANSI colors) and
adjust the colors in Vim using a colorscheme. Otherwise you can't
really use any colorscheme.

> > > The fun part is: I ssh'd into my Linux laptop from my Windows
> > > laptop.
> >
> > What do you set $TERM to then? I doubt a Linux system has any
> > specific terminfo/termcap entry for that. I don't even know what ssh
> > command would be able to make a connection from a Windows Terminal.
> > I've used Putty (with varying success) and the Secure Shell built
> > into Chrome, which works quite well.
>
> I ssh'd with MSYS's ssh binary. The $TERM value is undefined in Windows
> Terminal. ':set term?' in Vim-in-Linux-over-SSH-on-Windows shows
> 'xterm-256colors'. Don't know where that's being defined.

I don't know the details about this ssh implementation. I very much
doubt it actually simulates an xterm accurately. If it doesn't work,
try another ssh. Trying to make this setup work with MSYS may be a
waste of time.

> > > Meaning, I ran Vim on the Linux machine but in WT. There,
> > > 'ctermfg=7' still showed White, unlike the Vim on native Windows.
> > > Seems like WT does show 'ctermfg=7' as white, but the
> > > 'builtin_win32' term isn't supplying WT with proper
> > > whatchamacallit. (I'm probably wrong. This was just a guess.)
> >
> > The 'builtin_win32" entry predates Windows Terminal by many years, it
> > was made for the Windows console, and not for use with ssh. I don't
> > know what best to use for Windows Terminal, it has been under
> > development for quite a long time. Not sure if it can be considered
> > "stable" now. And I have no idea what to expect from an ssh command
> > running in Windows Terminal. What would it set $TERM to? What ssh
> > command is even available for this?
>
> I think I covered this above. And anyway, I was not intending to dive
> into terminal stuff. I thought 'ctermfg=7' was a case specially handled
> in Vim source, basically something like this:
>
> if (do_ctermfg && arg == 7) {
> arg = 0; // forcefully use black color
> }
>
> This ^ is what I was expecting. If true, I was gonna comment it out.
> Seems like this isn't the case. No problem. Thanks.

It's much more complicated than that. E.g. search for "cterm_ansi_idx"
in src/term.c, this is used specifically for MS-Windows. Thus if you
use an MS-Windows console and ssh to a Linux system this mapping is
missing and the colors are likely messed up.

--
Bad fashion can discourage normal people from interacting with the engineer
and talking about the cute things their children do.

[Enan Ajmain]

Thanks. Just a closing correction: the colors on
vim-that-is-actually-on-Linux-but-using-over-ssh-in-Windows is NOT
messed up. It shows perfectly. 'ctermfg=7' shows the proper color that's
set in the terminal emulator. It's the native Vim on Windows that shows
unexpected (or what I think is considered unexpected).

But anyway, thanks.

--
Enan

[Enan Ajmain]

There has been a development:
https://github.com/microsoft/terminal/issues/15408

--
Enan

[Enan Ajmain]

On Wed, 24 May 2023 02:30:21 +0600

A comment from j4james (a collaborator for MS Terminal):

> If it were up to me, I'd dump their current win32 code, and replace it
> with the whatever they're using on Linux, since that clearly already
> works in Windows Terminal when vim is run from WSL. It's probably a
> little more complicated than that, but there really should be no need
> for them to be using this weird mix of legacy console APIs interspersed
> with VT sequences.

This is not to put responsibility on Vim devs. This is just to let the
Windows users among Vim devs (Yegappan perhaps? He picked up my half
baked bugfix for 'shellcmdflag' for powershell last year and polished
it for final merge) know that maybe we can use unix logic for win32 as
well. Maybe. If someone wants to look into that.

(I'll investigate some and share my findings, but I know nothing about
terminal stuff so can't say I'll be useful.)

--
Enan

[Bram Moolenaar]

The Vim code has quite a few assumptions about how things work on
MS-Windows. The code to use a terminal is currently not included in the
MS-Windows build, only the GUI and console support is. Also including
the terminal support should be possible, but it will be a bit of a
puzzle to adjust the #ifdef's to include the code and then to decide (at
runtime) which code is active. We already have the VIMDLL macro, with
the gui.in_use flag, this would be an addition to that.

It should be doable, but it will be quite a bit of work to "get it
right". We also need to keep in mind that this third way of using Vim
on MS-Windows can be expected to require maintenance for a long time,
thus it should not be a "quick hack".

--
Don't drink and drive. You might hit a bump and spill your beer.