Vim cannot detect background correctly when ssh to a remote server

[Steven]

Hi,

Recently I use iTerm 2 to ssh to my remote server, then use Vim to edit files, something very simple like:

$ ssh root@<remote server>

# On my remote server
$ vim foo.md

I also use the macOS built-in Terminal to do the same practice, but then I've noticed that the syntax highlighting colors in Terminal and iTerm are different.

I've spent quite some time looking into this, and it turns out that Vim on my remote server sets background differently when using Terminal and iTerm.

# Using iTerm
$ vim
$ :set bg?
$ background=light

# Using Terminal
$ vim
$ :set bg?
$ background=dark

In fact both my Terminal and iTerm are using Solarized Dark theme, which means Vim background setting on iTerm is not correct. I'm using the latest iTerm (installed via Homebrew) and my remote server is running on Ubuntu 18.04 LTS, with the latest vim (installed via apt).

Some other explorations I've done:

1. On my local computer (instead of ssh to the remote), Vim backgrounds are both correct.
2. I haven't set background in ~/.vimrc on my remote server.
3. Both Terminal and iTerm declare $TERM as xterm-256color
4. Vim t_Co variable is 256 on both Terminal and iTerm

I'd really appreciate it if anyone could offer some help.

[Adrian Bool]

Hi Steven,

What are you doing sshing into an Ubuntu server as root? ;-)

The bg variable in vim is controlled by the $COLORFGBG variable.  On my (10.14.6) Mac, iTerm is setting this variable and Terminal is not.  First check this variable in both Macs. 

Then ensure the variable is passed through to the server by adding “SendEnv COLORFGBG” in ~\.ssh\config on your Mac and “AcceptEnv COLORFGBG” in /etc/ssh/sshd_config on the Ubuntu server.  Oh, and restart sshd on the server. 

Then, check this variable via both terminals on the server and see how vim bg is reacting...

Good luck...

aid

Sent from my iPhone

On 25 Feb 2020, at 02:45, Steven <steve...@gmail.com> wrote:



--
You received this message because you are subscribed to the Google Groups "iterm2-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iterm2-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/iterm2-discuss/89d60420-88a4-4ef8-8c68-02ad0c157b6a%40googlegroups.com.

[Adrian Bool]

Hi,

Some more info...

iTerm will only populate the $COLORFGBG variable if the "Set locale variables automatically" option is ticked within Settings -> Profile -> Terminal  – I think that should be enabled by default, but worth checking.

Also, there seems there is another way for the remote app (i.e. vim) to determine the colour of the terminal.  Try running the following command in each of iTerm/Termal both local/remote and see if any response is generated:

echo -n -e '\x1b]11;?\x07\n'

(On my system iTerm is responding to the above sequence with the current background colour; but Terminal is not.)

Cheers,

aid

[Steven]

Hi there,

Thanks a lot for the quick reply. It's true that iTerm sets the $COLORFGBG variable while Terminal does not. And it works to pass this variable to my remote server and make Vim display the proper background.

But I'm still confused. If I do not pass this variable, $COLORFGBG will not be set on the remote server no matter using Terminal or iTerm. In that case, why would Vim background display differently?

Also I've tried the given command and here's the output:

# Terminal, local
11;rgb:0000/1e2a/26d6

# Terminal, remote
11;rgb:0000/1e2a/26d6

# iTerm, local
11;rgb:00/2b/36

# iTerm, remote
11;rgb:00/2b/36

Seems that Terminal / iTerm are giving different results :(

[Rick Hornsby]

On Monday, February 24, 2020 at 10:25:47 PM UTC-6, Adrian Bool wrote:

Hi Steven,

What are you doing sshing into an Ubuntu server as root? ;-)

Not to pile on (also acknowledging it's a bit OT), but Adrian is right about this. Allowing root ssh access is a bad idea. Allowing remote root ssh access escalates from bad idea to dangerous. Since this is iTerm we're talking about - console/remote access type stuff - it's a good reminder for all of us.

I say this because I've watched the logs go by "sshd: too many failed attempts for root@" for a long time. The root account on this box isn't even enabled. And they've been trying for years, literally. They have all the time in the world, and only have to guess root's password right once. Since they are already trying 'root' - half of what they need to get in, if they can a) guess/stumble upon your root password (they have infinite time, and basically infinite attempts) or b) somehow exploit a vulnerability in ssh - your box is toast, and you probably won't even know it.

In most distros these days, you have to go out of your way to enable root logins. You're much, much more secure if you log in with a named account (ie `steven`) using an ssh keypair, and then `sudo su` to root when you need root privs.

[Steven]

On Tuesday, February 25, 2020 at 2:34:06 PM UTC+8, Rick Hornsby wrote:

Not to pile on (also acknowledging it's a bit OT), but Adrian is right about this. Allowing root ssh access is a bad idea. Allowing remote root ssh access escalates from bad idea to dangerous. Since this is iTerm we're talking about - console/remote access type stuff - it's a good reminder for all of us.

I say this because I've watched the logs go by "sshd: too many failed attempts for root@" for a long time. The root account on this box isn't even enabled. And they've been trying for years, literally. They have all the time in the world, and only have to guess root's password right once. Since they are already trying 'root' - half of what they need to get in, if they can a) guess/stumble upon your root password (they have infinite time, and basically infinite attempts) or b) somehow exploit a vulnerability in ssh - your box is toast, and you probably won't even know it.

In most distros these days, you have to go out of your way to enable root logins. You're much, much more secure if you log in with a named account (ie `steven`) using an ssh keypair, and then `sudo su` to root when you need root privs.

Thanks for the kindly reminder :). Actually I was just starting a temporary cloud instance to run some tests and will destroy it later, so I ssh as root (with key pairs and set Password Authentication off) for convenience. I'm aware that it's not a safe choice.

[Adrian Bool]

Hi,

I'm on Catalina machine now; and the Catalina Terminal is responding to the '\x1b]11;?\x07\n' escape sequence; as yours is doing below.

It is pretty obvious that the Catalina Terminal is returning the RGB colours as 16-bit values; whilst iTerm returning 8-bit values.

I've failed so far at finding where when in vim's source code it parses the response to this escape sequence – but my feeling at the moment is that vim is looking for a 16-bit response and rejecting iTerm's 8-bit response.  That's just a guess though...  Perhaps I'll be able to find it this evening...

Cheers,

aid

--
You received this message because you are subscribed to the Google Groups "iterm2-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iterm2-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/iterm2-discuss/e4d357ef-b289-48fa-8519-1e0ce1103ce4%40googlegroups.com.

[Steven]

Thanks and got it. At least it's not only me who can reproduce this issue... :P

What's more confusing is that Vim on my local computer (version 8.1, macOS built-in) can detect backgrounds correctly. Could it be that Vim on macOS and Ubuntu guesses terminal backgrounds differently?

Earlier when I was looking into this I found a stackoverflow question about how Vim guesses background color. Since I know almost nothing about C programming language, I hope it can be of some help :) 

[Adrian Bool]

Hi Steven,

I had a look at the VIM source code and its git commit log.   The code that handles this colour response is in src/term.c at around line 4955 (should anyone want to look themselves).

There was a patch applied to this section of code within vim on 05/06/2019 with the following comment:

Problem: 'background' not correctly set for 2-digit rgb termresponse.

Solution: Adjust what digit to use. (closes #4495)

Looks like this change was applied against vim version 8.1.  My Ubuntu 18.04 LTS systems are all running vim 8.0.1453 and hence seeing the issue with  iTerm.  I downloaded and compiled the latest source which gave me version 8.2.314. When running the 8.2 version of vim the dark/light is detected correctly with iTerm – with no need for environment variables (i.e. $COLORFGBG).

So, I guess you could download and compile a new vim on your server; or persuade George to move to a 16-bit response for the terminal colours in iTerm.

The iTerm update may well consist of updating the following function in VT1000Output.m:

- (NSData *)reportColor:(NSColor *)color atIndex:(int)index prefix:(NSString *)prefix {

NSString *string = [NSString stringWithFormat:@"%c]%@%d;rgb:%02x/%02x/%02x%c",

ESC,

prefix,

index,

(int) ([color redComponent] * 255.0),

(int) ([color greenComponent] * 255.0),

(int) ([color blueComponent] * 255.0),

7];

return [string dataUsingEncoding:NSUTF8StringEncoding];

}

To something like:

- (NSData *)reportColor:(NSColor *)color atIndex:(int)index prefix:(NSString *)prefix {

NSString *string = [NSString stringWithFormat:@"%c]%@%d;rgb:%04x/%04x/%04x%c", // 16-bit response for R,G,B

ESC,

prefix,

index,

(int) ([color redComponent] * 255.0 * 255.0), // shift value two bytes left

(int) ([color greenComponent] * 255.0 * 255.0), // shift value two bytes left

(int) ([color blueComponent] * 255.0 * 255.0), // shift value two bytes left

7];

return [string dataUsingEncoding:NSUTF8StringEncoding];

}

I've checked this out; and the change seems to work OK – at least with vim.  It is possible that other console applications only like 8-bit responses....

Cheers,

aid

--
You received this message because you are subscribed to the Google Groups "iterm2-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iterm2-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/iterm2-discuss/1d61c7e2-a12f-4e9c-963a-000f06c3ff93%40googlegroups.com.

[Steven]

I see. Thanks so much for your help :)

[George Nachman]

Sorry for the late reply. COLORFGBG doesn't get passed across an ssh connection as a security feature of ssh. You can either change this in your ssh server's configuration, or do the lazy thing which is to copy it to LC_COLORFGBG before sshing and then copy it from LC_COLORFGBG to COLORFGBG on the remote. That hack works because anything starting with LC_ gets sent by ssh.

Commit 5c5785f3632b8e90dd69f458411a8b8b17aa0599 makes iTerm2 respond with 16 bit color components to work around the vim bug.

To view this discussion on the web visit https://groups.google.com/d/msgid/iterm2-discuss/638D49B3-812C-4A62-91C9-6E9CBA0594D5%40logic.org.uk.