[vim/vim] [WIP] Supports Windows Terminal (#5546)

134 views
Skip to first unread message

Nobuhiro Takasaki

unread,
Jan 28, 2020, 1:47:04 AM1/28/20
to vim/vim, Subscribed

In term=win32, VTP code "reduction" is required.
First, do not redraw the right edge of the screen currently.
Next, remove the trick of the background color.
And, to avoid a terminal bug, the screen output buffer is flushed each time.
This is the content of the previous PR. Same as the first commit in this PR.

In the second commit, enter ambiguous width characters.
Without this change, NUL (0x00) would be entered instead.

Always keep track of Windows Terminal changes.
I just make it usable.

This patch has not been fully tested.


You can view, comment on, or merge this pull request online at:

  https://github.com/vim/vim/pull/5546

Commit Summary

  • Windows Terminal
  • Entering ambiguous width characters

File Changes

Patch Links:


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

K.Takata

unread,
Jan 28, 2020, 10:51:50 PM1/28/20
to vim/vim, Subscribed

@k-takata commented on this pull request.


In src/os_win32.c:

> @@ -232,6 +236,14 @@ static char_u *exe_path = NULL;
 
 static BOOL win8_or_later = FALSE;
 
+# if defined(__GNUC__) && !defined(__MINGW32__)  && !defined(__CYGWIN__)
+#  define AChar AsciiChar

It seems that AChar is not used.


In src/os_win32.c:

> @@ -232,6 +236,14 @@ static char_u *exe_path = NULL;
 
 static BOOL win8_or_later = FALSE;
 
+# if defined(__GNUC__) && !defined(__MINGW32__)  && !defined(__CYGWIN__)
+#  define AChar AsciiChar
+#  define UChar UnicodeChar
+# else
+#  define AChar uChar.AsciiChar

Same here.

Nobuhiro Takasaki

unread,
Jan 29, 2020, 4:28:24 AM1/29/20
to vim/vim, Subscribed

@ntak commented on this pull request.


In src/os_win32.c:

> @@ -232,6 +236,14 @@ static char_u *exe_path = NULL;
 
 static BOOL win8_or_later = FALSE;
 
+# if defined(__GNUC__) && !defined(__MINGW32__)  && !defined(__CYGWIN__)
+#  define AChar AsciiChar

Perhaps it should be used. I will fix it from now.

Nobuhiro Takasaki

unread,
Jan 29, 2020, 7:02:52 AM1/29/20
to vim/vim, Push

@ntak pushed 1 commit.


You are receiving this because you are subscribed to this thread.

View it on GitHub or unsubscribe.

K.Takata

unread,
Jan 29, 2020, 1:23:31 PM1/29/20
to vim/vim, Subscribed

@k-takata commented on this pull request.


In src/os_win32.c:

> @@ -303,6 +303,8 @@ is_ambiwidth_event(
 		&& ir->Event.KeyEvent.wRepeatCount == 1
 		&& ir->Event.KeyEvent.wVirtualKeyCode == 0x12
 		&& ir->Event.KeyEvent.wVirtualScanCode == 0x38
+		&& ir->Event.KeyEvent.UChar == 0
+		&& ir->Event.KeyEvent.AChar == 0

uChar.UnicodeChar and uChar.AsciiChar are a union.
Checking both is redundant.

Nobuhiro Takasaki

unread,
Jan 30, 2020, 1:47:10 AM1/30/20
to vim/vim, Subscribed

@ntak commented on this pull request.


In src/os_win32.c:

> @@ -303,6 +303,8 @@ is_ambiwidth_event(
 		&& ir->Event.KeyEvent.wRepeatCount == 1
 		&& ir->Event.KeyEvent.wVirtualKeyCode == 0x12
 		&& ir->Event.KeyEvent.wVirtualScanCode == 0x38
+		&& ir->Event.KeyEvent.UChar == 0
+		&& ir->Event.KeyEvent.AChar == 0

ReadConsoleInput "W" () Is the result of the UChar, so we only need to look at it ...

Nobuhiro Takasaki

unread,
Jan 30, 2020, 1:50:03 AM1/30/20
to vim/vim, Push

@ntak pushed 1 commit.


You are receiving this because you are subscribed to this thread.

Nobuhiro Takasaki

unread,
Mar 10, 2020, 2:17:50 AM3/10/20
to vim/vim, Subscribed

WindowsTerminal 1.0 release has been decided in April, what about this patch?
This patch may be required for a successful Vim experience with WT.

K.Takata

unread,
Mar 10, 2020, 2:22:51 AM3/10/20
to vim/vim, Subscribed

The title of this PR still includes "WIP". Is this ready?

Nobuhiro Takasaki

unread,
Mar 10, 2020, 2:37:14 AM3/10/20
to vim/vim, Subscribed

Oh. Fixed.

Bram Moolenaar

unread,
Mar 10, 2020, 2:16:20 PM3/10/20
to vim/vim, Subscribed

Have some people tried out this patch?

Nobuhiro Takasaki

unread,
Mar 10, 2020, 2:46:21 PM3/10/20
to vim/vim, Subscribed

I have not asked someone to test.
Presented here, 40 days later, I actually used every day, no problems.

K.Takata

unread,
Mar 11, 2020, 4:06:04 AM3/11/20
to vim/vim, Subscribed

@k-takata commented on this pull request.


In src/os_win32.c:

> @@ -7406,6 +7467,18 @@ vtp_sgr_bulks(
     vtp_printf((char *)buf);
 }
 
+    static void
+wt_init(void)
+{
+    wt_working = (mch_getenv("WT_SESSION") != NULL);

Is this the only way to detect Windows Terminal?
Is there any other way like termresponse?

Nobuhiro Takasaki

unread,
Mar 11, 2020, 10:04:08 AM3/11/20
to vim/vim, Subscribed

@ntak commented on this pull request.


In src/os_win32.c:

> @@ -7406,6 +7467,18 @@ vtp_sgr_bulks(
     vtp_printf((char *)buf);
 }
 
+    static void
+wt_init(void)
+{
+    wt_working = (mch_getenv("WT_SESSION") != NULL);

The method is official.
The original URA does not appear immediately. I have a memory in Microsoft's issues.
termresponse (CSI DA) returns the same result as ConPTY.

K.Takata

unread,
Mar 11, 2020, 11:35:04 AM3/11/20
to vim/vim, Subscribed

@k-takata commented on this pull request.


In src/os_win32.c:

> @@ -7406,6 +7467,18 @@ vtp_sgr_bulks(
     vtp_printf((char *)buf);
 }
 
+    static void
+wt_init(void)
+{
+    wt_working = (mch_getenv("WT_SESSION") != NULL);

termresponse (CSI DA) returns the same result as ConPTY.

Ah, Windows Terminal handles Win32 console programs via ConPTY, so it makes sense.

Dustin L. Howett (MSFT)

unread,
Mar 24, 2020, 2:53:53 PM3/24/20
to vim/vim, Subscribed

Speaking as the owner of the windows console and terminal infrastructure:

Please do not make this specific to Windows Terminal.
Terminal uses the same VT parser as the windows console host, which has worked since Windows 10.0.10240.0.
Any changes made to vim to use proper VT sequences on Windows should be dependent on the presence of ENABLE_VIRTUAL_TERMINAL_PROCESSING, not the presence of Windows Terminal.

This needs to be part of a larger change to Vim to use the console VT features that have existed since early 2015.

Dustin L. Howett (MSFT)

unread,
Mar 24, 2020, 2:54:43 PM3/24/20
to vim/vim, Subscribed

The correct implementation is to completely remove VTP code reduction in versions of Windows after 10240.

Nobuhiro Takasaki

unread,
Mar 24, 2020, 3:48:38 PM3/24/20
to vim/vim, Subscribed

I need to detect Windows Terminal.

There are multiple bugs only for ConhostV2-ConPTY with the same Device Attributes. I have to respond.
Conhost, ConhostV2, and Windows Terminal treat Ambiguous width characters completely differently. I have to respond.
ENABLE_VIRTUAL_TERMINAL_PROCESSING is untrusted and unimplemented in cmd.exe compatible terminals. I have to respond.

Finally, I have no authority, but there is a reason.

Dustin L. Howett (MSFT)

unread,
Mar 24, 2020, 3:52:36 PM3/24/20
to vim/vim, Subscribed

ENABLE_VIRTUAL_TERMINAL_PROCESSING is untrusted and unimplemented in cmd.exe compatible terminals. I have to respond.

ENABLE_VIRTUAL_TERMINAL_PROCESSING is implemented on all console hosts (cmd.exe compatible and incompatible) since Windows 10 10240.

Any bugs in ConPTY's behavior will apply to all conpty consumers, not just Windows Terminal. Detecting a single terminal emulator by the presence of one of its environment variables will make Vim fail to work properly on other terminal emulators.

The only way we can make ConPTY work for everyone is if we're made aware of these bugs.

Terminals on non-Windows platforms treat ambiguous width characters as narrow. That's just how they work. Because there is no way for an application to know what font the terminal is using (which would change the character width), there is no way for the terminal to handle Ambiguous characters in any special way. They have to be narrow, because narrow works over SSH and Telnet.

I would love to know all the issues so we can fix them for you 😄

Bram Moolenaar

unread,
Mar 24, 2020, 4:44:08 PM3/24/20
to vim/vim, Subscribed


Dustin L. Howett wrote:

Thanks for your input!

Keep in mind that for Vim we support systems back to Windows XP. So we
need to work around bugs in old systems. Usually that's by detecting
the version and doing something different depending on that.


> > ENABLE_VIRTUAL_TERMINAL_PROCESSING is untrusted and unimplemented in cmd.exe compatible terminals. I have to respond.
>
> ENABLE_VIRTUAL_TERMINAL_PROCESSING is implemented on all console hosts
> (cmd.exe compatible and incompatible) since Windows 10 10240.
>
> Any bugs in ConPTY's behavior will apply to _all_ conpty consumers,

> not just Windows Terminal. Detecting a single terminal emulator by the
> presence of one of its environment variables will make Vim fail to
> work properly on other terminal emulators.
>
> The only way we can make ConPTY work for everyone is if we're made
> aware of these bugs.

The problems with Asian characters were reported months ago. Perhaps
not in the right place?


> Terminals on non-Windows platforms treat ambiguous width characters as
> narrow. That's just how they work. Because there is no way for an
> application to know what _font_ the terminal is using (which would

> change the character width), there is no way for the terminal to
> handle Ambiguous characters in any special way. They have to be
> narrow, because narrow works over SSH and Telnet.

Different terminals handle ambiguous width characters in different ways.
Often using a configuration flag, thus detecting the terminal type
doesn't help. Vim has a runtime detection for xterm-like terminals
(outputing an ambiguous character and requesting the cursor position).

The choice of how ambiguous width characters are displayed is up to the
user, that's why they were marked as ambiguous by the Unicode working
group. I'm sure they would have used a known width if that had been
possible.

> I would love to know all the issues so we can fix them for you :smile:

We indeed hope we can make it work properly for as many users as we can.

--
[clop clop]
ARTHUR: Old woman!
DENNIS: Man!
ARTHUR: Man, sorry. What knight lives in that castle over there?
DENNIS: I'm thirty seven.
ARTHUR: What?
DENNIS: I'm thirty seven -- I'm not old!
The Quest for the Holy Grail (Monty Python)

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

Dustin L. Howett (MSFT)

unread,
Mar 24, 2020, 4:46:42 PM3/24/20
to vim/vim, Subscribed

@brammool ah, thanks! I'll give our backlog a quick scrub. If there's prior art for terminal emulators supporting configurable ambiguous character widths, I'm happy for us to conform! :)

Nobuhiro Takasaki

unread,
Mar 24, 2020, 5:15:50 PM3/24/20
to vim/vim, Subscribed

Ambiguity is narrow, justice in English-speaking countries. Wide, required for Asia. So ambiguous, do you plan to practice wide-width display?

Nobuhiro Takasaki

unread,
May 1, 2020, 12:37:59 AM5/1/20
to vim/vim, Push

@ntak pushed 1 commit.

  • 85fd8db Merge branch 'master' into wt and remove right edge redraw


You are receiving this because you are subscribed to this thread.

Mariusz-W

unread,
May 20, 2020, 7:48:50 AM5/20/20
to vim/vim, Subscribed

Can I join in the discussion? Yesterday, I installed Windows Terminal and was delighted to see that it natively supported a very broad section of UTF-8 via Cascadia font. My joy was, unfortunately, shortl ived when I discovered that the vim binary (version 8.1), that used in the CMD console, had serious issues with the keymaps that use dead keys. I upgraded to 8.2, the same problem persists, though it manifests itself slightly differently. On the left is what I enter, on the right is what appears in vim in Windows Terminal (CMD, or Powershell) :
₂ Îñ
² Îåñ
¹ Îí}
₁ Îí}

Note that entering ¹ and ₁ generates the same sequence. Many dead key inputs produce the same sequence Îñ .

Not all dead key combinations produce such aberrations, but the majority of them do.

The same vim binary in CMD console — the keyboard input produces expected results.
Outside vim, in Windows terminal — the keyboard input produces expected results.
The Cygwin binary in Windows Terminal — the keyboard input produces expected results.

Only the Windows vim binary in Windows terminal exhibits the described abnormal behavior.
_vimrc.zip

Christian Brabandt

unread,
Nov 1, 2024, 5:55:22 PM11/1/24
to vim/vim, Subscribed

Closed #5546.


Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/5546/issue_event/15075181271@github.com>

Christian Brabandt

unread,
Nov 1, 2024, 5:55:23 PM11/1/24
to vim/vim, Subscribed

It seems this got stalled and the original user no longer being active. So let me close this.


Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you are subscribed to this thread.Message ID: <vim/vim/pull/5546/c2452642134@github.com>

Reply all
Reply to author
Forward
0 new messages