vim.exe doesn't restore wide characters in cmd.exe

[mattn]

http://go-gyazo.appspot.com/07b65df3b2cdd94b.png

Since vim.exe uses ReadConsoleOutputA/WriteConsoleOutputA, wide characters are not remained on cmd.exe after exiting. Below is a patch to fix this.
On win9x, ReadConsoleOutputW/WriteConsoleOutputW doesn't work correctly, this patch check the OS is NT or later.

https://gist.github.com/mattn/5aed3a57d3995e31a377

- mattn

[mattn]

This patch still have a bug. I'll update this in later.

[Ken Takata]

Hi,

As mattn mentioned, currently vim.exe uses
ReadConsoleOutputA/WriteConsoleOutputA to save and restore the console buffer.
But I think using CreateConsoleScreenBuffer and SetConsoleActiveScreenBuffer
is a straight forward way. We don't need to save the console buffer by ourself
if we use these APIs, because it is a matter of an OS.

I attached patches for this.
The first patch implements this and just comments out old codes.
The second patch removes the old codes.
I separated the patches because of readability.

Actually, these patches still use ReadConsoleOutputW/WriteConsoleOutputW for
'norestorescreen'. We need to copy the last Vim screen to the original console
buffer when 'restorescreen' is not set.

Note: These patches won't work on Win9x, because I used Unicode version of
Win32 APIs directly. I don't think it matters, because no one complains that
the official 7.4 binary doesn't work on Win9x. (VC2005 is the last VC which
supports Win9x, but the official binary is built by VC2008.)

Regards,
Ken Takata

[Bram Moolenaar]

Ken Takata wrote:

> As mattn mentioned, currently vim.exe uses
> ReadConsoleOutputA/WriteConsoleOutputA to save and restore the console buffer.
> But I think using CreateConsoleScreenBuffer and SetConsoleActiveScreenBuffer
> is a straight forward way. We don't need to save the console buffer by ourself
> if we use these APIs, because it is a matter of an OS.
>
> I attached patches for this.
> The first patch implements this and just comments out old codes.
> The second patch removes the old codes.
> I separated the patches because of readability.
>
> Actually, these patches still use ReadConsoleOutputW/WriteConsoleOutputW for
> 'norestorescreen'. We need to copy the last Vim screen to the original console
> buffer when 'restorescreen' is not set.

I'll await comments for a little while before including this patch.

> Note: These patches won't work on Win9x, because I used Unicode version of
> Win32 APIs directly. I don't think it matters, because no one complains that
> the official 7.4 binary doesn't work on Win9x. (VC2005 is the last VC which
> supports Win9x, but the official binary is built by VC2008.)

I don't think we need to support Windows 95 and 98 anymore. I don't
have a computer anymore that runs it (and thus can't play some old
games...).

Windows XP is also deprecated, but it's still used in many places.
Thus Windows XP is the oldest we need to support.


--
Shit makes the flowers grow and that's beautiful

/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///

[mattn]

I tested this too. Looks good to me.

[Ken Takata]

Hi,

I found that after applying this patch, vim.exe doesn't work on winpty on mintty.
(https://github.com/rprichard/winpty)
Maybe winpty doesn't support CreateConsoleScreenBuffer / SetConsoleActiveScreenBuffer.
However, people who want to use Vim on mintty should use cygwin-vim or msys-vim,
not Win32-vim on winpty, IMO. So it isn't a problem.
I hope winpty will support CreateConsoleScreenBuffer / SetConsoleActiveScreenBuffer.

Regards,
Ken Takata

[Ken Takata]

Hi mattn and all,

Now I found a solution for this problem!
Mattn's patch restores the console buffer by calling WriteConsoleOutputW once,
but it causes a problem because buffer positions are based on ANSI character
count. We need to call ReadConsoleOutputW several times to save the console
buffer, so it's natural to call WriteConsoleOutputW the same times, and this
does solve the problem.

Please check the following patch:
https://bitbucket.org/k_takata/vim-ktakata-mq/src/378b825971eb7de7692cad5072001c41cab54b29/restore-unicode-chars.patch

Of course, this patch doesn't use CreateConsoleScreenBuffer, so this doesn't
cause the troubles caused by the patch 7.4.851.

Regards,
Ken Takata

[mattn]

I tested this. Thanks fixing this problem that bother me. You fix works fine. Bram, could you please incldue this?

- mattn

[Ken Takata]

Hi Bram,

Thank you for adding this item to the todo list:

> Patch for restoring wide characters in the console buffer.
> (Ken Takata, 2016 Jun 7)

BTW, this patch fixes the same problem as the following todo item:

> Patch to fix that wide characters do not work properly after exiting.
> (Yasuhiro Matsumoto, 2015 May 24) Better patch to come.

So these two items can be merged.

Regards
Ken Takata

[mattn]

Hi, Bram.

I hope to merge this. When using Visual Studio Code with terminal feature, vim always breaks the terminal like below.

https://camo.githubusercontent.com/b4af7147f7689b5540ca0efec9d50cdef38c3867/687474703a2f2f676f2d6779617a6f2e61707073706f742e636f6d2f353965643863623539626138313865392e706e67

[Ken Takata]

Hi Bram,

The same problem occurs when running vim.exe in (g)vim.exe using the terminal
feature. Multibyte characters becomes broken after exiting vim.exe.

If my patch is applied, the problem doesn't occur.

> https://bitbucket.org/k_takata/vim-ktakata-mq/src/378b825971eb7de7692cad5072001c41cab54b29/restore-unicode-chars.patch

I really hope this patch to be merged.

Regards,
Ken Takata

[mattn]

Bram, Any problem stil is there?

[Bram Moolenaar]

It's on the list of patches to include.

--
BEDEVERE: How do you know so much about swallows?
ARTHUR: Well you have to know these things when you're a king, you know.
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

[Ken Takata]

Hi Bram,

2017/8/10 Thu 5:57:25 UTC+9 Bram Moolenaar wrote:
> Yasuhiro Matsumoto wrote:
>
> > On Wednesday, August 2, 2017 at 7:43:41 PM UTC+9, Ken Takata wrote:
> > > Hi Bram,
> > >
> > > 2016/10/18 Tue 17:16:14 UTC+9 mattn wrote:
> > > > Hi, Bram.
> > > >
> > > > I hope to merge this. When using Visual Studio Code with terminal feature, vim always breaks the terminal like below.
> > > >
> > > >
> > > > https://camo.githubusercontent.com/b4af7147f7689b5540ca0efec9d50cdef38c3867/687474703a2f2f676f2d6779617a6f2e61707073706f742e636f6d2f353965643863623539626138313865392e706e67
> > >
> > > The same problem occurs when running vim.exe in (g)vim.exe using the terminal
> > > feature. Multibyte characters becomes broken after exiting vim.exe.
> > >
> > > If my patch is applied, the problem doesn't occur.
> > >
> > > > https://bitbucket.org/k_takata/vim-ktakata-mq/src/378b825971eb7de7692cad5072001c41cab54b29/restore-unicode-chars.patch
> > >
> > > I really hope this patch to be merged.
> > >
> > > Regards,
> > > Ken Takata
> >
> > Bram, Any problem stil is there?
>
> It's on the list of patches to include.

I think this patch is quite important.
The priority should be higher.

Regards,
Ken Takata