[vim/vim] Console Vim on Windows 11 sometimes fails to properly restore the original console screen contents (Issue #15775)
William Bresler
Steps to reproduce
In order to make this easily reproducible I suggest grabbing Vim source code and extracting it into a directory of one's choice. Then ...
C:\home\build> cd vim-9.1.0748\src
C:\home\build\vim-9.1.0748\src> cls
C:\home\build\vim-9.1.0748\src> type popupmenu.c
C:\home\build\vim-9.1.0748\src> vim -R popupmenu.c
From within Vim, to simulate editing activity, hit Ctrl-F 3 times (or PgDn). Then type :q<Enter> to exit Vim. Instead of restoring the the original screen contents with a new prompt right after the command line invoking Vim, the new prompt is in the middle of text much further back in the scrollback region of the console screen. Here are some screen shots of what happens.
- Initial screen after
typeing the file and entering the Vim command.
001-Initial-screen-with-vim-command.png (view on web)
- Vim screen after paging down in the file.
002-vim-screen-after-paging.png (view on web)
- The screen after exiting Vim.
003-after-exiting-vim.png (view on web)
Expected behaviour
Here is a screen shot of the expected result:
006-vim-with-correct-result.png (view on web)
I obtained it by using an earlier version of Vim.
Version of Vim
9.1.748
Environment
Windows 11 Home 23H2 (OS build 22631.4169)
The problem occurs with both the legacy console, implemented by conhost.exe, and the new console provided by Windows Terminal, implemented by openconsole.exe. I am using cmd.exe as the shell in both cases.
Since the problem ends up being dependent on the use of Virtual Terminal Processing (VTP) in the Windows console, this probably only affects Windows 10 and Windows 11. There are various checks in the code to determine whether the host system has a useable version of VTP. Since I do not have access to earlier versions of Windows anymore, I was not able to investigate what happens on Windows 7, 8, 8.1, and 10 prior to the update that included VTP for the first time.
I note that this is a rather new issue. Vim used to do the right thing until about two months ago.
Since the "Logs and stack traces" section does not allow Markdown, here is a screen shot of the logfile as seen from with Vim:
004-vim-with-ch_log.png (view on web)
And, lastly, here is a screen shot of the minimum change to correct this issue and the diff as an attachment:
005-git-diff-with-patches.png (view on web)
Logs and stack traces
I usually update my copy of Vim once a week. I sort of ignored the issue at first. After all, the "fix" was to type `cls` and keep going. When the issue was not corrected by later patches, I decided to investigate. Using `git bisect` and building each candidate seemed like the longer way around so I just grabbed a three-month old binary release from `<https://github.com/vim/vim-win32-installer/releases>`, unzipped it, and tried it. I then bisected the releases until I found the most recent one that worked. That was release V9.1.0663. The next release, conveniently, was v9.1.0664 and it did not work. So, patch 664 is the culprit, and it happens to be directly related to restoring the main console screen buffer. Well, patch 664 appears quite simple, but I figured I needed to figure out why the change broke the restoration of the main screen buffer. To that end, I added a bunch of `ch_log()` calls related to termcap mode, saving the screen buffer, restoring the buffer, etc. Invoking Vim with the `--log` option, I repeated the steps above. I have edited the logfile to show just the relevant parts that I added. Here it is: ==== start log session Mon Sep 30 15:24:19 2024 ==== 0.005889 : Entering mch_init_c() 0.006027 : mch_init_c() - calling SaveConsoleBuffer(g_cbOrig) 0.006064 : Entering SaveConsoleBuffer() 0.006117 : SaveConsoleBuffer() - cursor at (0, 1842) 0.006175 : SaveConsoleBuffer() - use_alternate_screen_buffer = 1 0.006203 : Exiting SaveConsoleBuffer() 0.008647 : mch_init_c() - switching to alternate screen buffer via VTP 0.008954 : Exiting mch_init_c() 0.009603 : Entering stoptermcap() 0.009641 : Exiting stoptermcap() - termcap_active = 0 0.078791 : Entering starttermcap() 0.078836 : mch_write() - ESC|S - calling termcap_mode_start() 0.078854 : Entering termcap_mode_start() 0.078867 : termcap_mode_start() - calling SaveConsoleBuffer(g_cbNonTermcap) 0.078882 : Entering SaveConsoleBuffer() 0.078917 : SaveConsoleBuffer() - cursor at (0, 24) 0.078939 : SaveConsoleBuffer() - use_alternate_screen_buffer = 1 0.078953 : Exiting SaveConsoleBuffer() 0.079206 : Exiting termcap_mode_start() 0.079227 : raw terminal output: "�|S" 0.079240 : Exiting starttermcap() 3.088597 : raw key input: ":" 3.088844 : SafeState: reset: key typed 3.090497 : raw terminal output: "�|v�|7m�|7f�|0b�|25;70H:�|1;9H" 3.091358 : SafeState: Start triggering 3.093047 : raw terminal output: "�|25;70H�|K�|25;1H:�|V" 3.093174 : looking for messages on channels 3.093251 : SafeState: back to waiting, triggering SafeStateAgain 3.264635 : raw key input: "q" 3.264801 : SafeState: reset: key typed 3.265094 : SafeState: Start triggering 3.265736 : raw terminal output: "q" 3.265862 : looking for messages on channels 3.265941 : SafeState: back to waiting, triggering SafeStateAgain 5.320008 : raw key input: " " 5.320082 : SafeState: reset: key typed 5.320173 : raw terminal output: " " 5.320332 : Exiting... 5.337471 : Entering mch_exit_c() 5.337522 : Entering stoptermcap() 5.337633 : raw terminal output: "�|v" 5.337669 : mch_write() - ESC|E - calling termcap_mode_end() 5.337692 : Entering termcap_mode_end() 5.337712 : termcap_mode_end() - calling SaveConsoleBuffer(g_cbTermcap) 5.337735 : Entering SaveConsoleBuffer() 5.337789 : SaveConsoleBuffer() - cursor at (0, 24) 5.337823 : SaveConsoleBuffer() - use_alternate_screen_buffer = 1 5.337846 : Exiting SaveConsoleBuffer() 5.337919 : termcap_mode_end() - calling RestoreConsoleBuffer(g_cbOrig, 1) 5.337954 : Entering RestoreConsoleBuffer() 5.337975 : Exiting RestoreConsoleBuffer() - use_alternate_screen_buffer = 1 5.338027 : termcap_mode_end() - SetConsoleCursorPosition(0, 1842) 5.338092 : termcap_mode_end() - SetConsoleCursorPosition() failed - error = 87 5.338176 : Exiting termcap_mode_end() 5.338277 : raw terminal output: "�|E�|V" 5.338311 : Exiting stoptermcap() 5.338331 : mch_exit_c() - switching back to original screen buffer via VTP 5.340214 : Exiting mch_exit_c() and leaving VIM The key is that RestoreConsoleBuffer() doesn't actually restore anything since use_alternate_screen_buffer = 1. Back in termcap_mode_end(), the code now tries to restore the cursor position from the main screen buffer, which, being 3000 lines, has a large scrollback region. However, the alternate screen buffer is always exactly the size of the console window, with no scrollback region. This causes the SetConsoleCursorPosition() call to fail. Error 87 is ERROR_INVALID_PARAMETER. (I note that the current Vim code never checks the return of SetConsoleCursorPosition(), I added that in along with the ch_log() calls.) So, it seems that it is essential to restore the code from prior to patch 664 to switch back to the main screen buffer before restoring the cursor position. The observed behavior is simply that when ":q<Enter>" is typed, the cursor is moved to the last line of the alternate buffer. When the vtp_print() call to restore the main screen buffer is finally reached in mch_exit_c() it switches back but now the cursor is at a row number equal to the console window height. That is why one can see the very top of the file I typed - popupmenu.c. In other scenarios, this would leave the cursor in the middle of random earlier text. To preserve the effect of patch 664 I also added a bit of code to issue the vtp_print() call to restore the main screen buffer only when termcap mode was not active, such as when invoking vim with "--cmd quit", the test case from patch 664. I note that sending the VTP sequence to switch back to the main screen buffer more than once does not cause any problems, it is an idempotent operation. However, in case that should ever change, I added the check so that it is only issued once. Also note that if the cursor position in the main screen buffer was within the first page of the buffer when vim is invoked, the bug would not manifest since the call to SetConsoleCursorPosition() would be valid within the alternate screen buffer as well. That is why the issue only appeared some of the time. Finally, I was using Visual Studio 2022 as my debugger. Since the issue depends on setting up a particular state of the console session, it seemed necessary to do that outside of Visual Studio, then launch Vim, then attach VS's debugger to the Vim process. To make that possible I added in a simple function to invoke Sleep() in a loop dependent on a variable which I could change from within VS's debugger, thus exiting the loop. It is invoked from within wmain() in os_w32exe.c. I bracketed it with an #ifdef #endif pair, and it only enters the loop if a special environment variable is set. This is a common debugging strategy when an issue is dependent on an outside context that can be difficult to construct within a debugger. If there is any interest, I am happy to make it available. Leave a reply to this issue.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
Restorer
It is possible that the situation is similar.
It has not been solved in WinNT 6.1 and beyond.
With garbage and the position of the cmd prompt.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.
William Bresler
Thanks for the reference to that earlier issue.
I agree that there are similarities with #12430. However, in this case, Vim was working as expected up through the inclusion of patch 9.1.0663. The next patch is what changed the behavior. That patch addressed the problem that Vim did not return to the main screen buffer at all if termcap mode was never entered. The simple test case of vim --cmd quit left the console session on the alternate screen buffer. The fix in patch 9.1.0664 moved the call to vtp_print() with the VTP control sequence to restore the main screen buffer from the function termcap_mode_end() to the function mch_exit_c(). This did indeed correctly handle the test case, but it sometimes broke the more general case, as I outlined above.
Discussion in #12430, and others referenced there, say the difficulty is that setting the cursor position after restoring the main screen buffer is a problem because Vim did not know the original cursor position. Vim code targeting Windows now uses a hybrid approach of continuing to call Console APIs when necessary and moving to VTP sequences whenever possible. In fact, Microsoft documentation of the Console APIs frequently contains the following recommendation:
This document describes console platform functionality that is no longer a part of
our ecosystem roadmap. We do not recommend using this content in new
products, but we will continue to support existing usages for the indefinite future.
Our preferred modern solution focuses on virtual terminal sequences for
maximum compatibility in cross-platform scenarios. You can find more information
about this design decision in our classic console vs. virtual terminal document.
So, Vim calls GetConsoleScreenBufferInfo() to obtain information such as the screen size, current text attributes, and current cursor position because there is no VTP equivalent for this functionality. In fact, since patch 9.1.0664 removed the VTP sequence to restore the main screen buffer from termcap_mode_end() the subsequent call to SetConsoleCursorPosition() using the position obtained by the original call to GetConsoleScreenBufferInfo() results in an error if the original cursor position is beyond the console's screen height. That is because the alternate screen buffer is always the same size as the console screen size, with no scrollback region. (I note that this is also the behavior of xterm and console emulators that are compatible with xterm.) Remember that any given console only has one cursor position, regardless of which screen buffer is currently mapped to the console window. Thus, the failure of the SetConsoleCursorPosition() call leaves the cursor where it was in the alternate screen buffer where Vim is being hosted. The later call to vtp_print() in mch_exit_c()` restoring the main screen buffer then displays the top of the scrollback region with the cursor at the bottom of the window. That is how the cursor ends up in the middle of random text.
The minimal patch I supplied restores the original vtp_print() call in termcap_mode_end() and adds a simple check to issue the same vtp_print() call in mch_exit_c() only if termcap mode was not yet active. I believe this corrects the cursor position problem while retaining the correction supplied by patch 9.1.0664.
Naturally, comments, suggestions, or corrections are most welcome.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.
Christian Brabandt
Closed #15775 as completed via ded5913.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.