I reported this potential bug as https://groups.google.com/g/vim_dev/c/Kt1BPSgvx-s a few weeks ago. I'm reposting it here in the hopes this is the right place :)
I've had a weird bug for about a year now,
where these four keyboard shortcuts would stop working eventually:
nnoremap <C-J> <C-W><C-J>
nnoremap <C-K> <C-W><C-K>
nnoremap <C-L> <C-W><C-L>
nnoremap <C-H> <C-W><C-H>
In particular, they would revert to their original function,
almost as if they were never mapped in my vimrc.
For example, <C-L> would revert to "clear", and <C-H> to "backspace"
The only way to resolve this issue would be to restart Vim.
With the help of @myitcv, who also ran into this bug,
we were able to narrow it down: it happens after we press <C-;>.
That is, control-semicolon.
Note that some terminals do not seem to emit escape codes for <C-;>.
For example, for xterm, one needs this extra setting in .Xresources:
XTerm*vt100.modifyOtherKeys: 1
I have been able to reproduce this with xterm 369 and vim 8.2.3441.
Both with only the options listed above, and no extra plugins.
With a feature like "modifyOtherKeys" disabled, the bug disappears,
as it seems like xterm just sends ";" when pressing <C-;>.
For clarity, here's what I do to reproduce it with two sample files:
I expected <C-;> to not break my configured shortcuts.
I don't really mind what the behavior of that shortcut by default is, otherwise.
I assume it should do nothing, given that my vimrc does not map it.
I've spent hours investigating this bug, and this is as far as I've got.
I'm hoping someone in this mailing list can shed some light into it.
My best guess is this is a bug in how Vim handles ANSI escape codes.
Note that if I "xterm -e dash" and press <C-;>,
I get the following escape code: ^[[27;5;59~
Here are other scenarios I've ruled out:
Terminal emulator bug; it also reproduces with my emulator of choice,
https://codeberg.org/dnkl/foot/, which implements "modifyOtherKeys".
Issue with my vimrc; reproduced with just those four lines.
Issue with vim running inside bash; happens with "xterm -e vim" too.
System shortcuts; Paul and myself can reproduce on different setups.
Any help appreciated. Happy to try patches or capture debug logs.
Linux 5.14.12, running Arch Linux, with Sway (wayland)
8.2.3441
No response
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub.
Triage notifications on the go with GitHub Mobile for iOS or Android.
![]()
I'm seeing the same thing in iTerm2 in CSI u mode.
Good to hear I'm not alone :) It took me nearly a year to pinpoint what triggered my shortcuts to stop working, so I wonder how many others are affected but do not have the right keywords to find this issue.
I've been hitting this issue as well since I've switched to the foot terminal.
Another key combo that triggers this issue for me is Ctrl+escape.
I'm working around this in foot by using the following configuration:
[key-bindings] noop=Control+semicolon Control+Escape
@brammool in insert mode, Ctrl-semicolon does indeed just insert a semicolon for me. In normal mode it's when that key combination then breaks all other shortcuts.
I did capture the escape sequence via xterm and dash above, for context:
Note that if I "xterm -e dash" and press <C-;>, I get the following escape code:
^[[27;5;59~
Note that you need a terminal that supports these escape sequences, like xterm with XTerm*vt100.modifyOtherKeys: 1, or foot.
The only way to resolve this issue would be to restart Vim.
When the issue occurs, does it help if you run a :! command like :!ls (or even an empty one)?
Any help appreciated. Happy to try patches or capture debug logs.
No idea whether it will give any useful data, but you can get a log with :help ch_logfile():
vim --cmd "call ch_logfile('/tmp/logfile', 'w')"
And see this comment.
the ch_logfile should contain the raw keycode that vim received.
When the issue occurs, does it help if you run a
:!command like:!ls(or even an empty one)?
I tried :!ls, and my other Ctrl shortcuts remain broken.
No idea whether it will give any useful data, but you can get a log with
:help ch_logfile():
I started vim with your command, and simply hit Ctrl-; and then :q<Enter>. The relevant part of the log file seems to be:
0.178136 : SafeState: back to waiting, triggering SafeStateAgain
0.278287 : looking for messages on channels
0.278304 : SafeState: back to waiting, triggering SafeStateAgain
3.196372 : raw key input: "^[[27;5;59~"
3.196515 : SafeState: reset: key typed
3.196645 : SafeState: Start triggering
3.200290 : looking for messages on channels
3.200309 : SafeState: back to waiting, triggering SafeStateAgain
3.701399 : SafeState: reset: key typed
3.701592 : SafeState: Start triggering
3.704909 : looking for messages on channels
3.704928 : SafeState: back to waiting, triggering SafeStateAgain
4.052235 : raw key input: ":"
4.052304 : SafeState: reset: key typed
4.055658 : SafeState: Start triggering
4.055778 : looking for messages on channels
4.055796 : SafeState: back to waiting, triggering SafeStateAgain
4.332246 : raw key input: "q"
4.332324 : SafeState: reset: key typed
4.332353 : SafeState: Start triggering
4.332391 : looking for messages on channels
4.332403 : SafeState: back to waiting, triggering SafeStateAgain
4.524517 : raw key input: "^M"
4.524587 : SafeState: reset: key typed
4.524643 : Exiting...
4.526603 SEND on 0(in): '[53,["shutdown"]]
That makes sense, as ^[[27;5;59~ is exactly the same escape code I mentioned earlier.
it happens after we press <C-;>
How do you press ;? Do you need to press another modifier (like shift) to produce it?
I can't reproduce, which makes me think that the issue depends on something outside Vim. Your issue reminds me of mine. In yours, the mappings stop working as soon as you press a CTRL chord. In mine, it's the opposite; they only start working after I press a CTRL chord. I was told it could be related to an 'im*' option.
For me it just produces a semicolon.
If I press CTRL-V ;, then I get a semicolon. But if I press CTRL-SHIFT-V ;, then I get ^[[27;5;59~ (just like the OP).
How do you press
;? Do you need to press another modifier (like shift) to produce it?
It literally is just Control-Semicolon. On the US keyboard layout, the semicolon can be pressed directly.
I can't reproduce, which makes me think that the issue depends on something outside Vim.
Did you double check that you're on a terminal emulator that actually emits these escape codes? You can verify by running dash, too, which simply prints the escape sequence when I hit it. Again, this is reproducible on xterm with the option I laid out above and otherwise vanilla xterm and vim, so I'd be very surprised if others cannot reproduce.
Did you double check that you're on a terminal emulator that actually emits these escape codes?
Yes, I checked multiple times now – in xterm – as well as several weeks ago when you posted the issue. No Vim config, and empty .Xresources.
You can verify by running dash, too, which simply prints the escape sequence when I hit it.
You mean, pressing CTRL-; right in the dash(1) shell? For me, this just produces ;.
Again, this is reproducible on xterm with the option I laid out above and otherwise vanilla xterm and vim, so I'd be very surprised if others cannot reproduce.
Well, be surprised then, because I can't reproduce. But I'm not denying the existence of your issue. I'm saying there is something else involved. If you look at my issue (the one I linked earlier), nobody confirmed it, not even the devs. Same thing: there must exist some missing setting which I would need to find for everybody to reproduce, but I haven't been able to find what it is, which is why I'm interested in your issue; if it gets fixed, maybe mine will be too.
and empty .Xresources.
And adding this doesn't make any difference:
XTerm*vt100.modifyOtherKeys: 1
In fact, I don't even need it for the modifyOtherKeys feature to work properly.
That's interesting, thanks for double checking. I'll remain quiet for now, as I don't think I have any information to add :)
Here is a gif:
It runs this shell command:
vim -Nu NONE -S <(cat <<'EOF'
nnoremap <C-J> <C-W><C-J>
nnoremap <C-K> <C-W><C-K>
nnoremap <C-L> <C-W><C-L>
nnoremap <C-H> <C-W><C-H>
nnoremap <C-;> <Cmd>echomsg 'C-; was hit'<CR>
vsplit
windo split
EOF
)
Notice that when I press CTRL-SHIFT-V then ; in insert mode, I get ^[[27;5;59~. And notice that in normal mode, when I press CTRL-;, the message C-; was hit is printed on the command-line. Both of these facts show that the modifyOtherKeys feature is enabled. In particular:
A known side effect is that in Insert mode the raw escape sequence is inserted
after the CTRL-V key. This can be used to check whether modifyOtherKeys is
enabled: In Insert mode type CTRL-SHIFT-V CTRL-V, if you get one byte then
modifyOtherKeys is off, if you get <1b>27;5;118~ then it is on.
Finally, notice that the cursor correctly jumps from a viewport to another when I press CTRL-H, CTRL-J, CTRL-K, CTRL-L, even after having pressed CTRL-;. This shows that all the mappings keep working.
It's just a wild guess, but I suspect that the issue is somehow related to some software component included in the desktop environment. I'm using XFCE 4.14.2. Here are the lines in the System section of $ inxi --full --extra 3 --filter:
System: Kernel: 5.11.0-38-generic x86_64 bits: 64 compiler: N/A Desktop: Xfce 4.14.2 tk: Gtk 3.24.13
info: xfce4-panel wm: xfwm4 dm: LightDM 1.30.0 Distro: Ubuntu 20.04.3 LTS (Focal Fossa)
For another data point, I use iTerm2 with CSI u mode enabled. The terminal emits ␛[59;5u for Ctrl-;, which is correct per the CSI u encoding. I can confirm this by running cat and observing that it prints ^[[59;5u in response to the key press.
Like Bram above, I also just see a semicolon in Vim when I press Ctrl-V Ctrl-; in insert mode. If Vim is really just receiving a semicolon, then is there a state change in the terminal which causes it to emit the lone semicolon instead of the full CSI u sequence?
I also just see a semicolon in Vim when I press Ctrl-V Ctrl-; in insert mode
Just to remove any ambiguity, when you wrote Ctrl-V, did you mean CTRL and v, or did you mean CTRL, SHIFT and v?
Just to remove any ambiguity, when you wrote Ctrl-V, did you mean CTRL and v, or did you mean CTRL and SHIFT and v?
If you want to see the full CSI sequence, you need to press CTRL, and SHIFT, and v at the same time. The SHIFT modifier is necessary.
If you want to see the full CSI sequence, you need to press CTRL, and SHIFT, and v at the same time. The SHIFT modifier is necessary.
Which reminds me that CTRL-SHIFT-v is bound in some terminals to paste the contents of the system clipboard. If you have such a keybinding, you need to find a way to temporarily disable it. Then, in insert mode, press CTRL and SHIFT and v at the same time, then release the 3 keys, then press CTRL and ; at the same time.
Apologies for the ambiguity. I meant Ctrl with lowercase v. But the end result in Vim is the same when I try Ctrl with uppercase V (Ctrl-Shift-v): both insert a lone semicolon into the buffer.
With cat, I can see that iTerm2 emits ␛[86;5u for Ctrl-Shift-v.
I'm not sure what is going on. When I use the channel log as suggested above, I also see that the raw key code is ^[[27;5;59~. But Vim handles that as expected, since no mapping is defined it gets converted back to a regular semicolon. I don't see how this code can cause further CTRL keys to stop working.
I started vim with your command, and simply hit Ctrl-; and then :q. The relevant part of the log file seems to be:
Could you do this (with your mappings, and while logging the channel activity):
$ vim +vs +vs)Ctrl-lCtrl-;Ctrl-lIIUC, the first Ctrl-l should produce the sequence ^[[27;5;108~, and focus the right viewport.
Then, the Ctrl-; should produce the sequence ^[[27;5;59~, and break your custom mappings.
I would like to see what the second Ctrl-l produces, and check whether there is any unexpected sequence between the first working Ctrl-l and the second broken one.
I would like to see what the second Ctrl-l produces, and check whether there is any unexpected sequence between the first working Ctrl-l and the second broken one.
I mean, another log (for Ctrl-l + Ctrl-; + Ctrl-l) would be nice.
I also tried using xterm 366 and vim 8.2.3458 with gui xfce4. Cannot reproduce it.
FWIW just to confirm @mvdan's mention of me above, I'm seeing exactly this issue.
I can reproduce in foot, but at the moment not in XTerm, despite having verified both terminal send exactly the same bytes! Maybe this is timing related? It would explain why some people see it, while others don't.
Furthermore, this isn't specific to C-;; it seems any \E[27;m;n~ escape triggers this.
FWIW: neovim does not have this issue.
I can reproduce in foot, but at the moment not in XTerm
Were the tests performed in the exact same environment?
I think xterm can only work in Xorg, while foot only in Wayland. A different display server matters, because it runs in-between the keyboard and Vim; Wayland might handle keypresses in very different ways compared to Xorg.
Obviously not, since, as you say, XTerm is X11 only, and foot is Wayland only.
However, vim doesn't get the keyboard input directly from the display server; the terminal emulator acts as a proxy in between. I.e. it doesn't matter that Wayland and X11 handles input differently, as long as the terminal emulator translates the key presses to the same escape sequences.
And that's what I meant with having verified XTerm and foot sends exactly the same bytes; when I press C-;, followed by C-h, the terminal application receives the exact same byte sequences.
This can be verified with sudo showkey -a. But, to be sure, I also wrote a small C program, whose sole purpose is to read input from the TTY and print whatever it receives:
#include <stdio.h> #include <sys/stat.h> #include <fcntl.h> #include <unistd.h> #include <assert.h> #include <termio.h> #include <poll.h> int main(int argc, const char *const *argv) { int fd = open("/dev/tty", O_RDONLY); assert(fd >= 0); struct termio old_term; if (ioctl(fd, TCGETA, &old_term) == -1) { printf("ioctl get failed.\n"); return 1; } struct termio new_term = old_term; new_term.c_lflag &= ~ICANON; new_term.c_cc[VMIN] = 0; new_term.c_cc[VTIME] = 0; if (ioctl(fd, TCSETA, &new_term) == -1) { printf("ioctl set failed.\n"); return 1; } while (1) { struct pollfd fds[] = {{.fd = fd, .events = POLLIN}}; int r = poll(fds, sizeof(fds) / sizeof(fds[0]), -1); if (r < 0) break; char buf[256]; ssize_t n = read(fd, buf, sizeof(buf)); if (n < 0) break; printf("got %zd bytes:\n ", n); for (size_t i = 0; i < (size_t)n; i++) printf("0x%02x ", buf[i]); printf("\n"); } close(fd); return 0; }
The output is the same on XTerm and foot:
^[[27;6;59~got 10 bytes:
0x1b 0x5b 0x32 0x37 0x3b 0x36 0x3b 0x35 0x39 0x7e
^Hgot 1 bytes:
0x08
Finally, given that other people have seen this issue on XTerm, I don't think this is an X11 vs. Wayland issue.
Obviously not, since, as you say, XTerm is X11 only, and foot is Wayland only.
OK, I wasn't sure. I was wondering whether there existed some utility which could let foot work in Xorg.
And that's what I meant with having verified XTerm and foot sends exactly the same bytes; when I press C-;, followed by C-h, the terminal application receives the exact same byte sequences.
I was more thinking about the timing with which the sequences are sent, rather than the sequences themselves.
Finally, given that other people have seen this issue on XTerm, I don't think this is an X11 vs. Wayland issue.
Well, here is how the OP described their system:
Linux 5.14.12, running Arch Linux, with Sway (wayland)
It mentions Wayland. Again, I don't use the latter, but I thought maybe there existed some utility which could allow xterm to work there.
xterm would run with xwayland, so that's X running under Wayland. I don't think wayland would be a factor to reproduce the issue, but perhaps it is. I'll try to repro without wayland.
I was more thinking about the timing with which the sequences are sent, rather than the sequences themselves.
Ah, gotcha. Yeah, I was thinking along those lines as well. My small test utility didn't catch any timing related differences; in all cases, the full sequence was read in a single read(). Still, XTerm is much slower than foot, and it it possible that the timing does differ in a more complex environment, like vim.
It mentions Wayland. Again, I don't use the latter, but I thought maybe there existed some utility which could allow xterm to work there.
Yes, X11 applications run on Wayland via XWayland. It is a normal X server, but using Wayland as backend instead of interfacing directly with the GPU. As such, XWayland presents regular X11 input to XTerm, but since XWayland is a Wayland client itself, it is affected by Wayland input handling.
I can reproduce this issue 100% of the time using XTerm(366) + X11 on Ubuntu 21.10:
$ vi --version
VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Oct 18 2021 21:10:17)
Included patches: 1-3535
Compiled by myitcv@myitcv
Huge version with GTK2 GUI. Features included (+) or not (-):
+acl +file_in_path +mouse_urxvt -tag_any_white
+arabic +find_in_path +mouse_xterm -tcl
+autocmd +float +multi_byte +termguicolors
+autochdir +folding +multi_lang +terminal
-autoservername -footer -mzscheme +terminfo
+balloon_eval +fork() -netbeans_intg +termresponse
+balloon_eval_term +gettext +num64 +textobjects
+browse -hangul_input +packages +textprop
++builtin_terms +iconv +path_extra +timers
+byte_offset +insert_expand -perl +title
+channel +ipv6 +persistent_undo +toolbar
+cindent +job +popupwin +user_commands
+clientserver +jumplist +postscript +vartabs
+clipboard +keymap +printer +vertsplit
+cmdline_compl +lambda +profile +virtualedit
+cmdline_hist +langmap -python +visual
+cmdline_info +libcall +python3 +visualextra
+comments +linebreak +quickfix +viminfo
+conceal +lispindent +reltime +vreplace
+cryptv +listcmds +rightleft +wildignore
+cscope +localmap -ruby +wildmenu
+cursorbind +lua +scrollbind +windows
+cursorshape +menu +signs +writebackup
+dialog_con_gui +mksession +smartindent +X11
+diff +modify_fname -sodium -xfontset
+digraphs +mouse -sound +xim
+dnd +mouseshape +spell +xpm
-ebcdic +mouse_dec +startuptime +xsmp_interact
+emacs_tags -mouse_gpm +statusline +xterm_clipboard
+eval -mouse_jsbterm -sun_workshop -xterm_save
+ex_extra +mouse_netterm +syntax
+extra_search +mouse_sgr +tag_binary
-farsi -mouse_sysmouse -tag_old_static
system vimrc file: "$VIM/vimrc"
user vimrc file: "$HOME/.vimrc"
2nd user vimrc file: "~/.vim/vimrc"
user exrc file: "$HOME/.exrc"
system gvimrc file: "$VIM/gvimrc"
user gvimrc file: "$HOME/.gvimrc"
2nd user gvimrc file: "~/.vim/gvimrc"
defaults file: "$VIMRUNTIME/defaults.vim"
system menu file: "$VIMRUNTIME/menu.vim"
fall-back for $VIM: "/home/myitcv/usr/vim/share/vim"
Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -DFEAT_GUI_GTK -pthread -I/usr/include/gtk-2.0 -I/usr/lib/x86_64-linux-gnu/gtk-2.0/include -I/usr/include/pango-1.0 -I/usr/include/atk-1.0 -I/usr/include/gdk-pixbuf-2.0 -I/usr/include/libpng16 -I/usr/include/x86_64-linux-gnu -I/usr/include/pango-1.0 -I/usr/include/harfbuzz -I/usr/include/pango-1.0 -I/usr/include/libmount -I/usr/include/blkid -I/usr/include/fribidi -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/harfbuzz -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/uuid -I/usr/include/freetype2 -I/usr/include/libpng16 -g -O2 -D_REENTRANT -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1
Linking: gcc -L/usr/local/lib -Wl,--as-needed -o vim -lgtk-x11-2.0 -lgdk-x11-2.0 -lpangocairo-1.0 -latk-1.0 -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lpangoft2-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lharfbuzz -lfontconfig -lfreetype -lSM -lICE -lXpm -lXt -lX11 -lXdmcp -lSM -lICE -lm -ltinfo -L/usr/lib/x86_64-linux-gnu -lluajit-5.1 -L/usr/lib/python3.9/config-3.9-x86_64-linux-gnu -lpython3.9 -lcrypt -ldl -lm -lm
so to whoever can reproduce it, can you please also show an example log like mentioned by @lacygoill here?
vim --cmd "call ch_logfile('/tmp/logfile', 'w')" +vs +vsfollowed by C-l, C-, and C-l. The last C-l does not work.
0.025311 : looking for messages on channels
0.025314 : SafeState: back to waiting, triggering SafeStateAgain
1.230769 : raw key input: "^L"
1.230824 : SafeState: reset: ins_typebuf()
1.231459 : SafeState: Start triggering
1.231613 : looking for messages on channels
1.231624 : SafeState: back to waiting, triggering SafeStateAgain
2.193147 : raw key input: "^[[27;5;44~"
2.193233 : SafeState: reset: key typed
2.193256 : SafeState: Start triggering
2.193296 : looking for messages on channels
2.193302 : SafeState: back to waiting, triggering SafeStateAgain
2.896333 : raw key input: "^L"
2.896391 : SafeState: reset: key typed
2.896414 : SafeState: Start triggering
2.897398 : looking for messages on channels
2.897409 : SafeState: back to waiting, triggering SafeStateAgain
4.757575 : raw key input: ":"
4.757633 : SafeState: reset: key typed
4.757701 : SafeState: Start triggering
4.757735 : looking for messages on channels
4.757741 : SafeState: back to waiting, triggering SafeStateAgain
4.858782 : raw key input: "q"
4.858833 : SafeState: reset: key typed
4.858863 : SafeState: Start triggering
4.858888 : looking for messages on channels
4.858893 : SafeState: back to waiting, triggering SafeStateAgain
5.028651 : raw key input: "a"
Full log: vim.log
I think I've narrowed down what "causes" this, or rather I've identified a config change which moves me from being able to reproduce this 100% of the time to not being able to reproduce it at all. Here is the config that reproduces the problem:
let &t_TI = "" let &t_TE = "" call ch_logfile("log.txt") nnoremap <C-h> <C-w>h nnoremap <C-j> <C-w>j nnoremap <C-k> <C-w>k nnoremap <C-l> <C-w>l
Vim launched via:
$ vi +vsp +vsp main.go
Key sequence post launch: Ctrl-l, Ctrl-;, Ctrl-l
I can no longer reproduce the problem if the first two lines above are commented out:
" let &t_TI = "" " let &t_TE = "" call ch_logfile("log.txt") nnoremap <C-h> <C-w>h nnoremap <C-j> <C-w>j nnoremap <C-k> <C-w>k nnoremap <C-l> <C-w>l
Here is another wild guess: maybe, on some systems, for some reason, pressing a chord such as CTRL-; alters some Vim terminal option which breaks the modifyOtherKeys feature.
To rule out this possibility, here is what you can do:
:call execute('set termcap')->split('\n')->writefile('/tmp/termcap.log')
Run the previous Ex command before the issue starts, then make a copy of the log (/tmp/termcap.log):
$ cp /tmp/termcap.log /tmp/termcap.good.log
Run the Ex command again, but this time after the issue has started.
Finally, diff the 2 logs to see whether any setting has changed before/after pressing the problematic chord:
$ vimdiff /tmp/termcap.log /tmp/termcap.good.log
I can no longer reproduce the problem if the first two lines above are commented out:
That's because those lines disable the modifyOtherKeys feature entirely. You can't map <C-;> without the latter.
Finally, diff the 2 logs to see whether any setting has changed before/after pressing the problematic chord:
No differences at all
No differences at all
Try again but replace set termcap with set! all:
:call execute('set! all')->split('\n')->writefile('/tmp/options.log')
The only remaining explanation I can see is that when the modifyOtherKeys feature was developed, it was only tested in a certain environment, and unknowingly makes some assumptions which can break on some other environments. FWIW, you can get a lot of information about your system with:
$ inxi --full --filter --extra 3
so it works with: XTerm*vt100.modifyOtherKeys: 2 ? Perhaps @ThomasDickey can shed some light?
More questions:
For those of you who can reproduce, once the issue starts, what's the output of :verbose nnoremap <C-L> (replace <C-L> with whatever lhs you use in one of your mappings)? Does the mapping still exist?
What's the output of this Ex command:
:echo maparg('<C-L>', 'n', 0, 1)
Do you see any difference before vs after pressing <C-;>?
Do you see any difference in the mappings table before vs after pressing <C-;>?
You can see the latter with :map.
You can get get logs and compare them like this:
# before the issue starts
:call execute('map')->split('\n')->writefile('/tmp/map.log')
$ cp /tmp/map/log /tmp/map.good.log
# after the issue starts
:call execute('map')->split('\n')->writefile('/tmp/map.log')
$ vimdiff /tmp/map/log /tmp/map.good.log
Instead of pressing <C-;>, what happens if you send the CSI sequence to the terminal via echoraw()?
:call echoraw("\<Esc>[27;5;59~")
Does it break the mappings too?
You can see the latter with
:map.
You can get get logs and compare them like this:
# before the issue starts :call execute('map')->split('\n')->writefile('/tmp/map.log') $ cp /tmp/map/log /tmp/map.good.log # after the issue starts :call execute('map')->split('\n')->writefile('/tmp/map.log') $ vimdiff /tmp/map/log /tmp/map.good.log
I forgot: prefix :map with :verbose to get more info:
:verbose map
^-----^
What's the output of this Ex command:
:echo maparg('', 'n', 0, 1)
Do you see any difference before vs after pressing <C-;>?
You can dump the output of maparg() in a buffer and make it more readable like this:
:new | put =maparg('<C-L>', 'n', 0, 1)->items()
For example, here is the output of my <C-L>
['lnum', 594]
['script', 0]
['mode', 'n']
['silent', 0]
['buffer', 0]
['noremap', 1]
['lhs', '<C-L>']
['lhsraw', '<80><fc>^DL']
['nowait', 0]
['expr', 0]
['sid', 52]
['rhs', '<Cmd>call window#navigate(''l'')<CR>']
['lhsrawalt', '^L']
For those of you who can reproduce, once the issue starts, what's the output of
:verbose nnoremap <C-L>(replace<C-L>with whatever lhs you use in one of your mappings)? Does the mapping still exist?
Yes, the mapping exists just the same as before hitting <C-;>
What's the output of this Ex command:
:echo maparg('<C-L>', 'n', 0, 1)
Do you see any difference before vs after pressing
<C-;>?
Output is:
{'lnum': 134, 'script': 0, 'mode': 'n', 'silent': 0, 'buffer': 0, 'noremap': 1, 'lhs': '<C-L>', 'lhsraw': '<80><fc>^DL', 'nowait': 0, 'expr': 0, 'sid': 1, 'rhs': '<C-w>l', 'lhsrawalt': '^L'}
No difference before vs after hitting <C-;>.
Do you see any difference in the mappings table before vs after pressing
<C-;>?
None, even with the verbose map output.
Instead of pressing
<C-;>, what happens if you send the CSI sequence to the terminal viaechoraw()?:call echoraw("\<Esc>[27;5;59~")Does it break the mappings too?
No, this does not break things like <C-;> does.
Try again but replace
set termcapwithset! all:
No differences for either of the two variants you suggested.
That's because those lines disable the modifyOtherKeys feature entirely. You can't map
<C-;>without the latter.
I'm not sure what to conclude from this. If those lines disable modifyOtherKeys, and this problem is reproducible in such an environment... to be honest I'm quite lost with most of the discussion so I'll stick to trying/reporting things as asked 😄
so does using XTerm*vt100.modifyOtherKeys: 2 make any difference here?
I'm not sure what to conclude from this. If those lines disable modifyOtherKeys, and this problem is reproducible in such an environment...
Oh, you're right. You were saying that the issue disappeared when you leave the modifyOtherKeys enabled. That makes the issue even more confusing than it is already, because that's not what the OP said in the report:
With a feature like "modifyOtherKeys" disabled, the bug disappears
so does using XTerm*vt100.modifyOtherKeys: 2 make any difference here?
This is a really good suggestion, but I think this xterm resource is a red herring which comes from a misunderstanding in the OP:
Note that some terminals do not seem to emit escape codes for <C-;>. For example, for xterm, one needs this extra setting in .Xresources:
XTerm*vt100.modifyOtherKeys: 1
It doesn't matter whether this resource is set or how. If it was relevant, it should be documented at :help modifyOtherKeys but it's not.
What matters are the Vim terminal options 't_TI' and 't_TE'. In an xterm-compatible terminal, 't_TI' is set like this:
:echo &t_TI
^[[>4;2m
When Vim sends this sequence to the terminal, the latter constructs escape sequences for keys when they are modified by SHIFT, CTRL or ALT/META. Like the ^[[27;5;59~ mentioned earlier.
As documented at man xterm (look for the pattern ^\s*modifyOtherKeys) this feature can be in 3 states:
The default is “0”:
0 disables this feature.
1 enables this feature for keys except for those with well-
known behavior, e.g., Tab, Backarrow and some special
control character cases which are built into the X11
library, e.g., Control-Space to make a NUL, or Control-3
to make an Escape character.
Except for those special cases built into the X11 library, the Shift- and Control- modifiers are treated normally. The Alt- and Meta- modifiers do not cause xterm to send escape sequences. Those modifier keys are interpreted according to other resources, e.g., the metaSendsEscape resource.
2 enables this feature for keys including the exceptions
listed. Xterm ignores the special cases built into the
X11 library. Any shifted (modified) ordinary key sends an
escape sequence. The Alt- and Meta- modifiers cause xterm
to send escape sequences.
You can control which state you want with the final digit in the sequence stored in 't_TI'. By default, it's 2:
:echo &t_TI
^[[>4;2m
^
But you can also choose 1. So, if people here want to check whether the state 1 makes a difference:
vim -Nu NONE -i NONE --cmd 'let &t_TI = "\<Esc>[>4;1m"' -S <(cat <<'EOF'
nnoremap <C-J> <C-W><C-J>
nnoremap <C-K> <C-W><C-K>
nnoremap <C-L> <C-W><C-L>
nnoremap <C-H> <C-W><C-H>
nnoremap <C-;> <Cmd>echomsg 'C-; was hit'<CR>
vsplit
windo split
EOF
)
And now, as I'm writing this command and making a test, I realize that – indeed – it does make a difference. I can reproduce the issue only when modifyOtherKeys is in the state 1.
So, I had many more questions, but since I can finally reproduce, I'll just make the tests on my machine (later), and report back here if I find anything interesting.
BTW, here is a tip which can fix your broken mappings without quitting Vim:
:execute "set t_TI=\e[>4;2m"
:!
Yes, the empty :! is necessary (or any :! command, like :!ls).
If you want to restore the state of modifyOtherKeys to 1:
:execute "set t_TI=\e[>4;2m"
:!
:execute "set t_TI=\e[>4;1m"
For the moment, I would suggest that you all use modifyOtherKeys in the state 2; unless you have good reasons to use the state 1.
:execute "set t_TI=\e[>4;2m"
:!
These command don't fix the broken mappings for me on foot.
Here's the log of me pressing Ctrl+p, followed by Ctrl+; folloed by Ctrl+p again.
@dnkl thank you so much for stepping in. You're getting to the heart of the issue much faster than I know how.
@dnkl thank you so much for stepping in. You're getting to the heart of the issue much faster than I know how.
But if the issue really comes from modifyOtherKeys=1, it's weird that you're affected, because by default – in Vim – modifyOtherKeys=2. What is the output of this Ex command in a terminal where you can reproduce the issue:
:verbose set t_TI?
Same question to anyone who can reproduce.
Also, can you confirm whether or not the previous tip work in xterm (not foot)?
Foot doesn't implement the
modidyOtherKeysescapes; it's always inmodifyOtherKeys=1mode.
So even if requesting modifyOtherKeys to be 2 as requested by Vim? Looks like we have found the problem here. Vim does not understand version 1 and foot insists on version 1 :/
Vim does not understand version 1 and foot insists on version 1 :/
Ah, you're right. There is nothing at :help modifyOtherKeys which suggests that Vim supports the feature in state 1.
Okay, we seem to have several problems here:
Regarding the former one (the foot issue): Is there a reason not to support full v2 support of modifyOtherKeys?
BTW: That sounds fairly familiar: https://groups.google.com/g/iterm2-discuss/c/H2QszJDQSu4/m/gz9lWTMAAQAJ
I pared down my .vimrc file to just:
noremap
noremap
noremap
noremapIf I start vim, then create a split (:vsp) I can navigate left and right by using ctrl-L and ctrl-H as expected. However, if I press an illegal key combination that generates a system bell (ctrl-; or ctrl-.) the ctrl-L and ctrl-H stop working (along with any other mapped ctrl key combination). However, the default vim ctrl-W command still works.
xterm: Not sure what the state is here and why v2 would not work for anybody experiencing this problem
I'm starting to think that there is no issue in xterm (unless you purposefully change 't_TI' from its default value, but you should not, right?). But the issue must affect foot and possibly terminals in macOS too (like iTerm2)?
BTW: That sounds fairly familiar: https://groups.google.com/g/iterm2-discuss/c/H2QszJDQSu4/m/gz9lWTMAAQAJ
Yes, you can experience the exact same issue in xterm if you set modifyOtherKeys=1. For example, you don't need to press <C-;>. <C-P> in command-line mode triggers the issue too.
OK, I can't help much further. I think there is definitely an issue to be fixed, but that's for the devs to decide.
Before I unsubscribe from this thread, one last note:
You're getting to the heart of the issue much faster than I know how.
We might have gotten to the "heart of the issue" much faster, and save a lot of time for the people who try to help, had you provided a reliable minimal working example right from the start.
In particular:
Note that some terminals do not seem to emit escape codes for <C-;>. For example, for xterm, one needs this extra setting in .Xresources:
XTerm*vt100.modifyOtherKeys: 1
That's not true. I gave the explanation here.
Here are other scenarios I've ruled out:
Terminal emulator bug; it also reproduces with my emulator of choice, https://codeberg.org/dnkl/foot/, which implements "modifyOtherKeys".
That's not true either. At least not with all the information I have read so far, and the tests I have performed. The issue does depend on the terminal.
Any help appreciated.
If "any help is really appreciated", try to answer questions when people ask for more info.
@lacygoill I did my best to provide a reliable way to reproduce the issue. I clearly failed, but I'm not sure you can blame that on me when there are a lot of factors here that I'm not familiar at all with. For example, I wasn't even aware that there were multiple versions of modifyOtherKeys.
And that's the reason I've stepped aside while others are actively discussing the details. If you have a vim developer and the foot developer narrowing down on the cause of this bug, and one of them can reproduce the bug, I don't think there's anything I can add to the discussion right this moment. And, since GitHub issues get hard to navigate once they get long, I think it's best to not add extra noise :)
@mvdan thanks. I am sure @lacygoill did not want to blame you. It's a complex topic with too many parts involved. So please do not worry too much. At least we know now and we are all a bit smarter :)
There is the "seenModifyOtherKeys" flag in the Vim source code, which defaults to FALSE and becomes TRUE once a key sequence with a modifier has been detected. That works as intended. I am very confused why your terminal sends a sequence with modifier one time, and then another sequence for other keys. I have no idea how Vim can detect modifyOtherKeys=1 mode. The xterm docs mentions "those with well-known behavior", but not a definition that Vim could use. Telling Vim (with an option) sounds like a very clumsy solution.
The main problem seems to be that there are so many terminal emulators that behave unfinished...
The reference to well-known behavior (aside from the specific examples mentioned in the manual page), is talking about the control-modifier applied to the top row of the keyboard. The X documentation isn't as well indexed.... but the IsControlAlias function in xterm uses that information, has a comment listing those codes.
The behavior for semicolon in xterm doesn't differ between modes 1 and 2 (see table).
Another solution is to not use the foot terminal emulator. Why do you use it anyway?
As this question wasn't addressed at anyone in particular, I'll just assume it was addressed to me, or to me and everyone else. It's imho a bit of a silly question. Why do some people like blue and others green? Why do some people prefer vim and others Emacs (in fact, the suggestion could just as well been to not use vim...). The answer is personal preferences.
But to give one, concrete, reason: foot is Wayland native. XTerm is not.
I also happen to be the developer of foot. And that's why I actually meant it when I brought up adding support for modifyOtherKeys=2. In fact, we already have a PR up: https://codeberg.org/dnkl/foot/pulls/791.
The codes need to be added to the foot termcap entries. That's how it is done for xterm (in the builtin xterm termcap).
Added too vim builtin termcap for foot? I'm asking, pushing even, since those caps are non-standard, and not present in any upstream terminfos. And foot's terminfo is upstream (as of ncurses-6.3).
In other words, unless vim adds some kind of builtin defaults for foot, users will still have to manually configure something; either manually setting t_TI/t_TE in their vimrc, or build and use a custom foot terminfo, or pretend foot is e.g. an xterm-256color.
FWIW (I'm biased, I know), I also think there's a bit too much focus on foot in particular here, while the problem is a bit more generic. Input breaks in any terminal emulator that implements modifyOtherKeys mode 1, but not 2. This may have happened a lot before, but that's only because very few terminal emulators implement any level of modifyOtherKeys. Now, whether you consider having support for mode 1 only a bug or not, is up for debate. I personally don't think it should be considered a bug, but then again, I'm obviously biased. I do think it's a bad idea to assume mode 2 is supported, when you haven't explicitly enabled it (i.e. when you haven't even emitted t_TI).
But, even if we all agree that having support for mode 1 only, is a bug - we still have the terminfo issue; vim will not automatically enable mode 2, unless it's using the builting xterm terminfo, or the user has compiled a custom terminfo.
I don't know foot, in fact, since before this issue, I never knew of it at all :) I suppose it shouldn't be a problem to add foot to the builting termcap for Vim to make this better. So as what does foot identify itself in $TERM and also for :echo v:termresponse?
The first thing to do would be to add the request-version entry. This needs to be done in the termcap/terminfo.
Then we can add code in src/term.c to recognize the version string. Note that "1;{version};0c" is used by other terminals as well, there is a risk of collision with another terminal emulator. Best is when the first number is unique, would it be possible to change now?
If the "foot" terminal can be recognized, then we could set t_TI and t_TE. But we do need a way to allow for the user to override this, I'm not sure what the best way is.
For the time being, I would suggest to tell foot users to add the settings in their vimrc. And probably only when modifyOtherKeys=2 has been implemented, as it is very unlikely that Vim is going to support modifyOtherKeys=1
—
Note that "1;{version};0c" is used by other terminals as well, there is a risk of collision with another terminal emulator. Best is when the first number is unique, would it be possible to change now?
The 1 parameter means "vt220", which other terminals probably use in their reply because that's what they're emulating. xterm uses 41, which also just means "vt410".
We have had problems before that some terminals show part of the string
XTVERSION is a CSI sequence, so it should hopefully be much less problematic than DCS sequences like e.g. this.
Note that "1;{version};0c" is used by other terminals as well, there is a risk of collision with another terminal emulator. Best is when the first number is unique, would it be possible to change now?
After some discussions, we've decided not to change foot's response to the secondary DA request.
As mentioned by @craigbarnes the 1 has meaning - it indicates the terminal is a vt220. Or in case of an emulator, that it implements vt220 functionality. Thus, pretending to be something we aren't is a bad idea. It's not unrealistic to assume that applications enable support based on the VT level indicated by the DA response. For the same reason, returning a made-up number is also a bad idea.
Support for modifyOtherKeys=2 has been merged to foot's master branch, and will be available in the next release.
Sending a request for which we don't know if there is going to be a reply at some point is also not ideal.
Is $TERM always "foot" for this terminal? Then we can just use the name plus the version that supports modifyOtherKeys=2.
Perhaps we can detect that t_TI and t_TE were not set (also not made empty) and then default to what we use for xterm.
Is $TERM always "foot" for this terminal?
Ncurses has two terminfos: "foot", and "foot-direct".
Some distributions package alternative terminfos (containing non-standard capabilities), typically named "foot-extra" and "foot-extra-direct", or similar.
As such, I'd recommend matching TERM against "foot*".
If you want strict matching, match against "foot" and "foot-direct", as we can add t_TI/t_TE to the alternative terminfos (note that these are only present on some distributions, and are typically opt-in - hence my push for adding a built-in termcap to vim)
—
What happens to the versions before modifyOtherKeys=2 was supported when sending the escape sequence to enable it?
Nothing, or would it use modifyOtherKeys=1? If nothing, then we don't need to check the reported version.
Nothing will happen; foot will ignore the escape. Note that this means foot will stay in mode 1, since this is the only mode foot has. I.e. there is no mode 0 - there are no escapes to turn off mode 1, and no foot configuration settings to change this.
Given this, I don't see any issues with skipping the version check. Sending the escape will not cause any more breakage than what we're currently seeing.
—
I too see this issue with iTerm + CSI-u mode. I don't think i ever press Ctrl-; or Ctrl-, Ctrl-. or some of the other keyboard bindings listed in some of the posts.
Nothing will happen; foot will ignore the escape. Note that this means foot will stay in mode 1, since this is the only mode foot has. I.e. there is no mode 0 - there are no escapes to turn off mode 1, and no foot configuration settings to change this. Given this, I don't see any issues with skipping the version check. Sending the escape will not cause any more breakage than what we're currently seeing.
OK, so we can check for the terminal name to start with "foot" and then set t_TE and t_TI like they are for the builtin xterm termcap. The user can change the values after setting 'term'.
…
-- GUARD #2: Wait a minute -- supposing two swallows carried it together? GUARD #1: No, they'd have to have it on a line. GUARD #2: Well, simple! They'd just use a standard creeper! GUARD #1: What, held under the dorsal guiding feathers? GUARD #2: Well, why not? The Quest for the Holy Grail (Monty Python) /// Bram Moolenaar -- @.*** -- http://www.Moolenaar.net \\ /// \\ \\ sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ /// \\ help me help AIDS victims -- http://ICCF-Holland.org ///
Please don't forget the iterm users :)
iTerm + attach to tmux session running on linux host over ssh. TERM set to xterm-256color.
Below config seems to fix the issue
let &t_TI = ""
let &t_TE = ""
if iTerm is not xterm compatible and apparently does not understand modifyOtherKeys version 2, then do not set TERM=xterm-256colors.
Experienced the same bug on foot terminal.
https://codeberg.org/dnkl/foot/issues/849
Vim version
VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Dec 02 2021 19:44:33) Patches included: 1-3582, 3602
Fixed with
let &t_TI = "\<Esc>[>4;1m"
let &t_TE = "\<Esc>[>4m"
—
Reply to this email directly, view it on GitHub.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
let &t_TI = "<Esc>[>4;1m"
That should be 2 instead of 1 right?
—
Reply to this email directly, view it on GitHub.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
Nope, 1 does the trick on my end (foot terminal).
—
Reply to this email directly, view it on GitHub.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
That should be 2 instead of 1 right?
Yes, it should. I made two replies in https://codeberg.org/dnkl/foot/issues/849, and somehow got the second one wrong.
—
Reply to this email directly, view it on GitHub.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you are subscribed to this thread.![]()
As of 63a2e36, is it now possible to solve this via e.g. set keyprotocol=foot:mok2?
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.![]()
The default value for 'keyprotocol' contains "foot:kitty". Does foot support both the modifyOtherKeys level 2 and the kitty keyboard protocol? And don't know which one would be preferred. The Kitty protocol has one relevant advantage: the Esc key is sent as an escape sequence, removing the delay for getting out of Insert mode. It also depends on how well the protocol is implemented and whether the terminfo entry contains the right values.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.![]()
Foot supports both modifyOtherKeys=2, and the kitty keyboard protocol. There are no known limitations in either implementation. I'm personally slightly biased towards the kitty protocol.
terminfo entry contains the right values.
I'm not aware of any missing entries
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.![]()
let &t_TI = "<Esc>[>4;1m"
let &t_TE = "<Esc>[>4m"
Thanks a lot, It helped me a lot
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.![]()
I suppose this can be closed then?
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.![]()
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.![]()
Closed #9014 as not planned.
—
Reply to this email directly, view it on GitHub.
You are receiving this because you are subscribed to this thread.![]()