Dear Bram
graywh
libtermkey so that LeoNerd can fix terminal input and Gvim and
everyone will be happy. Thanks.
Erik Falor
Bram Moolenaar
Graywh wrote:
Send me a patch and I'll look into it.
If you want a more useful response, use your real name and include more
information about what you actually want, what system, etc.
--
hundred-and-one symptoms of being an internet addict:
195. Your cat has its own home page.
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
Paul LeoNerd Evans
> Graywh wrote:
>
> > Please fix vim's input queue mechanism with key info structures a.la
> > libtermkey so that LeoNerd can fix terminal input and Gvim and
> > everyone will be happy. Thanks.
>
> Send me a patch and I'll look into it.
>
> If you want a more useful response, use your real name and include more
> information about what you actually want, what system, etc.
Hi Bram,
I would like this. For all systems. Everywhere.
This is a large undertaking. This isn't a "send me a patch" request.
I am talking about ripping out the byte-queue input system and replacing
it with a structured keypress queue. The lot. Everything. Remove the
"stuck in the 1970s all the world is a DEC-style glass virtual terminal"
input queue, and replace it with one that has structures; remembers
modifiers, etc.. With a structure we can properly represent "Ctrl-I" as
being the letter I with a control key, totally separate from the Tab.
Having done this, we can feed in the input events from a terminal in
EXACTLY the same way it currently works. We can take events out to match
them up to map/etc.. much neater. We can feed events in from GTK or
similar GUIs, without having to go near legacy 1970s terminal
technology, meaning that GTK (et.al) users can map Tab, Ctrl-I and
Ctrl-Shift-I, all independently of each other.
Having done this, I can then quite easily send you a patch to use
libtermkey if that is wanted, because then it will be able to recognise
Tab, Ctrl-I and Ctrl-Shift-I even over terminal from a
properly-configured xterm. But that is PURELY an extension mechanism. My
primary request is to fix the underlying queue mechanism.
This is an issue that has been dragging on for years now - 7 or 8 years
by my count. 7 or 8 years I have been sitting in #vim on Freenode; in
all that time almost every week we'll get someone or other asking how
they can map some key or other, that we have to keep explaining "No,
sorry, you can't. Not even in GTK.".
I am happy to sit and discuss it with you for potentially hours if
necessary. What I cannot do is "send a patch" - it's not a small simple
few-lines fix like that.
I think it is about time we ended this unfortunate situation, and
properly fixed the real underlying mechanism.
Yours, with my real name,
--
Paul "LeoNerd" Evans
leo...@leonerd.org.uk
ICQ# 4135350 | Registered Linux# 179460
http://www.leonerd.org.uk/
Ingo Karkat
> On Fri, Feb 04, 2011 at 07:14:29PM +0100, Bram Moolenaar wrote:
>>
>> Send me a patch and I'll look into it.
>>
>
I would love to have this, too, if only because after so many happy years with
Vim (a heartfelt thanks to Bram and all developers!), I'm running out of keys
for my custom mappings :-)
Though I'm too ignorant of the internals (and can therefore only offer to do
some testing), I agree that this probably cannot be done in a single patch. But,
with the Mercurial repository, it should be easy to create a feature branch that
allows devs to collaborate and test this, and only have this merged in after
it's done (hopefully in time for Vim 7.4 / 8.0?).
-- regards, ingo
Bram Moolenaar
Paul Evans wrote:
> I would like this. For all systems. Everywhere.
>
> This is a large undertaking. This isn't a "send me a patch" request.
>
> I am talking about ripping out the byte-queue input system and replacing
> it with a structured keypress queue. The lot. Everything. Remove the
> "stuck in the 1970s all the world is a DEC-style glass virtual terminal"
> input queue, and replace it with one that has structures; remembers
> modifiers, etc.. With a structure we can properly represent "Ctrl-I" as
> being the letter I with a control key, totally separate from the Tab.
>
> Having done this, we can feed in the input events from a terminal in
> EXACTLY the same way it currently works. We can take events out to match
> them up to map/etc.. much neater. We can feed events in from GTK or
> similar GUIs, without having to go near legacy 1970s terminal
> technology, meaning that GTK (et.al) users can map Tab, Ctrl-I and
> Ctrl-Shift-I, all independently of each other.
You can just keep the existing queue and make this work. It actually
already works for most keys, using a modifier sequence. Using
structures only makes it bigger.
Main problem is that this won't be backwards compatible, currently
CTRL-I is the same as Tab. I don't know how to make this work without
breaking backwards compatibilty.
> Having done this, I can then quite easily send you a patch to use
> libtermkey if that is wanted, because then it will be able to recognise
> Tab, Ctrl-I and Ctrl-Shift-I even over terminal from a
> properly-configured xterm. But that is PURELY an extension mechanism. My
> primary request is to fix the underlying queue mechanism.
>
> This is an issue that has been dragging on for years now - 7 or 8 years
> by my count. 7 or 8 years I have been sitting in #vim on Freenode; in
> all that time almost every week we'll get someone or other asking how
> they can map some key or other, that we have to keep explaining "No,
> sorry, you can't. Not even in GTK.".
Actually, in the GUI many key combinations do work. Not all though.
> I am happy to sit and discuss it with you for potentially hours if
> necessary. What I cannot do is "send a patch" - it's not a small simple
> few-lines fix like that.
>
> I think it is about time we ended this unfortunate situation, and
> properly fixed the real underlying mechanism.
>
> Yours, with my real name,
>
> --
> Paul "LeoNerd" Evans
--
A cow comes flying over the battlements, lowing aggressively. The cow
lands on GALAHAD'S PAGE, squashing him completely.
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD
Davertron
On Feb 17, 11:48 am, Paul LeoNerd Evans <leon...@leonerd.org.uk>
wrote:
> ICQ# 4135350 | Registered Linux# 179460http://www.leonerd.org.uk/
>
> signature.asc
> < 1KViewDownload
Paul LeoNerd Evans
> You can just keep the existing queue and make this work. It actually
> already works for most keys, using a modifier sequence. Using
> structures only makes it bigger.
OK, well which ever way works.. I'm not fussy about the internals of
implementation. So long as it is capable of knowing that Enter is not
Ctrl-M is not Ctrl-Shift-M, I am happy.
> Main problem is that this won't be backwards compatible, currently
> CTRL-I is the same as Tab. I don't know how to make this work without
> breaking backwards compatibilty.
:set oldterminalmode
:map <Ctrl-I> <Tab>
:map <Ctrl-M> <CR>
For that matter, make it default to this 1970s-like legacy behaviour and
if people want the ability to map newly-recognised keys,
:set newterminalmode
> Actually, in the GUI many key combinations do work. Not all though.
Well, then lets make them all work. Please?
Seriously - I am not the only person here. There is a constant stream of
users on #vim, every week you can guarantee someone will want to map
Ctrl-I or Shift-Space or Shift-Enter or some other key combination that
seems perfectly logical to any outside observer, to want to map. Every
week we have to tell them "no". I'm getting tired of it, when there
isn't any technical reason why it can't be done, only "it hasn't been
done this way so far".
So can we please have a plan? If you care deeply about retaining
1970s-like semantics, I'd be happy for a compromise solution. Some
boolean setting that defaults to a value whereby these aliases do occur,
where Ctrl-I and Ctrl-Shift-I -are- synonymous with Tab. But implemented
in a way where I or whichever knowledgable user can say "Yes vim, I know
that Ctrl-I used to mean Tab in the 1970s but I would like an extra
Ctrl key now and 'I' looks a good choice - let me have it". And then I
could toggle setting, and map it. Yes, I'm aware that not -every-
terminal can represent that. Yes I'm aware if I ever go use a real DEC
VT220 glass teletype I've now lost the ability to type it. But that
should be -my- problem - right now vim pointlessly restricts a whole
selection of possible key mappings from GTK or Mac OS X or Win32 or
proper xterms or all sorts of other places, simply for the
backward-looking legacy of retaining compatibility with what the
behaviour used to be in a time before many vim users were even born.
I think it's about time we opened this up a bit, and allowed users to
map the keypresses they want to.
Paul LeoNerd Evans
> already works for most keys, using a modifier sequence. Using
> structures only makes it bigger.
Oh, and I'm not sure I follow your logic here.
Yes, it would make it bigger. By maybe bytes. Consider
struct keypress {
uint16_t flags; /* Unicode or special */
uint16_t modifiers; /* A bitmask */
uint32_t codepoint;
};
that's 8 bytes. As compared a single byte keypress that's 7 bytes
bigger. Less so, compared to e.g. a 3 or 4 byte modifier prefixed
sequence. Not a massive growth here. Lets be really pessimistic and say
we're using 7 bytes extra. Lets now say we're really memory-constrained,
and can only afford a single extra kilobyte of memory. That's still 146
keypresses of queue we can afford. Is vim's queue that big? Can we
really afford no more than a single kilobyte?
Ofcourse, this was a simplistic structure. If we care more about saving
memory at a slight CPU overhead we could
struct keypress {
unsigned int is_special : 1;
unsigned int modifiers : 10;
unsigned int codepoint : 21;
};
that's 4 bytes, to store the same information, noting that Unicode is
only a 21bit code space, a Unicode/special flag, and storage space for
10 modifiers.
4 bytes.
Those variable-length-encoded prefixed keypresses are getting on for
that long anyway. Plus now we're in a fixed-length encoding, so perhaps
we've saved CPU time and associated code space to store the program used
to encode and decode them.
I'm not sure an argument of memory consumption can be made against using
this encoding scheme.
Plus, it's only the queue of incoming keypresses - that queue isn't
going to stay very big for very long.
Ben Schmidt
> going to stay very big for very long.
It's not just the input queue that's in question here, it's everywhere
in Vim where keypresses are represented. For instance, the right hand
sides of mappings are not primarily characters, but lists of keypresses.
They need the same amount of expressive power to work properly.
When macros are recorded, registers, which usually are primarily lists
of characters, are used to store keypresses. Likewise, for feedkeys() to
work, its input, a string, needs to be able to represent keypresses.
And I'm sure there are plenty more subtleties.
The bottom line, though, is that changing to a struct-based approach
could make the job absolutely huge, requiring reworking and/or
redesigning how maps, registers, etc. all work. And it might not even be
possible since, e.g. registers need to be able to do both characters and
keys. It would be much simpler to extend the current approach which uses
an 'escape mechanism' for representing special keys. Just hit qq and
type some arrow keys, q to end recording, and then inspect register q
and you'll see what's going on here. Vim uses a byte <80> followed by
two more bytes to represent special keys.
IMHO, what needs to be done is for this to be tidied up and made
consistent and documented so that these codes can be more readily
generated, interpreted, and even viewed and understood by users (e.g.
when I do :registers, it would be nice to see <S-Up> rather than
<80><fd>^D, when appropriate). Included in this is a clear specification
of what control characters mean, e.g. does ^I (<09>) mean tab or
control-I? And which of those keys can or does use the escape mechanism?
Then the behaviour of mappings needs to be defined--if there is a
mapping for ^I (<09>) and I push tab, will it be triggered? If in a
terminal which can't distinguish control-I and tab, and a ^I is
received, should the mapping for Tab or control-I be triggered? If
there's a mapping for ^I as well as Tab, which has precedence? All these
kinds of questions need clear answers, and sensible specifications and
design need to address them, avoid ambiguity, and take care to require
as little as possible work for users, plugin authors, etc. to update
their code and mappings.
Then the input code needs to be reworked in all the GUIs and in the
terminal handling to generate the appropriate internal codes,
consistently across all the different GUIs, etc., in line with that
specification/design.
Ben.
Adrien "Axioplase" Piérard
> in Vim where keypresses are represented.
> And I'm sure there are plenty more subtleties.
What I see here, if done, is a potential new major version of vim.
Such a new version may trade off backward compatibility for up-to-date
technology and/or design constraints.
Since the changes may be huge, I concur with the suggestion of
creating a branch, and having discussions *and* documentation on its
evolution.
You raised some very interesting questions, and I hope that other
people who know about the internals give their point of view too.
As far as memory usage is concerned, I am now very egoistic and
believe that I have enough RAM to run vim. If I don't, then I'll use
vi, or an old version of vim.
I used to collect and use old hardware, but I'm now getting my job
done and bringing home the bacon thanks to vim on machines with more
memory that I could think of not so long ago…
Anyway,
Cheers,
P!
--
Français, English, 日本語, 한국어
Stephen Lee
small suggestion:
Is it possible to put a parameter in .vimrc like:
set keymap=byte-queue, structured, etc..
(default is byte-queue)
so that both sets of mapping could be kept...
sorry if I see the problem too surface.
> /// sponsor Vim, vote for features --http://www.Vim.org/sponsor/\\\
> \\\ an exciting new programming language --http://www.Zimbu.org ///
Stephen Lee
the key mapping definition files like dvorak.vim, 101-keyboard.vim,
etc. and let people set in .vimrc?
It may be more flexible than just new or old term mode...
Philippe Vaucher
hard to make everyone happy on this subject by the mean of .vimrc
settings, like we have between vi and vim for :set compatible.
Wether we use structures or the byte 80 for special key is
implementation detail, I mean wether macros, mappings or registers
simply store a list of bytes or a list of structures shouldn't change
anything: you just need an explicit mapping between a displayed
keypress and a byte or a struct (<S-Up> can map internally to \x80\xFD
\xAA or { 1, MOD_SHIFT, KEY_UP } and it shouldn't matter).
Structures just look more sane as they're more generic/expressive and
accurate about the problem, they also are easier to expand on.
Philippe
Paul LeoNerd Evans
> FWIW, you have my vote for making this happen. I think it's not that
> hard to make everyone happy on this subject by the mean of .vimrc
> settings, like we have between vi and vim for :set compatible.
:set compatible does suggest the possibility for more similar options,
ways to optionally extend the featureset beyond its existing capability,
without upsetting people who rely on the current behaviour, limits and
all.
> Wether we use structures or the byte 80 for special key is
> implementation detail, I mean wether macros, mappings or registers
> simply store a list of bytes or a list of structures shouldn't change
> anything: you just need an explicit mapping between a displayed
> keypress and a byte or a struct (<S-Up> can map internally to \x80\xFD
> \xAA or { 1, MOD_SHIFT, KEY_UP } and it shouldn't matter).
>
> Structures just look more sane as they're more generic/expressive and
> accurate about the problem, they also are easier to expand on.
Another thought occurs to me: UTF-8 sequences vs. Alt keys.
I don't know if it's an accident of implementation, but I have never
been able to get them all working. Lets consider this example
é is U+00e9 in Unicode, encodes to two UTF-8 bytes C3 A9
C is U+0043 in Unicode, encodes to one UTF-8 byte 43
If vim simply uses an 8th-bit-high to indicate Alt+ prefix on simple
ASCII, then what does
C3
mean? Is it the first of two UTF-8 bytes, or is it Alt+C ?
Alternatively, perhaps we'll represent it as a two-byte sequence with an
<Escape> prefix. Now we've conflated someone typing <Escape> C from
someone typing Alt+C. Again, in the 1970s that may be what it was, but
these days it's far from clear from GUIs, that this behaviour is what's
wanted. Again, a simple 'set' option should default this:
:set noaltisescape (defaults to :set altisescape)
The reason I suggested a simple neat orthogonal structure representation
is that it can disambiguate all of the following cases in a clear
logical way; a way that can be easily fed from GUI events, and fed into
map tables or other internals.
Tab { 1, 0, KEY_TAB }
Ctrl-I { 0, MOD_CTRL, 'i' }
Ctrl-Shift-I { 0, MOD_CTRL, 'I' }
Escape C { 1, 0, KEY_ESCAPE }, { 0, 0, 'C' }
Alt+C { 0, MOD_ALT, 'C' }
é { 0, 0, 0xe9 }
Paul LeoNerd Evans
> >Plus, it's only the queue of incoming keypresses - that queue isn't
> >going to stay very big for very long.
>
> It's not just the input queue that's in question here, it's everywhere
> in Vim where keypresses are represented. For instance, the right hand
> sides of mappings are not primarily characters, but lists of keypresses.
> They need the same amount of expressive power to work properly.
Yes. And the scheme I created is no less expressive than the byte
queues. In fact it is more expressive, able to disambiguate neatly
situations that cannot normally be represented - see also my other mail
on the thread a few minutes ago.
http://groups.google.com/group/vim_dev/msg/7bb4cc9b127cddb3
> When macros are recorded, registers, which usually are primarily lists
> of characters, are used to store keypresses. Likewise, for feedkeys() to
> work, its input, a string, needs to be able to represent keypresses.
>
> And I'm sure there are plenty more subtleties.
Yes - again, my point of these structures is that they exactly _do_
represent these keypresses.
> The bottom line, though, is that changing to a struct-based approach
> could make the job absolutely huge, requiring reworking and/or
> redesigning how maps, registers, etc. all work. And it might not even be
> possible since, e.g. registers need to be able to do both characters and
> keys.
But every character -can- be represented as a key - namely, the key that
generates it. Let me again repeat my structure:
struct keypress {
unsigned int is_special : 1;
unsigned int modifiers : 10;
unsigned int codepoint : 21;
};
Any character has a Unicode codepoint (cp). That's represented by
{ 0, 0, cp }
Any keypress has a symbolic key number, taken from some arbitrary
enumeration - I care not where.
{ 1, mod, key }
Now, we can also represent those modified Unicode keys such as Ctrl-I
and Shift-Space which previously were impossible:
{ 0, MOD_CTRL, 'i' } /* Ctrl-I */
{ 0, MOD_SHIFT, ' ' } /* Shift-Space */
> It would be much simpler to extend the current approach which uses
> an 'escape mechanism' for representing special keys. Just hit qq and
> type some arrow keys, q to end recording, and then inspect register q
> and you'll see what's going on here. Vim uses a byte <80> followed by
> two more bytes to represent special keys.
>
> IMHO, what needs to be done is for this to be tidied up and made
> consistent and documented so that these codes can be more readily
> generated, interpreted, and even viewed and understood by users (e.g.
> when I do :registers, it would be nice to see <S-Up> rather than
> <80><fd>^D, when appropriate). Included in this is a clear specification
> of what control characters mean, e.g. does ^I (<09>) mean tab or
> control-I? And which of those keys can or does use the escape mechanism?
OK - well, if you feel confident that the existing prefix-escape
mechanism can completely an unambiguously represent all these possible
keystrokes, then sure, that way might result in less code change
overall.
But do make quite sure it can represent them. Specifically consider the
two tricky special-cases I suggested - repeated here again:
Tab { 1, 0, KEY_TAB }
Ctrl-I { 0, MOD_CTRL, 'i' }
Ctrl-Shift-I { 0, MOD_CTRL, 'I' }
Escape C { 1, 0, KEY_ESCAPE }, { 0, 0, 'C' }
Alt+C { 0, MOD_ALT, 'C' }
é { 0, 0, 0xe9 }
> Then the behaviour of mappings needs to be defined--if there is a
> mapping for ^I (<09>) and I push tab, will it be triggered? If in a
> terminal which can't distinguish control-I and tab, and a ^I is
> received, should the mapping for Tab or control-I be triggered? If
> there's a mapping for ^I as well as Tab, which has precedence?
I propose a pair of boolean settings, with the following defaults:
:set nomodifiedunicode
:set altisescape
Under these settings,
:map <Ctrl-I> ... vs :map <Tab> ... shall have the same effect, each
overwriting the effects of the other; last one wins.
Pressing the Tab key or Ctrl-I shall both invoke the last mapping
registered.
:map <Escape>C ... vs :map <Alt+C> ... shall behave similarly, each
overwrites the other. Typing either key sequence will invoke the
last map to be registered.
If a user decides "I want to use those extra Ctrl- keys", they can set
in their .vimrc
:set modifiedunicode
At this point,
:map <Ctrl-I> and :map <Tab> shall fill two -different-
slots of the mapping list, and typing either key will activate the
indicated mapping.
If a user further decides "I want the Alt modifier to not mean Escape
prefixing", then
:set noaltisescape
At which point,
:map <Escape>C and :map <Alt+C> shall also be different.
The only remaining ambiguity to be answered is, what happens in the
following case:
:set nomodifiedunicode
:map <Tab> ONE
:map <Ctrl-I> TWO
:set modifiedunicode
now press Tab or Ctrl-I
I don't have an easy answer for this case; no particular behaviour jumps
out at me as "obviously correct".
> All these
> kinds of questions need clear answers, and sensible specifications and
> design need to address them, avoid ambiguity, and take care to require
> as little as possible work for users, plugin authors, etc. to update
> their code and mappings.
>
> Then the input code needs to be reworked in all the GUIs and in the
> terminal handling to generate the appropriate internal codes,
> consistently across all the different GUIs, etc., in line with that
> specification/design.
I propose that the structure approach makes this simpler. It is now
clearly obvious how e.g. GTK should fill in these structures, as GDK
keypress events already have unicode / modifier bits or symbolic keys.
It's also obvious how to render such a structure as a string, possibly
wrapping <> around it, possibly prefixing C- or A- or whatever as
appropriate.
(( This sort of behaviour is already implemented by libtermkey, even
going so far as to support
char buffer[256];
termkey_snprint_key(tk, buffer, sizeof buffer, &key, TERMKEY_FORMAT_VIM);
for exactly this purpose. :) ))
Ben Schmidt
>>> going to stay very big for very long.
>>
>> It's not just the input queue that's in question here, it's everywhere
>> in Vim where keypresses are represented. For instance, the right hand
>> sides of mappings are not primarily characters, but lists of keypresses.
>> They need the same amount of expressive power to work properly.
>
> Yes. And the scheme I created is no less expressive than the byte
> queues. In fact it is more expressive, able to disambiguate neatly
> situations that cannot normally be represented - see also my other mail
> on the thread a few minutes ago.
Yeah, there was never anything lacking in your scheme in terms of
expressive power. It's just that it's not only the input queue that
would need to use it. It's a lot more.
>> When macros are recorded, registers, which usually are primarily lists
>> of characters, are used to store keypresses. Likewise, for feedkeys() to
>> work, its input, a string, needs to be able to represent keypresses.
>>
>> And I'm sure there are plenty more subtleties.
>
> Yes - again, my point of these structures is that they exactly _do_
> represent these keypresses.
Yes. But what happens when you then edit that macro by putting the
register into a buffer, changing it, and yanking it again? This is not
uncommonly done. How should the registers be stored in .viminfo? How do
you write the input to the feedkeys function as a string in vimscript?
Etc.. These are the kinds of issues I was trying to raise.
>> The bottom line, though, is that changing to a struct-based approach
>> could make the job absolutely huge, requiring reworking and/or
>> redesigning how maps, registers, etc. all work. And it might not even be
>> possible since, e.g. registers need to be able to do both characters and
>> keys.
>
> But every character -can- be represented as a key - namely, the key that
> generates it.
When there is one. Or a codepoint in some encoding. Yeah, your struct
does allow for that. The struct isn't inexpressive, it's just that it's
big, and if it needs to be used in a LOT of places (which seem to just
be increasing...input queue, mappings, registers, strings, buffers?!),
suddenly not just one thing is getting a bit bigger--a lot of things are
getting bigger--potentially a lot bigger! If every time you yank text
into a register it gets 12 times larger, that may cause some problems!
> Let me again repeat my structure:
>
> struct keypress {
> unsigned int is_special : 1;
> unsigned int modifiers : 10;
> unsigned int codepoint : 21;
> };
>
> Any character has a Unicode codepoint (cp). That's represented by
Well, a codepoint in whatever encoding you are using.
> { 0, 0, cp }
>
> Any keypress has a symbolic key number, taken from some arbitrary
> enumeration - I care not where.
>
> { 1, mod, key }
>
> Now, we can also represent those modified Unicode keys such as Ctrl-I
> and Shift-Space which previously were impossible:
>
> { 0, MOD_CTRL, 'i' } /* Ctrl-I */
> { 0, MOD_SHIFT, ' ' } /* Shift-Space */
Yeah. The benefit here is that different keyboard layouts can be
represented, e.g. keyboards which have single keys for accented
characters can represent those keys with modifiers, easily.
> OK - well, if you feel confident that the existing prefix-escape
> mechanism can completely an unambiguously represent all these possible
> keystrokes, then sure, that way might result in less code change
> overall.
I think with some careful design it could, which may well require some
reworking of how it's currently done. There is the issue, of course,
that <80> is a valid character in some encodings, too, and I don't know
if this is accounted for. It needs to be. So maybe the escape mechanism
needs to be a bit more formal (if it can be--but maybe there's no way
around this, or maybe that's a longer-term consideration). Definitely
the issues have to be bashed out. I still think this would be the
easiest and best way forward, though. But it's far from my call!
Regarding that whole meta issue that you raised earlier--I think, yes,
using an 8-bit-high representation for meta is completely out of the
question. Part of the input code should be transforming that to a proper
Vim-internal escape sequence for terminals that use it. Same for other
things. The input code should transform input into Vim's internal
representation which should be carefully designed not to be ambiguous,
etc., but which, nevertheless, is at its most fundamental, a byte stream
so it can be used in the input queue, mappings, registers, strings and
even buffers.
> But do make quite sure it can represent them. Specifically consider the
> two tricky special-cases I suggested - repeated here again:
>
> Tab { 1, 0, KEY_TAB }
> Ctrl-I { 0, MOD_CTRL, 'i' }
> Ctrl-Shift-I { 0, MOD_CTRL, 'I' }
>
> Escape C { 1, 0, KEY_ESCAPE }, { 0, 0, 'C' }
> Alt+C { 0, MOD_ALT, 'C' }
> é { 0, 0, 0xe9 }
Yes. This all has to be carefully accounted for.
Maybe the way to do it is to have printable characters represented just
as printable characters, 'default' control characters optionally just as
themselves (e.g. <09> for tab), to use when a terminal that can't
distinguish between this and Ctrl-I is used, and (whenever possible),
use <80>+flag_byte+character for control/special keys. One of the
flag_byte flags could be a 'special' flag, which means the 'character'
in the third byte would be some kind of mnemonic for the key (e.g. u for
the up arrow). When the key is not 'special' the character it produces
would be the third byte (and perhaps later for multibyte characters,
e.g. if ctrl is used with an accented character on keyboards that have a
specific key for this). How all this works depends a bit on how the
keyboard interfacing works--maybe it should just use keycodes if
characters are not readily obtainable. And maybe you need two flag bytes
if there are more than 7 modifier keys that need to be distinguished.
>> Then the behaviour of mappings needs to be defined--if there is a
>> mapping for ^I (<09>) and I push tab, will it be triggered? If in a
>> terminal which can't distinguish control-I and tab, and a ^I is
>> received, should the mapping for Tab or control-I be triggered? If
>> there's a mapping for ^I as well as Tab, which has precedence?
>
> I propose a pair of boolean settings, with the following defaults:
>
> :set nomodifiedunicode
> :set altisescape
>
> Under these settings,
>
> :map<Ctrl-I> ... vs :map<Tab> ... shall have the same effect, each
> overwriting the effects of the other; last one wins.
>
> Pressing the Tab key or Ctrl-I shall both invoke the last mapping
> registered.
>
> :map<Escape>C ... vs :map<Alt+C> ... shall behave similarly, each
> overwrites the other. Typing either key sequence will invoke the
> last map to be registered.
>
> If a user decides "I want to use those extra Ctrl- keys", they can set
> in their .vimrc
>
> :set modifiedunicode
>
> At this point,
>
> :map<Ctrl-I> and :map<Tab> shall fill two -different-
> slots of the mapping list, and typing either key will activate the
> indicated mapping.
What should Vim do in a terminal that can't distinuish between those
keys when this option is in effect? This is the same question as below,
really.
Maybe we need an annotation to the mapping, like <default> or something
that says "this mapping is acceptable to use when we're not sure if this
exact combo was pressed, but we got a code that 'could be it' in some
terminal."
> If a user further decides "I want the Alt modifier to not mean Escape
> prefixing", then
>
> :set noaltisescape
>
> At which point,
>
> :map<Escape>C and :map<Alt+C> shall also be different.
I think there might already be an option for this. Or something in
termcap? I seem to remember reading some stuff about it in the Vim
manual. I guess, though, the 'non-alt prefixing' of Vim probably means
to use 8-bit-high for meta, which we already said, is pretty useless.
I'm not sure two options are needed. Probably a single option suffices
to deal with both situations.
> The only remaining ambiguity to be answered is, what happens in the
> following case:
>
> :set nomodifiedunicode
> :map<Tab> ONE
> :map<Ctrl-I> TWO
> :set modifiedunicode
>
> now press Tab or Ctrl-I
>
> I don't have an easy answer for this case; no particular behaviour jumps
> out at me as "obviously correct".
I agree. If we go with the <default> annotation to mappings (preferably
with a better keyword), I guess that is the one that wins and takes
precedence. It may not even need to delete the other ones, so both slots
could remain active, but it would just run the 'default' one.
>> All these
>> kinds of questions need clear answers, and sensible specifications and
>> design need to address them, avoid ambiguity, and take care to require
>> as little as possible work for users, plugin authors, etc. to update
>> their code and mappings.
>>
>> Then the input code needs to be reworked in all the GUIs and in the
>> terminal handling to generate the appropriate internal codes,
>> consistently across all the different GUIs, etc., in line with that
>> specification/design.
>
> I propose that the structure approach makes this simpler. It is now
> clearly obvious how e.g. GTK should fill in these structures, as GDK
> keypress events already have unicode / modifier bits or symbolic keys.
> It's also obvious how to render such a structure as a string, possibly
> wrapping<> around it, possibly prefixing C- or A- or whatever as
> appropriate.
That's good for mappings, and I guess the \< escape in vimscript
strings, but it doesn't help for registers (which can end up in
buffers), etc..
Maybe it's a sensible 'middle point', though, for the input queue. The
GUI specific code could go into a structure like this, which generic
code can then translate into internal byte-stream representation as
appropriate, and then that byte-stream representation is the standard
throughout the rest of Vim. Basically the structure would only be used
as a mechanism to deal with the code that deals with hardware
keypresses, making it consistent between GUIs, platforms, etc..
> (( This sort of behaviour is already implemented by libtermkey, even
> going so far as to support
>
> char buffer[256];
> termkey_snprint_key(tk, buffer, sizeof buffer,&key, TERMKEY_FORMAT_VIM);
>
> for exactly this purpose. :) ))
Ben.
Philippe Vaucher
> Yes. But what happens when you then edit that macro by putting the
> register into a buffer, changing it, and yanking it again? This is not
> uncommonly done. How should the registers be stored in .viminfo? How do
> you write the input to the feedkeys function as a string in vimscript?
> Etc.. These are the kinds of issues I was trying to raise.
with some expansions e.g "<C-A><S-I><Tab>somechars<U-234>". I mean
this is simple key-as-text representation and then text-representation-
to-struct creation no?
For example, the user records a macro into register q that is then
displayed as "ihello<Tab>worlld<Esc><Left><Left>x". He then edits the
macro in a buffer, yanks it into register q, and when @q is executed
all it does is interpret the string?
We'll probably need some escape mechanism to differentiate between the
text "<Tab>" and the tab key, but this looks fairly easy to find no?
Wether it being a special byte before the <> block or some \ or
whatever.
Maybe I'm oversimplifying it but it'd help if you were more specific
about the "issues" the new design have. To me it looks like we just
need to agree on a new set of conventions by describing how the new
system would work for all the current (and future) macros/registers/
whatever scenarios.
I suggest maybe we make some kind of "new design" draft which would
answer all the issues raised (memory size/1970 support/macros
concerns) ?
Philippe
Milan Vancura
> register into a buffer, changing it, and yanking it again? This is not
> uncommonly done. How should the registers be stored in .viminfo? How do
> you write the input to the feedkeys function as a string in vimscript?
> Etc.. These are the kinds of issues I was trying to raise.
Hi.
Wouldn't be it same as now, only used more often? There is already a
possibility to write "<F4>" or "<S-Space>". The only problem is that, at least
the second, we can't press strongly enough to push it to vim :-)
But, on the other hand, you are right there still will be (and must be)
ambiguities. We can't do anything about that, in general - it's a user who must
decide how does he want to understand his keyboard.
For example: I, as Czech, have some Czech accented letters accessible via
modifier+key on my keyboard. To make the example more specific, think about
Mod5+s as "s with hook" (U0161) and redefined my keymap so Capslock key acts as
Mod5. It's up to me, and only me, if I define some vim mapping as
:map <Mod5+S> ...rhs...
- or -
:map <U0161> ...rhs...
Both do the same on my current keyboard but start to behave differently if I
change my keyboard setting in X window system, of course. If I switched to
another kind of Czech keyboard (called "typewriter one"), <U0161> appears at a
key of "number 3" and Mod5 would be on right Alt or not defined at all.
As I wrote above, we can't do anything about that, as far as I know.
Milan
--
Milan Vancura, Prague, Czech Republic, Europe
Stephen Lee
layouts instead of forcing any specific one to them.
In your case it seems you are talking about QWERTY and QWERTZ (from
wiki), so the following would be keyboard layouts in the new directory
called "Keyboard"
(General keyboard layouts)
QWERTY.vim
QWERTZ.vim
AZERTY.vim
QZERTY.vim
Dvorak.vim
Colemak.vim
JCUKEN.vim
Neo.vim
Turkish.vim
.
.
OR (Keyboard layout by Country Name)
and there would be 4 files:
QWERTYsequence.vim
QWERTZsequence.vim
QWERTYstructured.vim
QWERTZstructured.vim
OR (specific for your country)
CzechQWERTYsequence.vim
CzechQWERTZsequence.vim
CzechQWERTYstructured.vim
CzechQWERTZstructured.vim
OR simply (having advantage of adding new keyboard layouts in future
but disadvantage of difficult to find which is which when changing
them in command)
CzechLayoutA1.vim
CzechLayoutA2.vim
CzechLayoutB1.vim
CzechLayoutB2.vim
In .vimrc, there would be:
set keyboardlayout=....
Maybe a step forward to change the layout on-the-fly by the following
command (when changing keyboard setting in X window system):
:set keyboardlayout=...
I am not a programmer, but the concept is to make vim a converter and
convert keys on-the-fly:
Input ---> vim(search and map in the layout file) ---> output
The concept is inspired by the following plugins:
VimIM : Vim Input Method
ywvim : Another input method(IM) for VIM, supports all modes
and the Keyboard layout system in Windows because when inputting
Chinese we rely heavily on mapping different keys so as to generate
one Chinese character.
All we need would be desiging 2 sets of clear layout files for every
different kind of keyboard layout
Hope this help.
Stephen Lee
I just browsed through vim and found a directory called "keymap"
can we reuse and change or add different layout files in there?
Adrien "Pied" Piérard
2011/2/23 Stephen Lee <stephenl...@gmail.com>:
> Therefore I propose letting users choose their preferred keyboard
> layouts instead of forcing any specific one to them.
> In your case it seems you are talking about QWERTY and QWERTZ (from
> wiki), so the following would be keyboard layouts in the new directory
> called "Keyboard"
>
> (General keyboard layouts)
> QWERTY.vim
> ...
Hum, as a user of a home-made variant of dvorak-bépo (because my
keyboard, a kinesis, is *not* practical to use with standard bépo), I
hope it's going to be easy to set up and modify such a setting,
because you are not going to give an exhaustive list of all layouts…
Ben Fritz
On Feb 22, 8:55 pm, Adrien "Pied" Piérard <axiopl...@gmail.com> wrote:
> Hi.
>
> 2011/2/23 Stephen Lee <stephenletter...@gmail.com>:
> > Therefore I propose letting users choose their preferred keyboard
> > layouts instead of forcing any specific one to them.
> > In your case it seems you are talking about QWERTY and QWERTZ (from
> > wiki), so the following would be keyboard layouts in the new directory
> > called "Keyboard"
>
> > (General keyboard layouts)
> > QWERTY.vim
> > ...
>
> Hum, as a user of a home-made variant of dvorak-bépo (because my
> keyboard, a kinesis, is *not* practical to use with standard bépo), I
> hope it's going to be easy to set up and modify such a setting,
> because you are not going to give an exhaustive list of all layouts…
>
layouts. The proposal at hand is to replace Vim's input mechanism so
that ANY key, modified or otherwise, coming from ANY keyboard, will
work with Vim as a user would expect. Nobody is talking about forcing
specific keyboard layouts.
Benjamin R. Haskell
>> Graywh wrote:
>>
>>> Please fix vim's input queue mechanism with key info structures a.la
>>> libtermkey so that LeoNerd can fix terminal input and Gvim and
>>> everyone will be happy. Thanks.
>>
>> [...]
>
> I am talking about ripping out the byte-queue input system and replacing
> it with a structured keypress queue. [...]
>
> Having done this, I can then quite easily send you a patch to use
> libtermkey [...]
>
> Tab, Ctrl-I and Ctrl-Shift-I even over terminal from a
> properly-configured xterm. [...]
So, from other responses, it seems (to me) that the problem isn't
necessarily in the input queue, per se. "Special" keys are already
represented in some internal format (perhaps not-well-documented, since
it's internal, but still represented). (example of undocumented nature
is the K_IGNORE special key trick that stopped working when its keycode
was either changed or eliminated [1]).
Taking that as a given, I'm going to ignore that aspect of this issue.
My question is about libtermkey. I was curious enough to download it,
unpack it, `make` it, and run ./demo. And, surprisingly, given that it
seems to be the favored example, pressing <Tab>, then <Ctrl>+<i>, then
<Ctrl>+<Shift>+<i> gave me the output sequence:
<Tab>
<Tab>
<Tab>
So, I'm wondering: what does libtermkey bring to the table?
I tested under both rxvt-unicode (my preferred terminal emulator) and
uxterm (which, AFAIK, is a "properly-configured" xterm). If
"properly-configured" actually means "specially-configured", how would
using libtermkey be anything less than a huge PITA when working on
multiple systems that aren't so configured? Or is this a
chicken-and-egg problem? (Nothing is configured to send anything other
than 70's-style "<C-i>" == "<Tab>", so nothing ever will be.)
--
Best,
Ben
[1] http://groups.google.com/group/vim_use/browse_thread/thread/8c535e5cf2b35f63/319b3113716e52d5
Ben Schmidt
don't!) that is the convention on this list. I must admit, though, that
particularly after somebody else has already bottom-posted, top-posting
makes things a big mess. In future, please write your replies under
short, relevant extracts of the mail you are replying to.
> Therefore I propose letting users choose their preferred keyboard
> layouts instead of forcing any specific one to them.
Sorry, but this is not what we are discussing. Vim already has this
functionality. As you discovered, it can work through what it calls
'keymaps' (:help keymap). It also works with a number of different input
methods on different platforms, particularly when using Gvim.
> Maybe a step forward to change the layout on-the-fly by the following
> command (when changing keyboard setting in X window system):
>
> :set keyboardlayout=...
And there is even better than this already: Vim's 'iminsert' option
(:help iminsert). Of course, you can write mappings (as I have done) to
change the 'keymap' option between values, too.
> and there would be 4 files:
>
> QWERTYsequence.vim
> QWERTZsequence.vim
>
> QWERTYstructured.vim
> QWERTZstructured.vim
You are miles ahead of us, Stephen! It is indeed possible that this kind
of thing could evolve. In fact, the functionality for this part of it is
already there. What we need, though, is a way to get the structured
kepress information into vim in the first place, so that keymaps (and
more importantly, mappings) can refer to them.
Mappings are more important, because what we are discussing isn't really
about text input, but about shortcut keys. At the moment, Vim sometimes
can't tell the difference between Tab and Control-I (and a lot of other
things). And because Vim sometimes has no way of telling the difference,
at the moment, it doesn't even bother to notice the difference when it
*can* tell. We want to change that.
After we have changed that, then your idea of having multiple keyboard
layouts, some for when these keys can be differentiated, and some for
when they cannot, may become useful. I hope we won't need that idea,
though, but will be able to write one keymap file that works well in
both situations.
But that is for later.
First we need to get all the keys recognised as properly as possible.
Ben.
Ben Schmidt
>> Yes. But what happens when you then edit that macro by putting the
>> register into a buffer, changing it, and yanking it again? This is not
>> uncommonly done. How should the registers be stored in .viminfo? How do
>> you write the input to the feedkeys function as a string in vimscript?
>> Etc.. These are the kinds of issues I was trying to raise.
>
> Well, I don't see what the problem is? Just do it like today, maybe
> with some expansions e.g "<C-A><S-I><Tab>somechars<U-234>". I mean
> this is simple key-as-text representation and then text-representation-
> to-struct creation no?
>
> For example, the user records a macro into register q that is then
> displayed as "ihello<Tab>worlld<Esc><Left><Left>x". He then edits the
> macro in a buffer, yanks it into register q, and when @q is executed
> all it does is interpret the string?
OK, so you seem to be proposing changing the way the q and @ commands
work, so that instead of exposing internal key representation, they
record and play back <> notation. They would also have to represent < as
<lt> of course. The feedkeys() function would also need to be similarly
changed. It would be hard to know what to do about \<xxx> escapes in
strings (:help expr-string)--whether to make it a no-op or do the old
thing, or something else.
So this would be a backwards-incompatible change. It would not just
affect places where special keys are currently represented in internal
notation, but everywhere macros are used. Now there are currently
probably quite a lot of places where internal keys are represented, and
changing the internal representation as I suggest, may affect them; but
if the current representation is sparse enough, perhaps it can be done
in a backwards compatible way. There may be quite little impact, though,
as wherever possible, plugin authors use <> notation anyway; it is only
when they cannot do this that they resort to internal
representation--and even then, they often use \<xxx> escapes when they
can. There is no way your suggestion could be backwards compatible, I
don't think, and it could potentially affect a lot more macros (e.g. all
macros that insert xml and html, so use < and > a lot). It could break a
lot of plugins as well as home-grown code.
Now, that doesn't rule it out, and I think it is probably a way to solve
the problem. But I think it would need more careful thought than the
alternative option of changing the internal representation while keeping
it byte-stream based. I also think it would be much more work, because
it requires not just macro recording, but every problem involving keys
to be separately solved from scratch, and every one would probably be
backwards incompatible. And I think Vim and Bram have a history of
maintaining backwards compatibility (we still have :set compatible!) so
it may be that a more compatible change is preferred.
> Maybe I'm oversimplifying it but it'd help if you were more specific
> about the "issues" the new design have.
This would be easier if we actually had a solid 'new design'. We don't.
We just have a bunch of rough ideas. One of those ideas had a drafted
structure and suggestion that it be used only in the input queue, where
the memory inefficiency would not be a concern. I have pointed out that
there are many other problems to solve: right hand sides of mappings,
feedkeys(), registers for macro recordings (which indirectly affect
buffers, as registers must be able to be put and yanked), representation
in strings, and more (I just thought of the :normal command, and I'm
sure there is still more...). I pointed out that if you use the
structure to solve all these problems, memory efficiency is most
certainly a concern.
So I think I have been quite specific. I think I've been specific enough
for the purpose of this discussion.
> To me it looks like we just need to agree on a new set of conventions
> by describing how the new system would work for all the current (and
> future) macros/registers/ whatever scenarios.
Yep, this seems to sum up your approach nicely, which is basically
design and solve every key-related problem from scratch. The most
specific thing you have done is suggested a different way to using the
structure to solve at least the macro recording problem.
My response, to both that specific proposal and that general approach is
that I don't think it is a good way, because it is massively
backwards-incompatible and a lot of work.
> I suggest maybe we make some kind of "new design" draft which would
> answer all the issues raised (memory size/1970 support/macros
> concerns) ?
I suggested earlier a rough idea in response to the structure proposal
of extending/changing Vim's existing mechanism. That was also
springboarding off Bram's comment that he thought this could be done
using the existing mechanism. The existing mechanism is there for all to
see, so is already a much more well-developed design than any of the
alternatives. My rough idea of how it could be extended is in a previous
post, too. I think it would work, and be more efficient, less work, and
more compatible than the other directions we have explored.
But it's not zero work, and I'm not at the point where I can volunteer
to do work on it. Even putting together a careful detailed proposal is
probably not a contribution I can commit. I'm more than willing to
discuss and brainstorm ideas, though, and comment on any serious
proposal someone wants to make.
In the end, it's up to Bram as maintainer, and whoever's going to put
the (large amount of) time into implementing this, what goes ahead,
though.
But as a community member, I'd prefer it if it affected my code and
plugins as little as possible, and I consider it essential that Vim
doesn't become significantly less efficient because of this. And if it
were me considering an implementation, I'd certainly be looking for the
option that was the least work! YMMV.
Ben.
Paul LeoNerd Evans
> Taking that as a given, I'm going to ignore that aspect of this
> issue. My question is about libtermkey. I was curious enough to
> download it, unpack it, `make` it, and run ./demo. And,
> surprisingly, given that it seems to be the favored example,
> pressing <Tab>, then <Ctrl>+<i>, then <Ctrl>+<Shift>+<i> gave me the
> output sequence:
>
> <Tab>
> <Tab>
> <Tab>
>
> So, I'm wondering: what does libtermkey bring to the table?
It parses CSIs, the way that xterm et.al. send modified keys.
Start an old vim. Go into insert mode. Press Ctrl-Left.
xterm sends CSI 1;5D
Without a CSI parser, using only a 1970s-style terminfo-driven prefix
hunting algorithm, there is no terminfo key sequence matching this. So
vim gives up. It then sees Escape [ 1 ; 5 D. The upshot is that vim
leaves insert mode (Escape), does something unimportant with [ 1 ; then
deletes 5 lines of text (5D).
With a CSI parser such as libtermkey, this is recognised as a real CSI D
call, with the parameters 1 and 5. This is reported to the application
as Left (CSI D), with the Ctrl modifier (2nd param being 5).
The primary thing that libtermkey brings, is not blowing up and deleting
5 lines of your text just because xterm happened to send a valid CSI
sequence that isn't in the terminfo.
> I tested under both rxvt-unicode (my preferred terminal emulator)
rxvt uses its own, totally-incompatible encoding scheme for modified
keypresses. This scheme is different to any other terminal that has such
abilities (all the others follow xterm's CSI scheme). It is also
incompatible with ECMA-48 (whereas xterm extends it), inextensible
beyond Ctrl and Shift, and arbitrary (happening to pick three ASCII
characters seemingly at random, to represent the Shift, Ctrl and
Ctrl+Shift states).
> and uxterm (which, AFAIK, is a "properly-configured" xterm). If
> "properly-configured" actually means "specially-configured",
Specifically-configured means
modifyOtherKeys: 2
Except that on its own now breaks most other applications, because xterm
is too keen to use the CSI encoding scheme on "normal" keypresses, such
as Ctrl-D. xterm doesn't yet send the correct and useful sequences here,
but it is not difficult to fix its code to do so - I have been in
occasional contact with Thomas Dickey on this issue; I need to poke him
again sometime soon to check he's properly looking into it.
What is needed is a hybrid scheme, whereby the simplest key encoding is
used in a nicer way; where Ctrl-D would send the single 0x04 byte (so
kernel's pty driver can recognise "here's an EOF"), but Ctrl-Shift-D
sends CSI 68;5 u to encode it correctly.
> how
> would using libtermkey be anything less than a huge PITA when
> working on multiple systems that aren't so configured? Or is this a
> chicken-and-egg problem? (Nothing is configured to send anything
> other than 70's-style "<C-i>" == "<Tab>", so nothing ever will be.)
Rightnow, xterm will send the same byte for Tab, Ctrl-I or Ctrl-Shift-I.
Setting modifyOtherKeys to 2 will send real keypresses that it can tell,
but now everything else cannot.
Ultimately, yes. It is chicken and egg, in order for -everything- to
work in a happy shiney way. But even without terminal changes,
applications can benefit from using libtermkey now.
So to answer your question on "how would using libtermkey be anything
less than a huge PITA" - it isn't. It already recognises, via its
terminfo-driven parser, every keypress that terminfo/curses already
recognises. It cannot be worse. It is strictly better than, because of
the full CSI parser I mentioned above. Even if it cannot pass on that
information to its containing program, it can at least parse it. If
libtermkey was used by some application that has no concept of modifier
keys, then modifiers would just be ignored; i.e. that pressing Ctrl-Left
would simply be read as a Left keypress, not resulting in a beep and
deleting 5 lines of text (vim), or beeping and appending ;5D to your
shell prompt (readline, irssi, etc...).
Furthermore it is much more future-proof. Just as in 1970 nobody
considered modifier keys, right now nobody has, say, pressure-sensitive
keyboards, so nobody bothers reporting pressure information. Maybe in
another 20 years time we'll all have keyboards that can recognise a
gentle vs. hard press of a key, or multi-touch with two fingers, or
whatever. It would be quite natural to use the 3rd CSI argument, say, to
encode this.
Up CSI A
Ctrl-Up CSI 1;5 A
Forceful-Ctrl-Up CSI 1;5;3 A
A terminfo-driven string prefix parser will of course blow up here. The
code in libtermkey, right now today, will not blow up. It won't
-understand- that 3rd parameter, it cannot -report- that to its
containing application, but it will correctly parse it to ignore it -
reporting to its application the press of Ctrl-A, by ignoring the
Forceful part.
Using libtermkey today leaves terminal developers of the future the
freedom to extend the CSI parameters in whatever way they see fit,
knowing that if old applications can't make use of this new information,
they are not at least confused and inconvenienced by it.
Hope that helps,
John Little
...
Regards, John
Benjamin R. Haskell
> On Tue, Feb 22, 2011 at 11:43:39PM -0500, Benjamin R. Haskell wrote:
>>
>> [...] I'm wondering: what does libtermkey bring to the table?
>
> It parses CSIs, [...]
>
> Without a CSI parser, using only a 1970s-style terminfo-driven prefix
> hunting algorithm, there is no terminfo key sequence matching this. So
> vim gives up. [...]
>
>> I tested under both rxvt-unicode (my preferred terminal emulator)
>
> rxvt uses its own, totally-incompatible encoding scheme for modified
> keypresses. This scheme is different to any other terminal that has
> such abilities (all the others follow xterm's CSI scheme). It is also
> incompatible with ECMA-48 (whereas xterm extends it), inextensible
> beyond Ctrl and Shift, and arbitrary (happening to pick three ASCII
> characters seemingly at random, to represent the Shift, Ctrl and
> Ctrl+Shift states).
The whole situation is pretty arbitrary. E.g. ./demo's output under a
modifyOtherKeys:2 UXTerm, compressed to one row, for <Shift>+ the top
row of my typical US 105-key or variant:
<S-~> ! <S-@> # $ % <S-^> & * ( ) <S-_> + <Backspace>
Why the Shift modifier on ~, @, ^, and _, but not !, #, $, etc?
(Guessing hysterical raisins.)
Didn't realize ECMA-48 existed, though. Thanks.
>> and uxterm (which, AFAIK, is a "properly-configured" xterm). If
>> "properly-configured" actually means "specially-configured",
>
> Specifically-configured means
>
> modifyOtherKeys: 2
>
> Except that on its own now breaks most other applications, because
> xterm is too keen to use the CSI encoding scheme on "normal"
> keypresses, such as Ctrl-D. xterm doesn't yet send the correct and
> useful sequences here, but it is not difficult to fix its code to do
> so - I have been in occasional contact with Thomas Dickey on this
> issue; I need to poke him again sometime soon to check he's properly
> looking into it.
Cool. Works as advertised (i.e. currently not in anything other than
the `demo` program, but still interesting to see the ability is built
in).
> What is needed is a hybrid scheme, whereby the simplest key encoding
> is used in a nicer way; where Ctrl-D would send the single 0x04 byte
> (so kernel's pty driver can recognise "here's an EOF"), but
> Ctrl-Shift-D sends CSI 68;5 u to encode it correctly.
How do <Tab> and <Ctrl-i> fit into this scheme? What's the "simplest
key encoding"?
>> how would using libtermkey be anything less than a huge PITA when
>> working on multiple systems that aren't so configured? Or is this a
>> chicken-and-egg problem? (Nothing is configured to send anything
>> other than 70's-style "<C-i>" == "<Tab>", so nothing ever will be.)
>
> Rightnow, xterm will send the same byte for Tab, Ctrl-I or
> Ctrl-Shift-I. Setting modifyOtherKeys to 2 will send real keypresses
> that it can tell, but now everything else cannot.
>
> Ultimately, yes. It is chicken and egg, in order for -everything- to
> work in a happy shiney way. But even without terminal changes,
> applications can benefit from using libtermkey now.
>
> So to answer your question on "how would using libtermkey be anything
> less than a huge PITA" - it isn't. It already recognises, via its
> terminfo-driven parser, every keypress that terminfo/curses already
> recognises. It cannot be worse. It is strictly better than, because of
> the full CSI parser I mentioned above.
Thanks. This paragraph combined with the first section about parsing
unknown CSI sequences is plenty convincing. (trimmed rest)
--
Best,
Ben
Ben Schmidt
>
> It parses CSIs, the way that xterm et.al. send modified keys.
Of course, Vim can already do this to an extent.
:help keycodes
:help xterm-8bit
:help +termresponse
It's only keys like Control-I which are still indistingishable from keys
like Tab (and Enter, and Backspace, and I think shifted Control keys).
And perhaps that can't be fixed in the terminal now, but it can
certainly be fixed in the GUI.
And in doing so, the notation and representation for it can be tidied up
so there aren't a bunch of internal ambiguities and at least all the
remaining ambiguities are external ones that we are stuck with anyway
(and with luck will go away one day).
> rxvt uses its own, totally-incompatible encoding scheme for modified
> keypresses. This scheme is different to any other terminal that has such
> abilities (all the others follow xterm's CSI scheme). It is also
> incompatible with ECMA-48 (whereas xterm extends it), inextensible
> beyond Ctrl and Shift, and arbitrary (happening to pick three ASCII
> characters seemingly at random, to represent the Shift, Ctrl and
> Ctrl+Shift states).
And AFAIK, Vim doesn't deal with this. Does libtermkey or does it only
do CSI stuff?
Ben.
Ben Schmidt
> modifyOtherKeys:2 UXTerm, compressed to one row, for <Shift>+ the top row of my
> typical US 105-key or variant:
>
> <S-~> ! <S-@> # $ % <S-^> & * ( ) <S-_> + <Backspace>
>
> Why the Shift modifier on ~, @, ^, and _, but not !, #, $, etc? (Guessing
> hysterical raisins.)
This can't explain ~, but...
My guess is that these are keys which are low enough in ASCII that they
can be control keys. There should be 32 of them. A-Z makes 26, plus the
three above which are @ ^ and _, then there's [ ] and \, and that's it.
I guess you can test the theory and see if { } and | appear with shift
modifiers--though I guess they may not, since they actually generate
different characters. Still, would be interesting to know.
Maybe ~ by some other convention does something special with control.
IIRC, it is right down the other end of ASCII, at 126 or something.
Maybe it's equivalent to another one of the keys, if ctrl-a is the same
as ctrl-A, maybe ctrl-~ is equivalent to whatever is at
codepoint...126-32=94, which is definitely one of those symbols above...
It's strange, in my terminal, pressing ctrl-~ seems to do nothing,
whereas control with other 'non-control' characters makes a beep.
At any rate, since it's only these few keys which can be control
characters in the first place, it makes sense that these have the
special treatment, and the others don't.
Ben.
Paul LeoNerd Evans
> >rxvt uses its own, totally-incompatible encoding scheme for modified
> >keypresses. This scheme is different to any other terminal that has such
> >abilities (all the others follow xterm's CSI scheme). It is also
> >incompatible with ECMA-48 (whereas xterm extends it), inextensible
> >beyond Ctrl and Shift, and arbitrary (happening to pick three ASCII
> >characters seemingly at random, to represent the Shift, Ctrl and
> >Ctrl+Shift states).
>
> And AFAIK, Vim doesn't deal with this. Does libtermkey or does it only
> do CSI stuff?
It doesn't at the moment, but if someone wants to do the research into
what sequences it sends, and write some code in the library (similar to
driver-csi.c and driver-ti.c), I could add that in sure.
Stephen Lee
This is my first time joining a discussion, I had a hard time just to
open an account and send my suggestion. I use the default reply
function and I don't mean to upset anyone. Hopefull now this reply is
well-edited (I removed the quote).
About vim, I misunderstood that it was a keyboard problem. As Bram
said it is internally set as <CTRL-I> = <Tab> and people requesting
having structured input mechanism, all I can think of is to break
apart all internal linkages and revert to the their original settings:
I found that <CTRL-M> and <Enter> both have the same keymap code 0x0d,
and both <Tab> and <CTRL-I> gives 0x09, maybe that is why vim gives
the same output.
How about checking the Scan Codes directly?
<Enter> = 1c 9c
<L-CTRL> = 1d 9d
<M> = 32 b2
<Tab> = 0f 8f
<L-CTRL> = 1d 9d
<I> = 17 97
A shortcut table could be made to map between <Enter> and <L-CTRL>
+<M>, and load when vim starts.
So the new process would be:
detecting Scan Codes --> map between Scan Codes to alias like
"<Enter>" or "<Tab>"
Different keyboards from different countries maybe need different Scan-
Codes-to-alias mapping layouts, and therefore many layout files.
I know it is more complicated than getting the keycode conveniently
from the system, but this is all I know to my understanding.
About Paul's type:
> Tab { 1, 0, KEY_TAB }
> Ctrl-I { 0, MOD_CTRL, 'i' }
> Ctrl-Shift-I { 0, MOD_CTRL, 'I' }
changes but I really don't know how to differentiate if it is
systemwise that <Tab> and <CTRL-I> gives the same 0x09.
Also about your question:
> When there is one. Or a codepoint in some encoding. Yeah, your struct
> does allow for that. The struct isn't inexpressive, it's just that it's
> big, and if it needs to be used in a LOT of places (which seem to just
> be increasing...input queue, mappings, registers, strings, buffers?!),
> suddenly not just one thing is getting a bit bigger--a lot of things are
> getting bigger--potentially a lot bigger! If every time you yank text
> into a register it gets 12 times larger, that may cause some problems!
as to control the memory size? It has advantages of large capacity,
infinite undo, and can be called to check the complete list.
Sorry again if I gave useless suggestions.
Stephen
Philippe Vaucher
> We just have a bunch of rough ideas. One of those ideas had a drafted
> structure and suggestion that it be used only in the input queue, where
> the memory inefficiency would not be a concern. I have pointed out that
> there are many other problems to solve: right hand sides of mappings,
> feedkeys(), registers for macro recordings (which indirectly affect
> buffers, as registers must be able to be put and yanked), representation
> in strings, and more (I just thought of the :normal command, and I'm
> sure there is still more...). I pointed out that if you use the
> structure to solve all these problems, memory efficiency is most
> certainly a concern.
> that I don't think it is a good way, because it is massively
> backwards-incompatible and a lot of work.
> plugins as little as possible, and I consider it essential that Vim
> doesn't become significantly less efficient because of this. And if it
> were me considering an implementation, I'd certainly be looking for the
> option that was the least work! YMMV.
hold vim back in 1970. You made a lot of good points and reasons for
it to be so, but I'm always sad when good ideas are refused just
because of old scenarios. I mean I wouldn't care if vim went as far as
doubling its memory usage and broke backward compatibility if what I
gain in exchange is modernity, but I guess not everyone is like me. I
just think old hardware should use old software and that new hardware
should use new software, I understand backward compatibility is what
makes a software like vim able to run on my toaster, but I wouldn't
mind using an older version of vim on that toaster.
That said, I think there is a compromise. Vim has features, maybe the
new input mechanism could simply be a feature, something like
+enhanced_term_input (like we have for +python or +eval), and plugins
could simply check with stuffs like:
if has('enhanced_input') == 0
echom 'Vim wasn't compiled with +enhanced_input, the plugin's
default mappings (<C-i>, etc) were remaped'
call s:UseOldMappings()
endif
I think this would please everyone. People wanting vim not to change a
single bit can, people wanting it to evolve can. Of course there's
still all the issues discussed above to solve, but at least I think
with this proposal things we'll get less resistance ;)
Thanks,
Philippe
Philippe Vaucher
> new input mechanism could simply be a feature, something like
> +enhanced_term_input (like we have for +python or +eval), and plugins
> could simply check with stuffs like:
default, or smth like that. Maybe this was already proposed tho.
Ben Schmidt
> hold vim back in 1970. You made a lot of good points and reasons for
> it to be so, but I'm always sad when good ideas are refused just
> because of old scenarios.
I don't think anybody has refused the idea, and I don't think anybody
wants to keep Vim in the 1970s. We all want this. In fact, people who
don't want this probably wouldn't get involved in the discussion--unless
we looked like we were going to introduce some horribly backwardly
incompatible change that would affect them nastily.
So I would encourage you not to view this discussion as opposition to
the idea. Quite the reverse. This discussion is actually an important
part of making this idea happen. We need to discuss these things so we
can do it the best way possible, without *needlessly* breaking backward
compatibility, without *needlessly* or *significantly* lowering
efficiency, and without *unnecessarily* wasting people's time.
> That said, I think there is a compromise.
Unless you saw something that I didn't see, I don't think there is any
resistance that necessitates a compromise.
I think it goes without saying that we'll need some kind of
compatibility mode, whether it's by means of a +feature, an 'option', or
just some carefully thought-out behaviours. Some of these issues have
already been raised and solutions brainstormed (including your recent
suggestions, as well as in earlier mails by me and others).
> Of course there's still all the issues discussed above to solve, but
> at least I think with this proposal things we'll get less resistance
> ;)
As I said, I don't think there is any resistance to the idea. We're just
discussing how to do it. As someone suggested earlier, it would be best
to discuss and draft some documentation for this stuff before doing the
hard implementation work. Having something solid will help, and a
checklist of issues/concerns along with their solutions.
So, to move it to the next stage, is anyone in a position to volunteer
to write up a more specific design (which probably needs to be written
with reference both to this email discussion and the Vim source code)?
Also, is anyone in a position to volunteer to help with implementation
once we have a design?
Ben.
Ben Schmidt
> open an account and send my suggestion. I use the default reply
> function and I don't mean to upset anyone. Hopefull now this reply is
> well-edited (I removed the quote).
Yes, it was a much better reply, thanks, Stephen. And welcome to the
list. You picked a good lively discussion to get involved in!
> How about checking the Scan Codes directly?
>
> <Enter> = 1c 9c
> <L-CTRL> = 1d 9d
> <M> = 32 b2
>
> <Tab> = 0f 8f
> <L-CTRL> = 1d 9d
> <I> = 17 97
We should be able to do it at a higher level than this, and avoid having
to deal with scan-code -> character mapping. Also, scan codes simply
aren't available when running in terminal mode. Every GUI will be
different, and different terminals will need different handling, but we
should be able to get something structured from the OS/window
manager/terminal which details the key pressed and its modifiers. I
don't think this part of things will be a problem. And since translation
from the GUI-specific form to Vim internal form will happen very
quickly, efficiency won't be an issue here either.
> About Paul's type:
>
>> Tab { 1, 0, KEY_TAB }
>> Ctrl-I { 0, MOD_CTRL, 'i' }
>> Ctrl-Shift-I { 0, MOD_CTRL, 'I' }
>
> I see is a switch type setting, it should work and require less
> changes but I really don't know how to differentiate if it is
> systemwise that<Tab> and<CTRL-I> gives the same 0x09.
GUIs should be able to get the information, and some terminals (e.g. via
libtermkey as has been discussed in another part of the thread).
The trouble with the structure is that it is big, and contrary to the
original assumption, it't not really useful to restrict its use to the
input queue. We need a solution that will work in other parts of Vim
efficiently--maybe a byte-stream solution, maybe an escape character
followed by a packed structure, maybe something we haven't thought of
yet. I think the byte-stream method will work well, and is closest to
what Vim already does so least work.
> Also about your question:
>
>> When there is one. Or a codepoint in some encoding. Yeah, your struct
>> does allow for that. The struct isn't inexpressive, it's just that it's
>> big, and if it needs to be used in a LOT of places (which seem to just
>> be increasing...input queue, mappings, registers, strings, buffers?!),
>> suddenly not just one thing is getting a bit bigger--a lot of things are
>> getting bigger--potentially a lot bigger! If every time you yank text
>> into a register it gets 12 times larger, that may cause some problems!
>
> Is it possilbe to swap those registers and histories onto hard disk so
> as to control the memory size? It has advantages of large capacity,
> infinite undo, and can be called to check the complete list.
It's not out of the question, but it would require significant changes
to a lot of Vim, I think, and is not what really how you would expect
them to work. Also, a massive chunk of disk space isn't all that much an
improvement than a massive chunk of main memory!
Cheers,
Ben.
Bram Moolenaar
> This is my first time joining a discussion, I had a hard time just to
> open an account and send my suggestion. I use the default reply
> function and I don't mean to upset anyone. Hopefull now this reply is
> well-edited (I removed the quote).
>
> About vim, I misunderstood that it was a keyboard problem. As Bram
> said it is internally set as <CTRL-I> = <Tab> and people requesting
> having structured input mechanism, all I can think of is to break
> apart all internal linkages and revert to the their original settings:
>
> I found that <CTRL-M> and <Enter> both have the same keymap code 0x0d,
> and both <Tab> and <CTRL-I> gives 0x09, maybe that is why vim gives
> the same output.
[...]
So the problem is that many users expect CTRL-M to have the same effect
as Enter, just like people use CTRL-[ instead of Esc. And a few people
would make the CTRL-M act different from Enter, and CTRL-[ different
from Esc.
First problem is to actually detect what key was pressed, in most
terminal emulators this is not possible. In the GUI we can. Changing
terminal emulators to support this and making this work with Vim is a
separate issue, I'll not go into that here.
Then we need a way to make the extra information available to be used in
mappings, without breaking it for users relying on the current way.
Some things that are no acceptable:
- Have a setting to enable "the new way". This will break existing
stuff and make users pull their hair out because they don't know this
setting exists. Forget it.
- Change the input queue from a stream of bytes to some list of structs.
This isn't adding any functionality and breaks all kinds of mapping
and termcode handling, register execution, etc. Forget it.
What we can do is extend the existing modifier byte sequence. This is a
bit tricky, but it should work. So we add a new byte sequence with the
raw key encoded, plus modifiers. Thus for CTRL-[ you get the [ key with
the CTRL modifier.
When this new modifier is not mapped, it is discarded and the following
translated codes are used normally, just like now. Thus without any
mappings that use the new modifier it is guaranteed to work as before.
When executing a register where the new modifiers are missing it works
just like before.
When a new modifier is mapped, it is replaced by what it's mapped to, AND
the following old style byte sequence is consumed.
An alternative is to let the termcap handling translate the new modifer
into the old style key. That works because mappings are applied both to
the raw escape sequences and on the key codes they are converted to.
We also need a way to specify mappings with the new modifier, perhaps
using a special modifier X, thus you could do:
:map <X-C-[> :echo CTRL-[<CR>
:map <X-C-I> :echo CTRL-I<CR>
--
Not too long ago, compress was something you did to garbage...
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
Charles E Campbell Jr
> [snip]
> Then we need a way to make the extra information available to be used in
> mappings, without breaking it for users relying on the current way.
>
> Some things that are no acceptable:
> - Have a setting to enable "the new way". This will break existing
> stuff and make users pull their hair out because they don't know this
> setting exists. Forget it.
> - Change the input queue from a stream of bytes to some list of structs.
> This isn't adding any functionality and breaks all kinds of mapping
> and termcode handling, register execution, etc. Forget it.
>
> What we can do is extend the existing modifier byte sequence. This is a
> bit tricky, but it should work. So we add a new byte sequence with the
> raw key encoded, plus modifiers. Thus for CTRL-[ you get the [ key with
> the CTRL modifier.
>
Another way to extend this might be to switch from a byte queue to a
short integer queue. The lower 8-bits would be as it is now; the upper
eight bits would encode modifier keys (shift, ctrl, alt, meta, whatever).
Regards,
Chip Campbell
Ben Schmidt
> queue. The lower 8-bits would be as it is now; the upper eight bits would encode
> modifier keys (shift, ctrl, alt, meta, whatever).
I think sticking to the byte queue is best. So much in Vim relies on
text being represented in either a single byte encoding or UTF-8 (e.g.
when you use UCS-2 or UCS-4, Vim will use UTF-8 internally, I believe).
It will break less if we stick to a byte stream with escapes, as now,
IMHO.
Ben.
Philippe Vaucher
> the idea. Quite the reverse. This discussion is actually an important
> part of making this idea happen. We need to discuss these things so we
> can do it the best way possible, without *needlessly* breaking backward
> compatibility, without *needlessly* or *significantly* lowering
> efficiency, and without *unnecessarily* wasting people's time.
unfortunately seen this happen too often... thanks for setting me back
on the right track :)
> So, to move it to the next stage, is anyone in a position to volunteer
> to write up a more specific design (which probably needs to be written
> with reference both to this email discussion and the Vim source code)?
> Also, is anyone in a position to volunteer to help with implementation
> once we have a design?
room for work to do on the design part... I think he outlined a
specific task to do (extend the existing modifier byte sequence) and
put a big no on all the alternatives discussed.
All-in-all if this answers the needs we have I'm for it, and it looks
as a fairly easy task to do (compared to the struct rewrite).
Philippe
Philippe Vaucher
old subject back, I changed it again.
Philippe
Bram Moolenaar
Charles Campbell wrote:
The current special-key encoding is a bit clumsy. On the other hand,
there are situations where the queue can get very long (recursive
execution of mappings or "1000@@"). Using 16 or 32 bits for every
character is not attractive.
One way to solve this is to use up to 128 bit integers, making sure that
the higher bits are usually zero, and then use variable length encoding.
ASCII characters should then still take only one byte.
http://code.google.com/apis/protocolbuffers/docs/encoding.html
Nevertheless, what gets encoded in registers must still be a byte
sequence. And we need to be able to store double-byte and multi-byte
characters. Thus I don't think this will make things simpler.
--
This is the polymorph virus! Follow these instructions carefully:
1. Send this message to everybody you know.
2. Format your harddisk.
Thank you for your cooperation in spreading the most powerful virus ever!
Paul LeoNerd Evans
> So the problem is that many users expect CTRL-M to have the same effect
> as Enter, just like people use CTRL-[ instead of Esc. And a few people
> would make the CTRL-M act different from Enter, and CTRL-[ different
> from Esc.
That's OK. For them it can be mapped. Or even made default.
> First problem is to actually detect what key was pressed, in most
> terminal emulators this is not possible. In the GUI we can. Changing
> terminal emulators to support this and making this work with Vim is a
> separate issue, I'll not go into that here.
Changing terminal emulators is a done task; xterm has supported this
since 2008. In fact I asked Thomas Dickey to clarify this for me, and he
informs me it's all present and working.
> Then we need a way to make the extra information available to be used in
> mappings, without breaking it for users relying on the current way.
>
> Some things that are no acceptable:
> - Have a setting to enable "the new way".
Really?
Is that what we'd have said right back at the beginning of Vim? On the
subject of the 'compatible setting:
"Adding a setting to enable all the vim-incompatible changes, is too
complex for users to handle."
> This will break existing
> stuff and make users pull their hair out because they don't know this
> setting exists. Forget it.
Right now we have users _all the time_ pulling their hair out _because_
such a setting does not exist. Including myself. I would much rather
have to support hoards of confused users in #vim by saying to them
"Yes, sorry your vim can't recognise that keypress now, but set this
setting and then it will"
instead of
"No, sorry your vim cannot recognise that keypress at all"
If you would prefer to support people by telling them it is not
possible, rather than telling them how to make it possible, please come
to #vim and explain that to them. Every week.
> - Change the input queue from a stream of bytes to some list of structs.
> This isn't adding any functionality and breaks all kinds of mapping
> and termcode handling, register execution, etc. Forget it.
OK, well that's fair. It was intended as an implementation detail in any
case. I hadn't realised it would be possible with the existing byte
queue.
As long as the two triplets of keypresses I suggested originally can all
be represented uniquely, and without reference to timing information in
the Escape vs Alt+ case, then I'm happy with whatever internal
implementation makes it happen.
> What we can do is extend the existing modifier byte sequence.
<snip long description>
That all sounds a bit long and complicated, and I'm not sure I see
whether it's necessarily any easier to implement than the arguably
much-neater and more forward-compatible queue-of-structures idea I had
originally. But again, that's all internal implementation details -
whatever makes it work, I'm happy with.
> We also need a way to specify mappings with the new modifier, perhaps
> using a special modifier X, thus you could do:
> :map <X-C-[> :echo CTRL-[<CR>
> :map <X-C-I> :echo CTRL-I<CR>
X for "eXtended"? Could work, though would be useful to check that
doesn't collide with any named modifiers on any existing system. Perhaps
some non-letter punctuation symbol? Though nothing suitable comes to
mind.
Philippe Vaucher
> bit tricky, but it should work. So we add a new byte sequence with the
> raw key encoded, plus modifiers. Thus for CTRL-[ you get the [ key with
> the CTRL modifier.
Escape d, and é all independantly? I don't know what possibilities
extending the existing modifier byte sequence allows.
Philippe
Paul LeoNerd Evans
> The whole situation is pretty arbitrary. E.g. ./demo's output under
> a modifyOtherKeys:2 UXTerm, compressed to one row, for <Shift>+ the
> top row of my typical US 105-key or variant:
>
> <S-~> ! <S-@> # $ % <S-^> & * ( ) <S-_> + <Backspace>
>
> Why the Shift modifier on ~, @, ^, and _, but not !, #, $, etc?
> (Guessing hysterical raisins.)
That sounds like a buggy implementation. Due to all the vagueness of
what shift means in the context of Unicode (is ! really ! or is it
Shift-1) my specification called for explicitly ignoring the state of
the Shift key in the terminal, at the time the modified Unicode sequence
is created. If that isn't happening, I vote bug.
> Didn't realize ECMA-48 existed, though. Thanks.
It's essential in these matters. :)
Once you read the spec part for CSI, you realise how short and neat the
entire scheme really is, how extensible it is provided you actually have
a real CSI parser and not a dumb string-prefix matcher a.la.
termcap/terminfo/curses combinations.
> > modifyOtherKeys: 2
> >
> >Except that on its own now breaks most other applications, because
> >xterm is too keen to use the CSI encoding scheme on "normal"
> >keypresses, such as Ctrl-D. xterm doesn't yet send the correct and
> >useful sequences here, but it is not difficult to fix its code to
> >do so - I have been in occasional contact with Thomas Dickey on
> >this issue; I need to poke him again sometime soon to check he's
> >properly looking into it.
>
> Cool. Works as advertised (i.e. currently not in anything other
> than the `demo` program, but still interesting to see the ability is
> built in).
I am also recently informed by Thomas Dickey that
formatOtherKeys: 1
sends a slightly nicer CSI u rather than CSI 27~ encoding
> >What is needed is a hybrid scheme, whereby the simplest key
> >encoding is used in a nicer way; where Ctrl-D would send the
> >single 0x04 byte (so kernel's pty driver can recognise "here's an
> >EOF"), but Ctrl-Shift-D sends CSI 68;5 u to encode it correctly.
>
> How do <Tab> and <Ctrl-i> fit into this scheme? What's the
> "simplest key encoding"?
<Tab> 0x08 (ASCII HT)
<Ctrl-I> CSI 105;5u (i == 105 decimal in ASCII)
<Ctrl-Shift-I> CSI 73;5u (I == 73 decimal in ASCII)
Note that, due to what I said above, we have to ignore the shift key in
these CSI u sequences; the distinction between Ctrl-I and Ctrl-Shift-I
is simply that one is lowercase and one is capital.
> >>how would using libtermkey be anything less than a huge PITA
> >>when working on multiple systems that aren't so configured? Or
> >>is this a chicken-and-egg problem? (Nothing is configured to
> >>send anything other than 70's-style "<C-i>" == "<Tab>", so
> >>nothing ever will be.)
> >
> >Rightnow, xterm will send the same byte for Tab, Ctrl-I or
> >Ctrl-Shift-I. Setting modifyOtherKeys to 2 will send real
> >keypresses that it can tell, but now everything else cannot.
> >
> >Ultimately, yes. It is chicken and egg, in order for -everything-
> >to work in a happy shiney way. But even without terminal changes,
> >applications can benefit from using libtermkey now.
> >
> >So to answer your question on "how would using libtermkey be
> >anything less than a huge PITA" - it isn't. It already recognises,
> >via its terminfo-driven parser, every keypress that
> >terminfo/curses already recognises. It cannot be worse. It is
> >strictly better than, because of the full CSI parser I mentioned
> >above.
>
> Thanks. This paragraph combined with the first section about
> parsing unknown CSI sequences is plenty convincing. (trimmed rest)
Another thought occurs to me.
Run vim in xterm. It recognises Ctrl-Left, because vim handles
TERM=xterm specially to recognise this case.
Run vim in screen in xterm. It doesn't recognise Ctrl-Left, instead
interpreting it as Escape, 1; (ignored), 5C. Deletes 5 lines of text and
enters insert mode. Mass fail.
Run libtermkey's demo either in xterm or in screen-in-xterm. Observe
that Ctrl-Left works IN BOTH CASES. This is because it always applies
its CSI parser, so always understands xterm-alike CSI encoding of
Ctrl-Left (et.al.).
This fact -alone- should be enough to convince you. :)
Paul LeoNerd Evans
On Mon, Mar 07, 2011 at 03:46:35PM +0000, Paul LeoNerd Evans wrote:
> As long as the two triplets of keypresses I suggested originally can all
> be represented uniquely, and without reference to timing information in
> the Escape vs Alt+ case, then I'm happy with whatever internal
> implementation makes it happen.
The two triplets in question being
<Tab>
<Ctrl-I>
<Ctrl-Shift-I>
<Escape> C
<Alt+C>
é
Yes I am aware that on current byte-driven terminals there is no way
without using timing information to distinguish all these three cases in
the latter triplet here. This should not stop GUI systems from
distinguishing them however.
Specifically as to the terminal case, I have a solution to this, but it
requires the cooperation of terminals and applications to use libtermkey
or some equivalent code, so right now I don't want to distract the
argument. That is what derailed it last time when I tried this a couple
of years ago - I don't want that to happen again for now. :)
Benjamin R. Haskell
> On Fri, Feb 25, 2011 at 02:43:23PM +0100, Bram Moolenaar wrote:
>> First problem is to actually detect what key was pressed, in most
>> terminal emulators this is not possible. In the GUI we can.
>> Changing terminal emulators to support this and making this work with
>> Vim is a separate issue, I'll not go into that here.
>
> Changing terminal emulators is a done task; xterm has supported this
> since 2008. In fact I asked Thomas Dickey to clarify this for me, and
> he informs me it's all present and working.
Despite its popularity, xterm is not "most terminal emulators". I think
this is a great move on xterm's part (having a more-or-less consistent
set of escapes for modifiers). But, it's still a chicken-and-egg
problem. Once it's in xterm, it takes a while (if ever) to filter down
to derivatives. And even under xterm the setting that allows this to
work isn't the default.
I almost didn't send this, since I don't think the lack of terminal
support should be a show stopper (If any terminal supports it, Vim
should use that support), but claiming that it's a done deal because
xterm added a feature to support it a few years ago is naïve.
--
Best,
Ben
Benjamin R. Haskell
be a bit roughdraftish]
So, instead of enabling a single setting (usenewkeymaps or whatever),
the :map command needs a new "Yes, do what I'm telling you to do"
modifier? This seems totally backwards to me. The "new way" will cause
problems with old stuff. Why is that breakage unacceptable? This whole
suggestion seems like a major-version change anyway (not a point-version
change, where backwards compatibility is sacrosanct). The stuff it's
breaking was caused by having to do a workaround in the first place.
Consider:
" {intent} should do {rhs}
:map {lhs} {rhs}
Where {intent} describes the intended binding (what key should be
pressed to activate it), and {lhs} is what was actually typed. In the
following chart, {around} indicates that the only reason that particular
{lhs} was chosen was to work around the limitation of key mappings.
{new-works} indicates that using that same {lhs} would still work under
the "new way". (^x indicates entering a literal control character)
{intent} | {lhs} | {around} | {new-works} | Category
---------+---------+----------+-------------+---------
<Enter> | <C-m> | yes | no | Enter vs Ctrl+m
<Enter> | <Enter> | no | yes
<Enter> | ^M | yes | no(?)
<C-m> | <C-m> | no | yes
<C-m> | <Enter> | ? | no
<C-m> | ^M | yes | yes(?)
---------+---------+----------+-------------+---------
<Esc> | <C-[> | ? | no | Esc vs Ctrl+[
<Esc> | <Esc> | no | yes
<Esc> | ^[ | yes | no(?)
<C-[> | <C-[> | no | yes
<C-[> | <Esc> | ? | no
<C-[> | ^[ | yes | yes(?)
---------+---------+----------+-------------+---------
<Tab> | <C-i> | yes | no | Tab vs Ctrl-i
<Tab> | <Tab> | no | yes
<Tab> | ^I | yes | no(?)
<C-i> | <C-i> | no | yes
<C-i> | <Tab> | ? | no
<C-i> | ^I | yes | yes(?)
---------+---------+----------+-------------+---------
é | <M-i> | yes | no | é vs Alt/Meta-i
é | é | no | yes | (issue w/ 8th-bit meta)
<M-i> | <M-i> | no | yes
<M-i> | é | yes | no
Not sure if the chart is clear, but my point is:
I think that the people who expect, e.g., <C-i> to activate a <Tab>
mapping are the people who either:
1. Know what's going on behind the scenes, so are knowledgeable enough
to fix their mappings (fixing plugin mappings is a separate issue)
2. Were bitten by something that didn't work right, but found a
workaround and only use it due to inertia.
I highly doubt there are people who approach the :map command with the
*desire* that <C-i> or ^I would activate a :map whose {lhs} contains
<Tab> or that <Tab> would activate a <C-i> :map. This seems like a
clear case where backwards compatibility is undesirable.
The fact that plugins probably rely on this behavior is unfortunate, but
"Oh, that plugin's <Tab> binding is broken because Vim's key mapping
used to be broken, and so it was using the Ctrl-i workaround" seems like
a preferable position to be in.
--
Best,
Ben
Paul LeoNerd Evans
> >Changing terminal emulators is a done task; xterm has supported
> >this since 2008. In fact I asked Thomas Dickey to clarify this for
> >me, and he informs me it's all present and working.
>
> Despite its popularity, xterm is not "most terminal emulators". I
> think this is a great move on xterm's part (having a more-or-less
> consistent set of escapes for modifiers). But, it's still a
> chicken-and-egg problem. Once it's in xterm, it takes a while (if
> ever) to filter down to derivatives. And even under xterm the
> setting that allows this to work isn't the default.
>
> I almost didn't send this, since I don't think the lack of terminal
> support should be a show stopper (If any terminal supports it, Vim
> should use that support), but claiming that it's a done deal because
> xterm added a feature to support it a few years ago is naïve.
This is regrettable, yes. But not entirely true.
Until xterm added the extended CSI notation using the 2nd parameter for
modifiers in e.g. CSI 1;5C for Ctrl-Left, no terminal in the world did
it.
Now, apart from rxvt which uses its own scheme, prettymuch every
terminal does it, and does it -that way-. These things aren't completely
set in stone, but they do take a long long time to filter around.
I'm very keen to try to shepherd people into one standard and
consistently-designed way of doing things. That rxvt does modified keys
differently is regrettable, and a consequence of splintered reinvention
of wheels. I'm very keen to try not to let that happen any more.
It's to this end I tried initially writing this spec for modified
keypresses:
http://www.leonerd.org.uk/hacks/fixterms/
It comes a little whiney and amateur at the moment, because I'm no good
at the social engineering side of trying to motivate a world-full of
1970's-era engineers into trying to fix a big problem. :) But the
technical details are there, awaiting someone with better writing skills
than I to neaten it up a bit.
Philippe Vaucher
> > As long as the two triplets of keypresses I suggested originally can all
> > be represented uniquely, and without reference to timing information in
> > the Escape vs Alt+ case, then I'm happy with whatever internal
> > implementation makes it happen.
>
> The two triplets in question being
>
> <Tab>
> <Ctrl-I>
> <Ctrl-Shift-I>
>
> <Escape> C
> <Alt+C>
> é
>
> Yes I am aware that on current byte-driven terminals there is no way
> without using timing information to distinguish all these three cases in
> the latter triplet here. This should not stop GUI systems from
> distinguishing them however.
to map those triplets independently as wanted? I'm not familiar enough
about vim's byte queue system to tell if the modifier byte really
answers our needs here.
Thanks.
Philippe
Ben Schmidt
>>> As long as the two triplets of keypresses I suggested originally can all
>>> be represented uniquely, and without reference to timing information in
>>> the Escape vs Alt+ case, then I'm happy with whatever internal
>>> implementation makes it happen.
>>
>> The two triplets in question being
>>
>> <Tab>
>> <Ctrl-I>
>> <Ctrl-Shift-I>
>>
>> <Escape> C
>> <Alt+C>
>>
>> Yes I am aware that on current byte-driven terminals there is no way
>> without using timing information to distinguish all these three cases in
>> the latter triplet here. This should not stop GUI systems from
>> distinguishing them however.
>
> Can someone (Bram?) tell us if Bram's proposed solution would allow us
> to map those triplets independently as wanted?
Of course it does. Bram wouldn't have suggested it if it wouldn't work
at all....
> I'm not familiar enough about vim's byte queue system to tell if the
> modifier byte really answers our needs here.
I don't think we need to keep the same byte-stream representation as we
currently have, and I don't think that's what Bram meant. I think we can
change it. As long as it remains a byte-stream representation.
And I don't think it will be possible to do what we need without at
least extending it. At the moment, keys are a fixed length of 3 bytes
(one of which is the 'escape byte' 0x80, and the other two of which have
restricted values). There simply isn't enough value-space to do what we
need, and it's possibly not very future-proof even if we can squeeze it
in. It's also a bit of a mess, IIRC. It would be better to give it an
overhaul. And it probably needs to use variable-length sequences.
A thought occurred to me a couple of days ago: would it be appropriate
to actually use CSI as the byte-stream representation? Or something like
it. It is a byte-stream representation, right? And it's extensible,
right, so it is less likely to need a big overhaul in future? Does it
allow for 'private use' stuff? Because Vim needs a way to represent a
bunch of other events as keystrokes (mouse clicks, scroll wheel, its
magic script-local meta-character thing, representing the CSI escape
character itself, etc.). Also, is it easy to skip over CSI sequences if
necessary with simple logic?
I think there are some nice/helpful things that could be done
design-wise though. For example, Vim is often used with enc=utf-8 these
days. I think it would be wise to, if possible, ensure all key sequence
representations are invalid UTF-8 in their entirety. Then they could be
put in buffers and yanked back into registers without ambiguity
(hopefully), i.e. there wouldn't be the ambiguity of whether the escape
character was part of a key sequence that should be yanked as such, or
part of the buffer text which should be escaped upon yanking.
See, Vim already deals with the CSI character in its byte-stream code,
and escapes it, so some aspects of this are already somewhat accounted
for. But I wonder when precisely it is escaped, and whether at times it
is escaped when it shouldn't be, or not when it should be. It would be
good to design to avoid this problem.
(I happened to be hacking on some Vim code recently where this came into
play, so I noted it with interest.)
Ben.
Paul LeoNerd Evans
> A thought occurred to me a couple of days ago: would it be appropriate
> to actually use CSI as the byte-stream representation? Or something like
> it. It is a byte-stream representation, right? And it's extensible,
> right, so it is less likely to need a big overhaul in future? Does it
> allow for 'private use' stuff? Because Vim needs a way to represent a
> bunch of other events as keystrokes (mouse clicks, scroll wheel, its
> magic script-local meta-character thing, representing the CSI escape
> character itself, etc.). Also, is it easy to skip over CSI sequences if
> necessary with simple logic?
Sure. CSI is intended for just this. A CSI sequence is matched by a
regexp
CSI [\x20-\x2f]? [\x30-\x3f]* [\x40-\x7e]
Rougly "possibly-one punctuation character, then a bunch of numbers, ; or
;, then finally a signle letter."
The "private use" area in CSI is the character range \x70-\x7e. One of
these, CSI u (\x75) is already in use by xterm and libtermkey to encode
modified Unicode characters. To my knowledge, the remaining are free.
Plus, vim could even use this encoding internally if it wanted.
Tom Sorensen
up time and time again on #vim in Freenode, and Paul still wants this
to happen, as do many of us.
On Fri, Feb 25, 2011 at 8:43 AM, Bram Moolenaar <Br...@moolenaar.net> wrote:
>
>
> So the problem is that many users expect CTRL-M to have the same effect
> as Enter, just like people use CTRL-[ instead of Esc. And a few people
> would make the CTRL-M act different from Enter, and CTRL-[ different
> from Esc.
>
> First problem is to actually detect what key was pressed, in most
> terminal emulators this is not possible. In the GUI we can. Changing
> terminal emulators to support this and making this work with Vim is a
> separate issue, I'll not go into that here.
Actually, that's exactly the issue that is being addressed. Vim
shouldn't have to address it -- in theory termcap/terminfo were
supposed to do this, but they didn't really do so well. Which is why
we now have difficulty telling the difference between <ESC>o and
<A-o>, as well as things like <ESC> vs <C-[>.
Ultimately the proposal is that we let the raw keyboard input (for
terminals) be handled by libtermkey, and it figures out if what was
pressed was <ESC> or <c-[> or <C-[>o vs <ESC>o vs <A-o>. It then
passes a stream of information to vim, which vim interprets properly.
And yes, by default there would be a mapping (either explicit or
pseudo-builtin) that treats <ESC> and <C-[> as the same, but that can
then be changed by the user, just as they can now with any other
mapping.
> Then we need a way to make the extra information available to be used in
> mappings, without breaking it for users relying on the current way.
The only breakage I can foresee is if someone actually does a :map
<c-i> foo instead of :map <Tab> foo -- then, yes, you'd end up
with different behavior. I think this is within the realm of
acceptable change, particularly if it's clearly stated in the release
notes (and yes, I think this is a Vim 8 thing).
> Some things that are no acceptable:
> - Have a setting to enable "the new way". This will break existing
> stuff and make users pull their hair out because they don't know this
> setting exists. Forget it.
I actually agree -- it should be just the way things work. And if it's
done right then it won't break anything (beyond the above).
> - Change the input queue from a stream of bytes to some list of structs.
> This isn't adding any functionality and breaks all kinds of mapping
> and termcode handling, register execution, etc. Forget it.
The termcode handling will be replaced by this, so that's not an
issue. Mapping is also not an issue beyond the above.
Registers are more of an issue. No doubt. Can we think on this to come
up with a solution? Perhaps the register needs to become more like a
:map, with special keys expressed specially? Otherwise I think we're
stuck since you cannot differentiate between <ESC> and <c-[>, and
there's no way to express things like <c-space>.
> What we can do is extend the existing modifier byte sequence. This is a
> bit tricky, but it should work. So we add a new byte sequence with the
> raw key encoded, plus modifiers. Thus for CTRL-[ you get the [ key with
> the CTRL modifier.
Which is precisely what libtermkey emits, and how it can differentiate
between ctrl-[ and alt-ctrl-[, etc.
Really what happens is that the terminal input buffer starts to look a
great deal more like events that the GUI versions get. Instead of
<ESC>o you get "o with alt depressed".
To be very clear -- some of this will depend on a terminal with the
proper support being used. But anything that's built on libvterm
already has that, as well as xterm. But if you don't have those then
moving to this new scheme should not break ANYTHING. Your term would
still send <ESC>o and libtermkey would tell vim it saw <o with alt>.
But if you do have a terminal with the right support, or you're using
the GUI... well, to quote a bit from :help design-multi-platform,
"Support all the keys on the keyboard for mapping."
It would be great to get closer to that.
Tom Sorensen
Joseph Pea
Paul Evans suggested to use a modifiedunicode option (along with some other options) to enable the "new features":
:set nomodifiedunicode
:map <Tab> ONE
:map <Ctrl-I> TWO
:set modifiedunicode
and Bram said that it would be too complicated. I agree.
Bram followed by suggesting a mapping prefix "X-" for the new mappings we all want:
:map <X-C-[> :echo CTRL-[<CR>
:map <X-C-I> :echo CTRL-I<CR>
The simplest solution (that Benjamin Haskell suggests in his previously mentioned table) is to make the keys map exactly how the user wants.
map <tab> :echo "tab key pressed"
map <c-i> :echo "control plus I pressed"
map <c-s-i> :echo "control shift I pressed"
map <esc>c :echo "escape c pressed"
map <a+c> :echo "alt plus c pressed"
map é :echo "eh?"
Users of the new vim will just simply have to change their mappings accordingly.
If you were doing:
map <c-i> :echo "the real tab"
Well, don't.. Change it to:
map <tab> :echo "the real tab"
That's not too much to ask for, especially since the influx of new users aren't aware of these problems when they start using vim and will try literal mappings because it's intuitive. They will not map <c-i> thinking "oh, now I've modified what tab does". No. They'll instead be thinking "why on earth does ctrl+i not work. aaaarrrghh?" and pull their hair out.
If you're someone who's used to doing
map <c-[> echo "I've changed the escape key"
Well, then you'll just have to adapt. We live in a much more modern world.
Adding options like modifiedunicode or prefixed map identifiers like <x-c-[> is unnecessary. My .vimrc should simply *just work* the way i intend and expect when I map <c-[>. How this all gets implemented in the inner workings of vim is irrelevant when considering how .vimrc should behave. Doing
imap <c-bs> :Whatever<cr>
in .vimrc works just fine in gVim, as it should in console vim.
So, basically, nothing needs to change in the way that I can currently write my .vimrc file (no new settings, no new mapping identifiers, just everything works).
The most trouble this will cause is that some people will have to rewrite their mappings with the correct keystroke identifiers, which is very easy to do.
I hope this proposal becomes reality sometime sooner than later!
Paul LeoNerd
Joseph Pea <tru...@gmail.com> wrote:
> The most trouble this will cause is that some people will have to
> rewrite their mappings with the correct keystroke identifiers, which
> is very easy to do.
>
> I hope this proposal becomes reality sometime sooner than later!
This has been dragging on for years now (I think it was 8 years ago
that I first commented this to vim-dev@).
We've yet to even receive agreement /that/ the problem really exists,
let along agreement on /how/ to fix it, or heaven forbid, a patch to
actually fix it.
ZyX
1. Recognition and mapping keys that currently cannot be mapped.
2. Keeping backward compatibility with old behavior: e.g. keeping <Tab> and <C-i> the same while allowing to map them separately.
I think that these problems should have separate solutions. To solve 1. we must solve two problems
1. Have an agreement for new escaping scheme for byte queue. I may suggest taking some unused <80>{C1}{C2} sequence and agreeing that <80>{C1}{C2}{CMod}{Key}<ETX> means that
1. User pressed key named {Key} which is either a key name (like Up) or a character in &encoding with the exception that it cannot be control character. Key name obviously must be normalized.
2. This key was pressed with modifiers {CMod} which contains a bit field:
Bit idx Meaning
7 Always 1. Does not allow byte to be zero.
6 Shift
5 Control
4 Alt
3 Win
2 Reserved (X11 definitely has something to fill it with, but I do not know what. Super?)
1 Reserved.
0 Reserved.
// 0. <ETX> is simply an end-of-key marker.
3. Any key that can be represented with already existing <80>{C1}{C2} sequence *must* be represented with it. Control-{latin letter} without additional modifiers *must* be represented with control character. Latin letter without modifiers except for shift *must* be represented as itself. And so on: existing scheme should be kept for compatibility.
2. Second problem is pure technical: someone must sit down and code this. Maybe use some terminal library, maybe not. The main idea of p. 1. is that we need *no* changes if we *not* want to support new functionality (this is mainly needed not because someone may not want, but because we may be unable to find developers to code this functionality into specific supported GUI).
Solving top 2. (backward compatibility) issue with the above agreement on backwards compatibility in escaping may be deferred until there is an agreement while continuing to work on more urgent problem with not mappable sequences. As I mostly use ,{key} mappings and not control sequences I do not have a strong opinion on what to do with this: <Tab>=<C-i> and <Tab>≠<C-i> are both fine, though I got used to use <Tab> for completion and <C-i> as an opposite for <C-o> (for consistency). Do not think this will change as built-in functionality.
If we solve top 1. solving top 2. will be much more trivial unless found agreement will be something insane, like DWIM behavior based on statistics of <Tab> hits after typing <C-i>.
/#!/JoePea
On Sun, 13 Oct 2013 14:00:15 -0700 (PDT)+1
Joseph Pea <tru...@gmail.com> wrote:
> The most trouble this will cause is that some people will have to
> rewrite their mappings with the correct keystroke identifiers, which
> is very easy to do.
>
> I hope this proposal becomes reality sometime sooner than later!
This has been dragging on for years now (I think it was 8 years ago
that I first commented this to vim-dev@).
We've yet to even receive agreement /that/ the problem really exists,
let along agreement on /how/ to fix it, or heaven forbid, a patch to
actually fix it.
Paul LeoNerd
"/#!/JoePea" <tru...@gmail.com> wrote:
> I think you, Paul Evans, would be a good choice for the mentor who
> would guide the students through the effort.
insides of vim. I know terminals, and I know what outward-facing
black-box behaviour vim ought to be capable of providing on a terminal,
but I've no idea how to go about making it do so.
Paul LeoNerd
ZyX <zyx...@gmail.com> wrote:
> 2. Second problem is pure technical: someone must sit down and code
> this. Maybe use some terminal library, maybe not.
basically what I keep making every few years.
If the input queue internals are updated to support arbitrary key
sequences, then it becomes trivial to attach something like my
libtermkey to feed that input queue from the terminal.
http://www.leonerd.org.uk/code/libtermkey/
The first hurdle is getting anyone sufficiently close-to-core to accept
/that/ the problem needs fixing, and only thereafter to accept /how/.
Nikolay Pavlov
On Oct 16, 2013 4:43 PM, "Paul LeoNerd" <leo...@leonerd.org.uk> wrote:
>
> On Tue, 15 Oct 2013 12:03:41 -0700 (PDT)
> ZyX <zyx...@gmail.com> wrote:
>
> > 2. Second problem is pure technical: someone must sit down and code
> > this. Maybe use some terminal library, maybe not.
>
> Not wanting to sound like a broken record, but this suggestion is
> basically what I keep making every few years.
>
> If the input queue internals are updated to support arbitrary key
> sequences, then it becomes trivial to attach something like my
> libtermkey to feed that input queue from the terminal.
>
> http://www.leonerd.org.uk/code/libtermkey/
>
> The first hurdle is getting anyone sufficiently close-to-core to accept
> /that/ the problem needs fixing, and only thereafter to accept /how/.
As far as I see discussion always stucks at discussing backward compatibility. My suggestion is that as we cannot make an agreement backward compatibility should be kept fully (regarding things like CTRL-I vs TAB, not undistinguishable CTRL-L and CTRL-SHIFT-L) and code should be written. Not continue discussing WTF we are going to do with tabs. Not trying to push backwards incompatibility. And not writing code that will be tricky like having something to specify "here we mean Tab", "here we mean CTRL-I" and "here we mean any of them" which will likely mean having a bunch of hacks in mapping processing code. All these may be written later if needed.
It is better to not have an agreement on what to do with Tabs and have a patch to the input system then both not have an agreement and not have patch. Also if you do not introduce incompatibility, but have written a patch without Bram explicitly writing that this patch will be accepted once written you have rather good chances to make it accepted with the support from community. If you do introduce incompatibility this will be more tricky.
Also did not Bram already agree on the existence of the problem?
/#!/JoePea
/#!/JoePea
map <tab> :echo "does something"
map <c-i> :echo "does something"
map <tab> :echo "does something"
map <c-i> :echo "does something *else*"
--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
---
You received this message because you are subscribed to a topic in the Google Groups "vim_dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vim_dev/2bp9UdfZ63M/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vim_dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
Ingo Karkat
> On Sun, 13 Oct 2013 14:00:15 -0700 (PDT) Joseph Pea
> <tru...@gmail.com> wrote:
>
>> I hope this proposal becomes reality sometime sooner than later!
>
> +1
>
> This has been dragging on for years now (I think it was 8 years
> ago that I first commented this to vim-dev@).
> We've yet to even receive agreement /that/ the problem really
> exists, let along agreement on /how/ to fix it, or heaven forbid, a
> patch to actually fix it.
(especially newcomers to Vim), I've been meticulously tagging all
related questions that regularly come up on Stack Overflow and related
sites (by referencing Paul's name in my answers):
https://www.google.com/search?q=site%3Astackoverflow.com%20%2B%22Ingo%20Karkat%22%20%22foremost%20Paul%20LeoNerd%20Evans%22
(about 29 results)
https://www.google.com/search?q=site%3Asuperuser.com+%2B%22Ingo+Karkat%22+%22foremost+Paul+LeoNerd+Evans%22
(3 results)
That's more than 30 confused and eventually disappointed users (over the
past ~ 1.5 years).
To regain momentum, I think it would help if Bram would briefly signal a
willingness to consider this for a future Vim 8.0. Then, after a
re-evaluation of the floated proposals, we'd hopefully agree on an
approach and have some volunteers start implementing on a branch off 7.4.
I'd be certainly willing to comment and try out patches. My personal
motivation is that I'm running out of keys for short custom mappings :-)
-- regards, ingo
Bram Moolenaar
compatible. That means encoding keys on top of what's already there,
and falling back to the ordinary key if the key + modifier isn't mapped.
How to detect the modifiers for many terminals in a portable way,
without requiring installing an obscure library (at least Ubuntu must
have it), I don't know.
--
An indication you must be a manager:
You feel sorry for Dilbert's boss.
Nikolay Pavlov
> How to detect the modifiers for many terminals in a portable way,
> without requiring installing an obscure library (at least Ubuntu must
> have it), I don't know.
I think that vim is popular enough to expect distribution maintainers to package library once we start using it. With some lag, greater as this library will likely be optional, but add.
> --
> An indication you must be a manager:
> You feel sorry for Dilbert's boss.
>
> /// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
> /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
> \\\ an exciting new programming language -- http://www.Zimbu.org ///
> \\\ help me help AIDS victims -- http://ICCF-Holland.org ///
>
> --
> --
> You received this message from the "vim_dev" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+u...@googlegroups.com.
Paul LeoNerd
Bram Moolenaar <Br...@moolenaar.net> wrote:
> How to detect the modifiers for many terminals in a portable way,
> without requiring installing an obscure library (at least Ubuntu must
> have it), I don't know.
All modified keys come in a CSI sequence, where the second numeric
argument encodes the modifiers.
CSI P1; P2 u --- Modified Unicode character P1
CSI 1; P2 [ABCDFH] --- Modified Up/Down/Right/Left/End/Home
CSI 1; P2 [PQRS] --- Modified F1/F2/F3/F4
CSI P1; P2 ~ --- Modified "function" key
-- where the first few numbers encode
2 Ins
3 Del
5 PageUp
6 PageDown
In each case P2 encodes 1 + modifier state as a bitmask, 1 = Shift, 2 =
Alt, 4 = Ctrl; so
P2 1 unmodified
2 S-
3 A-
4 A-S-
5 C-
6 C-S-
7 C-A-
8 C-A-S-
All is explained on
http://www.leonerd.org.uk/hacks/fixterms/
I don't /actually/ care so much whether I can map Ctrl-Shift-I or not,
in vim; what I do care about is that vim understands THE BASIC PARSING
OF A CSI SEQUENCE and doesn't go "beep, leave insert mode, undo the
last 5 changes now" just because it got confused and saw a CSI nnn;5u
encoding of a Ctrl-letter sequence. So please; even if vim doesn't
do /anything/ with that modifier, I would like vim to be able to
RECOGNISE a CSI sequence, and know that it DOES NOT STOP until the
first character in the range 0x40-0x7e - no matter how many other
digits or other symbols were found first.
Without that, it means every single time my finger happens to slip and
I type some unrecognised Ctrl- sequence currently, I instantly leave
insert mode, undo the past 5 changes, and if I so much as press a
-single- extra key now I am likely to branch the undo history and
prevent my being able to "redo" those falsely-undone changes back in.
This is getting /incredibly/ frustrating.
Paul LeoNerd
Paul "LeoNerd" Evans <leo...@leonerd.org.uk> wrote:
> > How to detect the modifiers for many terminals in a portable way,
> > without requiring installing an obscure library (at least Ubuntu
> > must have it), I don't know.
>
> Then let me explain it to you Bram; it's very simple.
>
> All modified keys come in a CSI sequence, where the second numeric
> argument encodes the modifiers.
>
> CSI P1; P2 u --- Modified Unicode character P1
>
> CSI 1; P2 [ABCDFH] --- Modified Up/Down/Right/Left/End/Home
You already know this one Bram - vim /already/ recognises modified
arrow keys when TERM=xterm. Just extend that logic to the remaining
CSI-encoded keys, and accept CSI u as "modified unicode" and we're
done. Just please don't hardcode it to only "TERM=xterm", but accept it
always. Or at the very least, also accept it under TERM=screen,
TERM=tmux, or any other situation where it is likely these keys will
come up; but ideally all terminals. Don't make it conditional.
/#!/JoePea
I already said this: It's fine to add so long as it's 100% backwards
compatible. That means encoding keys on top of what's already there,
and falling back to the ordinary key if the key + modifier isn't mapped.
Let me describe how the current behavior works and two possible ways the future behavior would work.
Currently, pressing <tab> or pressing <c-i> will echo "hello".
1) The first way this could work in the future, when vim differentiates ever key combination, is that pressing <tab> or pressing <c-i> would continue to echo "hello". However, the user could override the default <c-i> behavior by adding a new map in which case <c-i> would take on an all-new behavior:
map <c-i> :echo "world"<cr>
How to detect the modifiers for many terminals in a portable way,
without requiring installing an obscure library (at least Ubuntu must
have it), I don't know.
--
An indication you must be a manager:
You feel sorry for Dilbert's boss.
/// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
/// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org ///
\\\ help me help AIDS victims -- http://ICCF-Holland.org ///
--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
---
You received this message because you are subscribed to a topic in the Google Groups "vim_dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vim_dev/2bp9UdfZ63M/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vim_dev+u...@googlegroups.com.
Michael Henry
> > How to detect the modifiers for many terminals in a portable way,
> > without requiring installing an obscure library (at least Ubuntu must
> > have it), I don't know.
>
> I think that vim is popular enough to expect distribution
> maintainers to package library once we start using it. With
> some lag, greater as this library will likely be optional, but
> add.
features, a hypothetical extra library would be required only
for users that enable the feature. As I routinely rebuild Vim
anyway, I'd be delighted to install an extra library or two to
gain this feature. Once the feature becomes well-supported and
its worth proven, distro maintainers will get requests to enable
this feature in Vim, and they'll start packaging any dependent
libraries accordingly. But even if they cannot be convinced to
support this feature, users like myself will still have the
benefits (which are substantial, IMO).
Michael Henry
Paul LeoNerd
"/#!/JoePea" <tru...@gmail.com> wrote:
> I've given this some thought. The backwards compatibility that users
> should experience should be as simple as possible.
>
> For example, let's suppose someone has the following in their vimrc.
>
> map <tab> :echo "hello"<cr>
>
> Let me describe how the current behavior works and two possible ways
> the future behavior would work.
>
> Currently, pressing <tab> or pressing <c-i> will echo "hello".
>
>
>
> pressing <c-i> would continue to echo "hello". However, the user
> could override the default <c-i> behavior by adding a new map in
> which case <c-i> would take on an all-new behavior:
>
> map <tab> :echo "hello"<cr>
> map <c-i> :echo "world"<cr>
>
> <c-i> would no longer echo "hello", it would echo "world". The same
> would apply to other aliases.
if #vim on Freenode is anything to go by) would find that more
intuitive and correct; most aren't even aware of the Tab == C-i
identity anyway. Those few people who do rely on C-i actually meaning
the same as Tab will know enough to double-map them anyway.
> Most new users to vim are
> also new to terminals, and therefore they (like me) won't know that
> pressing <tab> is the same as pressing <c-i>. They'll discover this
> after having pulled their hair out over a <c-i> mapping in their
> vimrc.
> > How to detect the modifiers for many terminals in a portable way,
> > without requiring installing an obscure library (at least Ubuntu
> > must have it), I don't know.
>
> I agree this might be tricky and not fully supported in terminals at
> first, but we should at least have this new functionality fully
> supported in graphical Vim. As Paul said, it's likely that if vim
> implements this new functionality, many terminal developers will
> incorporate support for the new key sequences within due time. If at
> least one terminal emulator supports it, I'll be extremely happy!
directly into the vim tree - I specifically allow that sort of thing by
it having the MIT licence. In case that isn't sufficient I will
specifically state that now: I will allow you to copy the libtermkey
sources into vim, alter or modify them, and use them in any way you see
fit in order to make this feature work.
Initial research on a new blank vim looks promising. While it won't
recognise the keys you press, it does at least allow storing of 'map's
for them:
<C-S-i> "Again"
<C-i> "World"
<Tab> "Hello"
<C-m> "Again"
<C-j> "World"
<Enter> "Hello"
Curiously, while it can handle é vs <M-C> and <A-C>, it does seem to
consider M- and A- distinct:
<M-C> "World"
<A-C> "Hello"
é "Again"
Michael Henry
> I already said this: It's fine to add so long as it's 100% backwards
> compatible. That means encoding keys on top of what's already there,
> and falling back to the ordinary key if the key + modifier isn't mapped.
compatibility out-of-the-box, which is a fine (and longstanding)
goal for Vim.
I'd be happy to have a Vim option to control this feature. It
could default to providing 100% compatible key processing. If
the user changes this option, he would get clean support for key
modifiers with some slight backward incompatibilities.
For example, the aliasing of control keys (e.g., CTRL-I being
equivalent to <Tab>) is a historical artifact that I suspect has
no value to the vast majority of users. If there were no
compatibility concern and it's weren't *already* true that
CTRL-I aliases <Tab>, would anyone seriously argue that we ought
to *add* that feature to Vim? I suspect not. To me, that's a
convincing argument to do the simplest possible kind of
backward compatibility, since very few users actually need the
old behavior.
So I suggest that a single global option that simply switches on
support for extended modifier for all keys, regardless whether
those keys are mapped, may well be good enough and might make
the implementation simple enough to become reality. The day the
option appears in Vim, I'll put it at the top of my .vimrc :-)
Michael Henry
ZyX
> On Sat, Oct 26, 2013 at 12:50 PM, Bram Moolenaar <Br...@moolenaar.net> wrote:
> > I already said this: It's fine to add so long as it's 100% backwards
> > compatible. That means encoding keys on top of what's already there,
> > and falling back to the ordinary key if the key + modifier isn't mapped.
>
> This is very encouraging to me. I read this as 100%
> compatibility out-of-the-box, which is a fine (and longstanding)
> goal for Vim.
>
> I'd be happy to have a Vim option to control this feature. It
> could default to providing 100% compatible key processing. If
> the user changes this option, he would get clean support for key
> modifiers with some slight backward incompatibilities.
>
> For example, the aliasing of control keys (e.g., CTRL-I being
> equivalent to <Tab>) is a historical artifact that I suspect has
> no value to the vast majority of users. If there were no
> compatibility concern and it's weren't *already* true that
> CTRL-I aliases <Tab>, would anyone seriously argue that we ought
> to *add* that feature to Vim? I suspect not. To me, that's a
> convincing argument to do the simplest possible kind of
> backward compatibility, since very few users actually need the
> old behavior.
Also some users (including me) are used to use Ctrl-I and Tab interchangeably. It is not much problem to restore old status if <C-i> and <Tab> started to have different meaning even without any options for backwards compatibility, thus I would not object against patch that will diversify them.
I do not get though why <C-B> and <C-S-B> should ever have the same meaning though, and it is what is meant by “100% backwards compatibility”.
> So I suggest that a single global option that simply switches on
> support for extended modifier for all keys, regardless whether
> those keys are mapped, may well be good enough and might make
> the implementation simple enough to become reality. The day the
> option appears in Vim, I'll put it at the top of my .vimrc :-)
This option (if implemented) is also much likely a candidate for resetting when 'compatible' is reset. Thus adding it at the very top is not a good idea if you have `set nocompatible` there. And it is good to have `:scriptencoding utf-8` as the very first line in the vimrc.
>
> Michael Henry
Paul LeoNerd
ZyX <zyx...@gmail.com> wrote:
> Please explain how you are going to differentiate CTRL-I and Tab in
> random terminal emulator. Some may be configured to output either as
> CSI sequence, but not all. This is not simply historical artifact.
some terminals lack an F12 key. Vim has no problem storing maps from
keys it knows the terminal cannot possibly generate. If users are
mapping from keys their terminal cannot type that is their problem.
> I do not get though why <C-B> and <C-S-B> should ever have the same
> meaning though, and it is what is meant by “100% backwards
> compatibility”.
example, without being quite aware that it wasn't C-B.
Andre Sihera
>> Please explain how you are going to differentiate CTRL-I and Tab in
>> > random terminal emulator. Some may be configured to output either as
>> > CSI sequence, but not all. This is not simply historical artifact.
> You can't. Does that matter? Some people write a :map from F12, but
> some terminals lack an F12 key. Vim has no problem storing maps from
> keys it knows the terminal cannot possibly generate. If users are
> mapping from keys their terminal cannot type that is their problem.
>
this that all users who don't fit into the ideas of the vocal few
"obviously" have a problem.
I'll tell you one good reason why people are mapping keys from
their terminal that that cannot physically type, and that is keyboard
remote control (synergy, virtual machine environments, etc.).
I, for one, have all my F-keys mapped up onto other keys because
the host system I am using to remote control intercepts all the F-
keys before they get passed onto the remote system, so I have to
map them up.
Additionally, TAB, for example, in Japan can be a language input
modifier key for switching between kanji/kana and Roman alphabet.
Some keyboards have an additional dedicated key for this but not
all do. On a remote link you may have to use Ctrl+I to get a tab
character to come out as the tab key locally can be intercepted at
a very low level by Japanese (or other foreign) operating systems.
I do not believe I am the only one who uses keyboard/mouse remote
control to another system and edits a file in ViM, nor am I the only
one who uses a computer in more than one language and uses VIM
keyboard mappings to circumvent local system inadequacies with
regards to multi-lingual support. However, as I guarantee that most
of these other people don't "hang out" on "#vim" for most of their
lives I take the view that the "vast majority" of ViM users (including
myself) would be annoyed if their keyboard mappings just suddenly
stopped working because of a few who can't see beyond their English-
only QWERTY-compliant wire-connected keyboards.
Andre Sihera
> On 28/10/13 02:58, Paul LeoNerd Evans wrote:
>>> Please explain how you are going to differentiate CTRL-I and Tab in
>>> > random terminal emulator. Some may be configured to output either as
>>> > CSI sequence, but not all. This is not simply historical artifact.
>> You can't. Does that matter? Some people write a :map from F12, but
>> some terminals lack an F12 key. Vim has no problem storing maps from
>> keys it knows the terminal cannot possibly generate. If users are
>> mapping from keys their terminal cannot type that is their problem.
>>
> I'll tell you what the real problem is. It's bare-faced arrogance like
> this that all users who don't fit into the ideas of the vocal few
> "obviously" have a problem.
>
> I'll tell you one good reason why people are mapping keys from
> their terminal that that cannot physically type, and that is keyboard
> remote control (synergy, virtual machine environments, etc.).
> I, for one, have all my F-keys mapped up onto other keys because
> the host system I am using to remote control intercepts all the F-
> keys before they get passed onto the remote system, so I have to
> map them up.
>
> Additionally, TAB, for example, in Japan can be a language input
> modifier key for switching between kanji/kana and Roman alphabet.
key should be Caps Lock, not TAB. However, the principle of the
argument remains the same. International keyboards use the "not so
frequently used" keys for local idiomatic functions and so stripping
away the traditional "archaic mappings" may remove the only way of
entering a key in some situations.
Ben Fritz
> On 28/10/13 02:58, Paul LeoNerd Evans wrote:
>
> >> Please explain how you are going to differentiate CTRL-I and Tab in
>
> >> > random terminal emulator. Some may be configured to output either as
>
> >> > CSI sequence, but not all. This is not simply historical artifact.
>
> > You can't. Does that matter? Some people write a :map from F12, but
>
> > some terminals lack an F12 key. Vim has no problem storing maps from
>
> > keys it knows the terminal cannot possibly generate. If users are
>
> > mapping from keys their terminal cannot type that is their problem.
>
> >
>
> I'll tell you what the real problem is. It's bare-faced arrogance like
>
> this that all users who don't fit into the ideas of the vocal few
>
> "obviously" have a problem.
>
>
I think he's saying that if your terminal cannot tell a <C-I> from a <Tab>, then neither will Vim be able to. And that if a user tries mapping them separately but the terminal cannot actually send a <Tab> that differs from <C-I> then it is the user's fault for using a dumb terminal while trying to use this feature in Vim.
>
> I'll tell you one good reason why people are mapping keys from
>
> their terminal that that cannot physically type, and that is keyboard
>
> remote control (synergy, virtual machine environments, etc.).
>
> I, for one, have all my F-keys mapped up onto other keys because
>
> the host system I am using to remote control intercepts all the F-
>
> keys before they get passed onto the remote system, so I have to
>
> map them up.
>
>
And you'll be able to continue doing this. Nobody is suggesting removing the ability to:
:nmap <A-F>1 <F1>
:nmap <A-F>2 <F2>
etc. (or the reverse, I'm not 100% sure what you mean by "map them up")
What is being suggested is that you could do:
:nnore <Tab> :echo "hello"<CR>
:nnore <C-I> :echo "world"<CR>
If you want <C-I> and <Tab> to do the same thing, then do this instead:
:nmap <C-I> <Tab>
or this:
:nnore <C-I> :DoSomething<CR>
:nnore <Tab> :DoSomething<CR>
>
> Additionally, TAB, for example, in Japan can be a language input
>
> modifier key for switching between kanji/kana and Roman alphabet.
>
> Some keyboards have an additional dedicated key for this but not
>
> all do. On a remote link you may have to use Ctrl+I to get a tab
>
> character to come out as the tab key locally can be intercepted at
>
> a very low level by Japanese (or other foreign) operating systems.
>
>
>
> I do not believe I am the only one who uses keyboard/mouse remote
>
> control to another system and edits a file in ViM, nor am I the only
>
> one who uses a computer in more than one language and uses VIM
>
> keyboard mappings to circumvent local system inadequacies with
>
> regards to multi-lingual support. However, as I guarantee that most
>
> of these other people don't "hang out" on "#vim" for most of their
>
> lives I take the view that the "vast majority" of ViM users (including
>
> myself) would be annoyed if their keyboard mappings just suddenly
>
> stopped working because of a few who can't see beyond their English-
>
> only QWERTY-compliant wire-connected keyboards.
I'm not sure whether there are more people surprised by the inability to map <C-I> separate from <Tab>, or whether there are more that would be frustrated that they must now manually map <C-I> to <Tab>. But I do know the first seems more logical to fix than the second.
Since the second is the current behavior, then I suppose I'd support either having an option to turn this on (probably part of cpoptions), or better, falling back to the current behavior if no mappings exist.
And yes, I mean NO mappings (of that synonymous pair). Because if you define a mapping for <Tab> and want <C-I> to trigger it, you could define a second <C-I> mapping in the same place. But if you want to keep the default behavior of <Tab> and still want to map <C-I> to something, you should not need to use two mappings to accomplish that.
Bram Moolenaar
It's already implemented that a key-with-modifiers that is not mapped falls
back to the key-without-modifiers. There is no reason to make it more
complicated for the user who hasn't read all the documentation of all
options (hint: nobody has).
--
ARTHUR: What does it say?
BROTHER MAYNARD: It reads ... "Here may be found the last words of Joseph of
Aramathea." "He who is valorous and pure of heart may find
the Holy Grail in the aaaaarrrrrrggghhh..."
ARTHUR: What?
BROTHER MAYNARD: "The Aaaaarrrrrrggghhh..."
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD
Nikolay Pavlov
I have. Though can't say I remember it all.
> --
> ARTHUR: What does it say?
> BROTHER MAYNARD: It reads ... "Here may be found the last words of Joseph of
> Aramathea." "He who is valorous and pure of heart may find
> the Holy Grail in the aaaaarrrrrrggghhh..."
> ARTHUR: What?
> BROTHER MAYNARD: "The Aaaaarrrrrrggghhh..."
> "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD
>
> /// Bram Moolenaar -- Br...@Moolenaar.net -- http://www.Moolenaar.net \\\
> /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
> \\\ an exciting new programming language -- http://www.Zimbu.org ///
> \\\ help me help AIDS victims -- http://ICCF-Holland.org ///
>
> --
> --
> You received this message from the "vim_dev" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_dev" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to vim_dev+u...@googlegroups.com.
Michael Henry
> Michael Henry wrote:
>> So I suggest that a single global option that simply switches on
>> support for extended modifier for all keys, regardless whether
>> those keys are mapped, may well be good enough and might make
>> the implementation simple enough to become reality. The day the
>> option appears in Vim, I'll put it at the top of my .vimrc :-)
>
> So we add an option that breaks mappings? No, not a good plan.
mappings :-) I was thinking of and discussing mostly the :map
functionality of Vim, not (yet) the terminal-handling side, and
talking about behavior for a terminal where the codes for <Tab>
key and <C-I> are distinguishable.
Also, I'm not familiar with all the inner workings of Vim, so
the suggestions below might be laughable to those with better
understanding. So I thought I'd try to clarify it some more.
Worst-case, I figure at least you'll get a laugh :-)
Today, users that have both :map <Tab> and :map <C-I> will find
that the most recently defined map clobbers the other, because
Vim aliases them even in the definition of the mappings. I'd
like to see Vim eliminate the enforced aliasing at this layer
(the :map layer) for several reasons.
I think new users do not find this aliasing intuitive. There is
no intuitive reason why pressing CTRL-I should be the same thing
as pressing Tab, unless you have a priori knowledge of terminal
history.
If a terminal has a separate Tab key with a key code that is
distinguishable from the key code for CTRL-I, I'd like those
keys to remain distinguishable as Vim processes them. As much
as possible, I'd like each key on my keyboard to be uniquely
recognizable to Vim, so that I get maximum value out of these
keys. Personally, I don't get any benefit from the fact that
the Tab key is just another way of pressing CTRL-I. Though
people have found a way to get some benefit out of this aliasing
that is forced upon us by many current terminals, it seems like
a mis-feature for Vim to force it as well. Users that want two
keys to be aliases can give them identical mappings. But users
that want to treat them separately can't do it as long as Vim
aliases them under the hood.
A terminal that sends identical key codes for different key
presses is not as flexible as one that sends unique codes. We
have to deal with terminals as they exist today, which means we
have to handle hardware aliasing of Tab and CTRL-I (for
example). What if Vim were to handle this in the early stages
of terminal input, by decoding a given key code into a set of
keys aliases specific to the terminal's capabilities? So for
this particular terminal, ASCII code 9 would be translated to
the set {<Tab>, <C-I>}. Mappings would match if the mapping's
key is in the set of aliases returned from the terminal layer.
So if the user has mapped only one of <Tab> and <C-I>, Vim will
use that mapping. Vim might search mappings in reverse order of
definition so that a more recent mapping will win out in the
event of a tie. That way, a user can control on a per-mapping
basis which alias will win out in a given mapping mode.
I hope that the suggested changes above would be transparent to
most users, but there would be ways that users could detect
things were different (for example, by setting a mapping for
<C-I> and trying to query that same mapping using <Tab>). I
don't see a way to make it 100% transparent without leaving some
vestige of baked-in aliasing, which I feel should ideally be
removed everywhere except in the early terminal-handling stages;
that way, when we get terminals without inherent aliasing, we
can be rid of the aliasing problem entirely.
So the purpose of the option I was proposing was to
control whether Vim would continue to alias certain key
combinations at the point of mapping.
> It's already implemented that a key-with-modifiers that is not
> mapped falls back to the key-without-modifiers. There is no
> reason to make it more complicated for the user who hasn't
> read all the documentation of all options (hint: nobody has).
no mapping as that same key without modifiers. I'm not sure I
understand how that works, nor why it's considered desirable. I
especially don't see how having that fall-back makes things less
complicated for users that haven't read the documentation. In
my way of thinking, the intuitive behavior for a keyboard is
that two different keys generally perform two different
behaviors (unless they are labeled the same, such as the two
Enter keys on a typical PC keyboard).
If Vim were changed to distinguish key names like <Tab> and
<C-I> throughout most of the code base, and to deal with
terminals that do not have unique key codes by return a set of
aliases for a given key press, I suspect that most users would
not notice any negative effects. If that level of compatibility
were good enough, then we wouldn't need any user-facing option.
My suggestion for an option was only to provide 100%
compatibility for users that want to keep the baked-in
assumption that Tab and CTRL-I (for example) are permanently
identical keys.
Michael Henry
Paul LeoNerd
Michael Henry <v...@drmikehenry.com> wrote:
> Sorry I was not clear. I certainly wasn't trying to break all
> mappings :-) I was thinking of and discussing mostly the :map
> functionality of Vim, not (yet) the terminal-handling side, and
> talking about behavior for a terminal where the codes for <Tab>
> key and <C-I> are distinguishable.
We already have many supported ways to run vim where vim itself /can/
distinguish a <Tab> from a <C-I> (i.e. any of the GUI-related gvims).
It would be nice if two maps in a gvim could distinguish properly.
Once vim's internals can correctly keep these and all the other similar
cases separate /then/ we can consider adding terminal recognition for
those terminals that can represent it; but that's a second-stage.
You don't need me, or a fancy terminal library to do that. You already
have GTK, Win32, etc... GUI input support. You just need to fix vim's
internals.
Michael Longval
I would like to add my voice to the request for an overhaul of Vim's key-input-handling.
Precisely my problem is with Ctrl-i, (like most everyone who complains about this problem).
I am using Vim in my EMR (Electronic Medical Record) system. Although I have been using Vim for a long time, I still have not gotten used to the hjkl directional mapping. (I am 50, so I reserve the right to be cranky about stuff...)
I have modified my .vimrc to give me an inverted-T in normal mode (jkl with i on top), but I want to be able to move around in insert mode with corresponding Ctrl- modified map.
We all know where this goes......
As a tool Vim is very impressive, fast and infinitely customizable.
It is however hamstrung to a certain degree by it's origins. (I don't think it was bad engineering... it had to be made to work with the technology of the moment.)
I think that if Vim were to be "fixed" (some will say it isn't broken... I disagree) it would be used by many more people.
To all those who have contributed to Vim, I say "Thank You".
We all just want to make it a better tool.
Michael Longval
/#!/JoePea
Wow, cool!! I thought I was the only one using the IJKL inverted T!!! And my H key is the insert key. I'm 28. :D I've inoremapped mine to move the cursor around while alt is pressed, word by word with Ctrl, and up and down 10 lines at a time with Ctrl. I've also mapped many other keys to IJKL, e.g. H for text object, moving between windows after pressing <c-w> uses IJKL, IJKL instead of HJKL after pressing Z in normal, etc. Basically every use case. I've got a couple left to do when I find some spare time for it. I need to redo them all non-recursively. IJKL makes so much incredible sense.
But yeah, I use Ctrl+i to move the cursor, so pressing tab moves the cursor, which is annoying, so I've gotten used to Ctrl+n for autocompletion and Ctrl+t for tabbing, but as soon as this issue is fixed, I will gladly put my tab key back into use for tabbing and autocompletion.
/#!/JoePea
--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
---
You received this message because you are subscribed to a topic in the Google Groups "vim_dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vim_dev/2bp9UdfZ63M/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vim_dev+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Bram Moolenaar
imap <C-I> <Up>
imap <C-J> <Left>
imap <C-k> <Down>
imap <C-l> <Right>
It does require gvim, since a terminal doesn't make a difference between
Tab and CTRL-I.
Perhaps you were complaining about terminal emulators?
--
If evolution theories are correct, humans will soon grow a third
hand for operating the mouse.
Michael Longval
My guess is that there are lot of heretics like you and I.
Cheers!
Michael Longval, MD
Michael Longval
My EMR is in Python3 and I run it in iTerm2 on MacOS (+Homebrew).
I could use Macvim but then I'd be going between two windows. (PITA)
There is no way to make Macvim the host (display) for my Python program? That way I (might) be able to just use Macvim.
BTW : thanks for your great work and dedication to this fine tool!
Michael Longval, MD
Michael Longval
I replied by Email and your email addresses were quoted in the reply.
I erased my email replies and reposted without the email addresses.
Mike
(.... writes another "THINGS NOT TO DO" in his little moleskine notebook...)
/#!/JoePea
Ingo Karkat
> Not sure what your problem is. This works just fine:
>
> imap <C-I> <Up>
>
> Tab and CTRL-I.
what's causing so many people grief, and is the motivation for
redesigning Vim's input handling.
,----[ demo ]----
| :imap <C-I> C-I
| :imap <Tab> Tab
| :imap <C-I>
| i <Tab> Tab
`----
-- regards, ingo
/#!/JoePea
Paul "LeoNerd" Evans
"/#!/JoePea" <tru...@gmail.com> wrote:
> Hmmm, yep. I just tested. gvim and MacVim both don't differentiate
> tab and ctrl_i!
>
> On Sat, Oct 4, 2014 at 12:34 PM, Ingo Karkat <sw...@ingo-karkat.de>
> wrote:
>
> > On 04-Oct-2014 15:43 +0200, Bram Moolenaar wrote:
> >
> > > Not sure what your problem is. This works just fine:
> > >
> > > imap <C-I> <Up>
> > >
> > > It does require gvim, since a terminal doesn't make a difference
> > > between Tab and CTRL-I.
now. You keep deflecting this down to make it sound like the terminal's
fault, when we both know it isn't.
You and I both know full well that terminals don't distinguish them; I
accept that. That's why I designed a better system, in cooperation with
Thomas Dickey (the current xterm author). I have a terminal now that
distinguishes any and all possible combinations of keypresses, and
programs that understand it. Most of the programs I run regularly now
do understand this - I can type Ctrl-I and Ctrl-Shift-A and so on
absolutely fine. Vim is one of the few programs remaining that doesn't.
(see attached screenshot-1).
Vim - I am talking specifically about vim here - conflates the possible
keypresses of Ctrl-I vs Tab, of Ctrl-M(or is it Ctrl-J I forget) and
Enter, of Ctrl-H and Backspace. It further conflates Ctrl-S and
Ctrl-Shift-S, etc etc... And lets not get started on Unicode vs.
Alt-letters.
Blaming terminals for this is just deflecting from the fact that vim's
internals aren't sufficiently generic to represent the possible
keypresses, regardless of how they arrive. That 1980s-style terminals
couldn't do it is one thing but that is no excuse that a 2014-style
GTK/Win32-driven GUI program cannot.
You cannot reply to the original poster of this email and claim that it
works, until you can perform the following test IN GVIM to demonstrate
it so.
:imap <Tab> You typed tab
:imap <C-i> You typed Ctrl-I
:imap <C-S-I> You typed Ctrl-Shift-I
Then press all three keypresses and show it inserting different text.
Do this in gvim, so as to avoid any reason to blame the terminal. For
me, right now in GTK2-driven gvim, I get the Ctrl-Shift-I version all
three times. (see attached screenshot-2)
Only when that works can you reply to the OP and say "this works".
http://www.leonerd.org.uk/ | https://metacpan.org/author/PEVANS
Paul "LeoNerd" Evans
Bram Moolenaar <Br...@moolenaar.net> wrote:
> Perhaps you were complaining about terminal emulators?
But more than that I've got out and actually tried to do something to
fix them in this regard. I worked with Thomas Dickey to design a new
scheme for universally encoding any modified keypress, Unicode-printing
or special, on a terminal.
http://www.leonerd.org.uk/hacks/fixterms/
To be clear - this is a small extension of the existing scheme that
remains 100% backward compatible - any keypress that could be detected
before, is still sent the same way. This just gives a new way to encode
new keypresses that previously did not make sense.
I then went and wrote a virtual terminal emulator library and
GTK2-based wrapper program that uses it
http://www.leonerd.org.uk/code/libvterm/
http://www.leonerd.org.uk/code/pangoterm/
I then wrote a keyboard input library and surrounding terminal UI
system that uses it
http://www.leonerd.org.uk/code/libtermkey/
http://www.leonerd.org.uk/code/libtickit/
All that remains to be done is for programs that attempt to interpret
incoming keypresses be suitably adjusted so as to understand these new
incoming byte sequences, and all will be sorted, and then finally OP
will be able to type Ctrl-I properly, as was his want.
I have been talking with the Neovim project on the subject of a modern
UI toolkit for use in the terminal frontend. Whether or not they end up
using libtickit for this, they are still aware of the improvements and
modernisations I have been working on, and when the time comes for them
to rebuild the terminal frontend properly I'm quite confident they'll
be in a position to support all these modified keypresses. If you
wanted the same support in core vim, all you'd need to do is implement
the key encoding scheme, as given in the first link.
I am done complaining. I am done fixing it. As far as I'm concerned it
is now fixed. All that remains is for other people to come and join the
party.
Will you be one of them?
Ingo Karkat
(especially newcomers to Vim), here's an updated tally of related
questions that regularly come up on Stack Overflow and related sites:
https://www.google.com/search?q=site%3Astackoverflow.com%20%2B%22Ingo%20Karkat%22%20%22foremost%20Paul%20LeoNerd%20Evans%22
(about 32 results)
https://www.google.com/search?q=site%3Asuperuser.com+%2B%22Ingo+Karkat%22+%22foremost+Paul+LeoNerd+Evans%22
(6 results)
I'm happy to see that Paul is still pursuing this issue; Bram, why don't
you get him and other developers finally started on designing and
implementing a solution by briefly signaling a willingness to consider
this for a future Vim 8.0?!
-- regards, ingo
Bram Moolenaar
without breaking lots of things. Don't forget that users today rely on
CTRL-I doing the same as Tab. Only when they are mapped separately
should the mean something different.
Your page doesn't say how to switch between the old mode, where CTRL-I
produces a 0x09 character, and one where it produces a different
character. This is required for the terminal to be used with programs
that do not support the new codes. One can't expect to have all
programs that a user uses to suddenly accept the new characters, thus a
switch between two modes is required. Vim could switch to the new mode
and take care of a default set of mappings to the old meaning. That's a
lot of work though.
Also, I don't see any note about different language keyboards. There
are many, and the mechanism should work for all, with proper
documentation what happens for different keyboards. Also, I don't see
anything for keypad keys, the numlock key and other keys that some
keyboards have that change what other keys mean.
--
hundred-and-one symptoms of being an internet addict:
226. You sit down at the computer right after dinner and your spouse
says "See you in the morning."
Bram Moolenaar
cause more trouble than it solves, and is very limited otherwise. In
other words: it is not a good solution.
We could do something for the GUI, I thought there was a todo item for
that already. Can't find it now... The idea basically is that when Tab
and CTRL-I are both mapped, they will be considered to be different.
This is required for existing mappings to keep working. It will still
break when someone tries to overrule a mapping for Tab with a mapping
for CTRL-I though. Hopefully that doesn't happen very often.
--
hundred-and-one symptoms of being an internet addict:
Michael Longval
**Please remember IANAVD (I am not a Vim Dev)**
Isn’t there something to learn from Python.
When Python 3 came out, it broke stuff, lots of it.
But the advantages were worth it. (IMHO)
Some people had too much invested in Python 2.6 however, so they stayed with that (and bug fixes continu on that version, but the new stuff is in Python3)
Vim broke with Vi (see: set nocompatible) yet was successful.
Vim8 with a 21st century input mechanism, can do the same.
Road map:
*Move official Vim8 (call it V8) onto GitHub.
*Set design goals:
- Keep it CharityWare (above all else)
- New input engine
- Wall to wall UTF-8 support
- Wall to wall Python3 scripting.
See who comes to work on it.
Yes anyone who reads this will say: “Ya right sounds easy, but it’s way harder than that.”
I have NO ARGUMENT with that.
BUT my guess is that if a guy with the authority of Bram Moolenaar says: “We are going to break Vim to make it better for the future.”, people will listen. (Many who are much younger than both you and I, Bram, and have a lot more energy.)
Respectfully,
Michael LONGVAL, MD
Paul "LeoNerd" Evans
Bram Moolenaar <Br...@moolenaar.net> wrote:
> This hack means backwards compatibility is dropped, it can't be
> included without breaking lots of things. Don't forget that users
> today rely on CTRL-I doing the same as Tab. Only when they are
> mapped separately should the mean something different.
differently to Tab.
> Your page doesn't say how to switch between the old mode, where CTRL-I
> produces a 0x09 character, and one where it produces a different
> character. This is required for the terminal to be used with programs
> that do not support the new codes. One can't expect to have all
> programs that a user uses to suddenly accept the new characters, thus
> a switch between two modes is required.
and default off until some application wanted to enable it. I'll give
that some thought - hopefully Thomas and I can find a suitable mode
number for it.
In any case, I suspect that's going to be a minor cornercase of the
specific keys of Ctrl-I, Ctrl-H and Ctrl-J overlapping with
Tab/Backspace/Enter. The remaining ones, like Ctrl-Shift-A, Alt-letter,
etc... are already key combinations that people won't be typing unless
they mean it. And lets not forget that already for years, terminals
have been sending such key events like Ctrl-Shift-Up as CSI 1;6 A and
thus confusing most input systems pre-existing. Already vim handles
these with the * hack. I'd love to see the "if $TERM==xterm then use
the * hack" code replaced with a nice proper generic CSI parser, which
will then understand these sequences.
If nothing else, vim should at least recognise an incoming
CSI 65;5u
when it sees it as being Ctrl-A, rather than its current behaviour of
getting confused, beeping, leaving insert mode, repeating the last t
motion 65 times, then undoing my last 5 changes. That isn't helpful in
the slightest.
Even if you do nothing else from this discussion, I wouldn't mind if
you made vim handle these CSI encodings a little better - undoing my
last 5 changes is never what I wanted to happen :) There's surely no
danger in interpreting these incoming sequences correctly /if/ you
happen to see them - whether or not you see them is then up to the
terminal.
> Vim could switch to the new
> mode and take care of a default set of mappings to the old meaning.
> That's a lot of work though.
the new to the old, if that would work.
> Also, I don't see any note about different language keyboards. There
> are many, and the mechanism should work for all, with proper
> documentation what happens for different keyboards. Also, I don't see
> anything for keypad keys, the numlock key and other keys that some
> keyboards have that change what other keys mean.
change the number keypad between cursor/application sense, and plain
number sense. In effect, with numberlock on the numberpad should act
identically to the regular number keys; with it off it should act
identically to the cursor control keys.
As to other language keyboards I'm not quite sure what concern you have
here. Other language layouts add new Unicode symbols that wouldn't
otherwise be accessible - this scheme already copes just fine with
Unicode. E.g. if you now find yourself with a real Ä (U+00e4 / U+00c4)
key, that's no problem. U+00c4 is 196 decimal. Ctrl-Ä, for example, is
then represented by CSI 196;5u. Does this answer your concern, or is
there still something remaining here?
Paul "LeoNerd" Evans
Bram Moolenaar <Br...@moolenaar.net> wrote:
> We could do something for the GUI, I thought there was a todo item for
> that already. Can't find it now... The idea basically is that when
> Tab and CTRL-I are both mapped, they will be considered to be
> different. This is required for existing mappings to keep working.
> It will still break when someone tries to overrule a mapping for Tab
> with a mapping for CTRL-I though. Hopefully that doesn't happen very
> often.
Doubly-so would be if, even on a terminal, I could write one of my many
hacky 'map the byte sequences' fixes, to teach vim how to read the
incoming keypresses.
I.e. if I could then
:map ^[[105;5u <Ctrl-i>
:map ^[[73;5u <Ctrl-I>
then at least vim would recognise that I had typed a Ctrl-i, instead of
undoing my last 5 changes. I mean, I could write that map right this
very second, but without the feature named above I'd have nothing to
map it /to/ on the RHS.
On a more general note though, I find the idea of using :map on byte
sequences to teach vim about new keypresses that aren't programmed in
to be unsatisfactory. It gets upset with timing, and it doesn't apply
during 'paste mode. Would it be possible to define a more robust
mechanism for doing these kinds of fixes?