serious keyboard events key codes inconsistencies
Vadim Zeitlin
I spent almost the entire day yesterday working on getting the keyboard
event generation for MSW and GTK in sync but unfortunately I'm still far
from the goal (and this is without mentioning wxOSX/Cocoa...). While I
could fix some bad problems and will fix the 2 other ones I posted
questions about when it becomes clear what the consensus is, there is one
big problem that I simply don't know what to do about.
It concerns the key codes in key events. We had already noticed this
problem before when someone complained that the "+/=" key generated key
events with code '+' while under GTK it generated '='. This was discussed
5.5 years ago in this thread
http://thread.gmane.org/gmane.comp.lib.wxwidgets.devel/55266/
but without reaching any real conclusion and nothing was done about it.
However I think we underestimated the severity of the issue because we
already have just about all problems that were mentioned in the thread
above. Notably, Stefan wrote that it would be crazy to generate a key event
with a key code not present on the key at all. But, of course, we do it for
many keys already: just not on the US keyboard. With the German keyboard
layout you get ';' key events for the "ü/Ü" key for example. Not only ';'
isn't present on this key, it's *not* the same key as the one that has ';'
in US layout neither. Rather, it's the '[/{' US keyboard key which
generates ';' key events on a keyboard with German layout under MSW. And
this is not an exception at all, e.g. events with '~' key code are
generated for the physical ';/:' key which is 'ö/Ö' in German layout and so
on. Of course, just to make things a bit more confusing, some keys do
overlap with their key codes: for example, ']/}' key generates events with
'+' code and it is indeed labeled '+/*'. And '\'/"' key (I use backslash
for C notation, I mean the one to the left of Enter does generate key codes
with '\'' key code even though it's labeled 'ä/Ä'. But in general there is
no relationship whatsoever between the key code in the event and the key on
the keyboard.
But wait, perhaps you thought this was a MSW-only problem. To check this I
installed German keyboard layout on my bog standard (Debian) Gnome. There
pressing the ']/}' key, which is labeled as '+/*' in German, results in key
events with '=' key code!! I guess I can understand this: '=' is '+' and
'+' is on this key but I can't seriously claim it makes any sense. wxGTK
seems to be even more broken than wxMSW because pressing Shift with this
key results in key events with '8' key code which breaks the invariant that
the key code for key events doesn't depend on Shift state. And, again, it's
not an exception but the rule, e.g. '|/\' key generates key events with '3'
key code...
To summarize, the key codes currently are simply useless for non-letters
(and maybe for letters only and I just didn't find a keyboard layout
example showing it). You simply can't guarantee anything about them:
neither that the key code they carry is actually accessed by this key nor
that the key pressed is in the same position as the key with the given key
code in the US layout nor even that you get the same key code for pressing
a key with and without Shift.
So basically nothing you can do with them is going to work correctly on
non-US keyboards. The question is: can we do something to make them useful?
And for this, I guess, we need to understand what exactly are these events
useful for? Does anybody have any ideas?
Thanks,
VZ
Carsten Fuchs
On 06.09.2010 12:47, Vadim Zeitlin wrote:
> To summarize, the key codes currently are simply useless for non-letters
> (and maybe for letters only and I just didn't find a keyboard layout
> example showing it). You simply can't guarantee anything about them:
> neither that the key code they carry is actually accessed by this key nor
> that the key pressed is in the same position as the key with the given key
> code in the US layout nor even that you get the same key code for pressing
> a key with and without Shift.
>
> So basically nothing you can do with them is going to work correctly on
> non-US keyboards. The question is: can we do something to make them useful?
> And for this, I guess, we need to understand what exactly are these events
> useful for? Does anybody have any ideas?
To my understanding, the only payload that key up/down events (as opposed to char events) can and
should carry is the "raw" OEM scancode. That scancode is basically just a number that describes more
or less the physical position of the key on the keyboard, or just an abstract/opaque number that
identifies the key, but has no additional meaning. In particular, only the OS (but not the
application) is able to convert a scancode to a meaningful character, which the app receives in form
of a character message.
(Insofar I've always been somewhat astonished by design of the wxWidgets library to (attempt to)
associate a character value to key up/down events...)
The typical case where such events are useful (and in fact, without alternative), is whenever the
keyboard is not used for entering text, but just as an input device with (many) buttons. One typical
example is the camera control in any 3D application, or when the keys are used as piano keys, etc.
Under Windows, I used long years the following code for having my own key up/down event system. The
details are not important, but please note the comment in the middle:
LRESULT CALLBACK WinProc(HWND hWnd, UINT MessageID, WPARAM wParam, LPARAM lParam)
{
bool IsKeyDown=false;
switch (MessageID)
{
// ...
case WM_KEYDOWN:
case WM_SYSKEYDOWN: IsKeyDown=true; // Intentional fall-though here.
case WM_KEYUP:
case WM_SYSKEYUP:
{
// Für lParam vgl. Petzold S. 227 und 215.
CaKeyboardEventT CKE;
CKE.Type=IsKeyDown ? CaKeyboardEventT::CKE_KEYDOWN : CaKeyboardEventT::CKE_KEYUP;
CKE.Key =(lParam >> 16) & 0xFF; // Convert the OEM scancode into a CK_* key type.
// I've made some experiments, i.e. printing out the value of CKE.Key for all sorts of
keys.
// It turns out that the upmost (8th) bit of the OEM scancode (23rd bit in lParam) is
// apparently never used, but instead the flag for extended keys (24th bit in lParam)
// is set for some keys.
//
// Additionally, if I set the 8th bit whenever the extended flag is set,
// it turns out that the result exactly matches the DirectInput DIK_ constants list!
//
// Also see Petzolds description about the "lParam" value for WM_*KEY* messages
// at pages 215-217, especially the sections about "OEM-Scancode" and
// "Flag für erweiterte Tasten"!
if ((lParam & (1 << 24))!=0) CKE.Key|=0x80;
// Store the event in the event queue.
if (SingleWin.CKE_Queue_Size<CKE_QUEUE_MAX)
{
SingleWin.CKE_Queue[(SingleWin.CKE_Queue_Start+SingleWin.CKE_Queue_Size) &
(CKE_QUEUE_MAX-1)]=CKE;
SingleWin.CKE_Queue_Size++;
}
break; // Intentionally break, not return 0, so that the DefWindowProc() is called.
}
case WM_CHAR:
{
// ...
return 0;
}
// ...
}
return DefWindowProc(hWnd, MessageID, wParam, lParam);
}
An application or game that uses key up/down codes for camera control, usually has to present a user
interface to make keys configurable, and when the user wants to have key down event for key "123"
associated with camera "move forward", then the app can only let the user know that it uses key
"123" for camera "move forward" control. It generally cannot print what symbol is physically printed
on that key on the users keyboard (it may guess though, e.g. based on US keyboard layout, or based
on a "nearby" char event, etc., but no guarantees (and none are needed)).
In summary, if it was up to me, I would stop trying to assign the code in the key up/down events
more meaning than can be assigned. In other words, let's have *different* numbers (and datatypes)
for key up/down and character events!
I would even go as far and derive the key code that wx associates with its key up/down events with
the same code as above:
KeyCode =(lParam >> 16) & 0xFF;
if ((lParam & (1 << 24))!=0) KeyCode|=0x80;
because that not only matches DirectInput, but also (most likely) the result that the equivalent
BIOS interrupt back in the old days produced. With the same argument, I assume (but really cannot
prove) that that is even portable at least to wxGTK: For information, see my X code in the above
scenario at http://trac.cafu.de/browser/cafu/trunk/Libs/OpenGL/OpenGLWindow.cpp?rev=150#L696
Best regards,
Carsten
--
Cafu - the open-source Game and Graphics Engine
for multiplayer, cross-platform, real-time 3D Action
Learn more at http://www.cafu.de
Vadim Zeitlin
CF> To my understanding, the only payload that key up/down events (as
CF> opposed to char events) can and should carry is the "raw" OEM scancode.
This is logical but this is not at all the way they work currently. E.g.
'Q' key (on the US keyboard) generates key code 'Q' but it generates 'A'
with French keyboard layout because this is how it's labeled there. And
this works well because your Ctrl-A accelerators and such still work just
as expected. The position of the key doesn't really matter in this case,
what matters is that 'a' and 'A' (or 'q' and 'Q') are always on the same
key. And for the layouts that don't have English letters at all (e.g.
Russian) 'Q' reverts to being just 'Q' which is expected and correct again
because Ctrl-Q shortcuts actually are expected to work like this on systems
with Russian locale.
So while we really need to keep the key events working as expected for all
alphabetic keys. There they're (relatively) well defined and work as
expected and we shouldn't break this. Troubles start with the characters
that are not always on the same key.
CF> That scancode is basically just a number that describes more or less
CF> the physical position of the key on the keyboard, or just an
CF> abstract/opaque number that identifies the key, but has no additional
CF> meaning. In particular, only the OS (but not the application) is able
CF> to convert a scancode to a meaningful character, which the app receives
CF> in form of a character message.
This is not quite correct, both Windows and X do provide ways to translate
scan codes to characters.
CF> (Insofar I've always been somewhat astonished by design of the
CF> wxWidgets library to (attempt to) associate a character value to key
CF> up/down events...)
If you're speaking about Unicode character value for it, I agree. But a
key event must have a key code to be useful. The question is how to define
it.
CF> The typical case where such events are useful (and in fact, without
CF> alternative), is whenever the keyboard is not used for entering text,
CF> but just as an input device with (many) buttons. One typical example is
CF> the camera control in any 3D application, or when the keys are used as
CF> piano keys, etc.
Again, you are correct, such cases do occur and scan codes should indeed
be used for them. They're probably a minority though and even if they're
not, I'm quite sure they don't cover 100% of key events usage. In any case,
in this situation you should just use scan code (which is available as raw
key code in wxKeyEvent) and while we could improve the situation by having
some portable scan codes translation, it can be already done.
My question is about what to do in all the other cases. E.g. suppose you
define '+' and '-' as accelerators for zooming in and out. It makes sense
to use key events here because you may want to do something when the key is
released. Currently there is no way to do it with wxWidgets.
CF> Under Windows, I used long years the following code for having my own
CF> key up/down event system. The details are not important, but please
CF> note the comment in the middle:
I think it just restates MSDN description of LPARAM of keyboard events. In
particular the bit 24 is known as KF_EXTENDED in Windows headers.
Anyhow, to summarize, using scan codes is certainly possible but it's not
the same as key codes and we can't change this because of compatibility
concerns: at least for letters the key codes work reasonably well and we
shouldn't break this.
The question is whether there is any useful way to define key codes for
the rest of alphanumeric keys (because for things like cursor arrows and
such there are no problems neither, at least until someone defines a layout
changing them...) or if we should just give up and say that their value is
completely undefined -- which would nicely describe the current state of
affairs.
Regards,
VZ
Robert Roebling
Hi,
on my Linux with a German keyboard layout, I get "roughly" the
keycodes (when looking at the output in the text/samples) as
I would expect:
Keycode for "A" is "A" (no matter which control key I press)
Keycode for "=0}" is "0" (no matter which control key I press)
Keycodes for NUM keys is NUM_END or NUM1 depending on whether or
not I press SHIFT - and I don't know if that is intended.
The test code currently asserts for German Umlaute and this needs
to be looked at further, but in all this doesn't look wrong to me.
You didn't somehow forget to activate the locale/keyboard layout
or something like that?
Vadim Zeitlin
RR> on my Linux with a German keyboard layout, I get "roughly" the
RR> keycodes (when looking at the output in the text/samples) as
RR> I would expect:
RR> Keycode for "A" is "A" (no matter which control key I press)
This is right, letters do work as expected, both under Windows and Linux.
The problems only happen for non-special non-letter keys.
RR> Keycode for "=0}" is "0" (no matter which control key I press)
Well, this is one of the keys that doesn't change too much in German
layout. So it's still '0' just as in the US one. But try '+/*' key (to the
left of Enter, as shown at http://www.aufgehts.com/sup_Germankeyboard.html).
Or the other keys I mentioned.
RR> Keycodes for NUM keys is NUM_END or NUM1 depending on whether or
RR> not I press SHIFT - and I don't know if that is intended.
I didn't think it was but it seems to behave the same under Windows
(WXK_NUMPAD7 without Shift, WXK_NUMPAD_HOME with it) so maybe it is. In any
case, it's yet another issue to be at least documented.
RR> The test code currently asserts for German Umlaute and this needs
RR> to be looked at further, but in all this doesn't look wrong to me.
Sorry, which test code and where does it assert? I'm confused because I
didn't check my new keyboard unit test in yet...
RR> You didn't somehow forget to activate the locale/keyboard layout
RR> or something like that?
I simply went to "Keyboard settings" in Gnome and added a couple of
layouts, then clicked the icon in the taskbar to change them. Everything
works as intended everywhere, e.g. I can type German text in a text editor
and so on. So I really don't think there is anything wrong with my system
setup, rather I think you just didn't try the problematic keys. Once again,
try '+/*' and '\'/#' ones. The key codes they produce make no sense at all.
The question is: can we define a rule for the key codes they should
generate?
Regards,
VZ
Robert Roebling
Vadim Zeitlin wrote:
> On Mon, 06 Sep 2010 18:41:57 +0200 Robert Roebling wrote:
>
> RR> on my Linux with a German keyboard layout, I get "roughly" the
> RR> keycodes (when looking at the output in the text/samples) as
> RR> I would expect:
> RR> Keycode for "A" is "A" (no matter which control key I press)
>
> This is right, letters do work as expected, both under Windows and Linux.
> The problems only happen for non-special non-letter keys.
>
> RR> Keycode for "=0}" is "0" (no matter which control key I press)
>
> Well, this is one of the keys that doesn't change too much in German
> layout. So it's still '0' just as in the US one. But try '+/*' key (to the
> left of Enter, as shown at http://www.aufgehts.com/sup_Germankeyboard.html).
> Or the other keys I mentioned.
I think I tested them all now
"+*~" always yields "+" is I expected
"#'" yields "#" without SHIFT and something else with SHIFT (the latter
is wrong)
"<>|" ",;" ".:" "-_" all yield the values wihout SHIFT and thus those
which I expected
>
> RR> Keycodes for NUM keys is NUM_END or NUM1 depending on whether or
> RR> not I press SHIFT - and I don't know if that is intended.
>
> I didn't think it was but it seems to behave the same under Windows
> (WXK_NUMPAD7 without Shift, WXK_NUMPAD_HOME with it) so maybe it is. In any
> case, it's yet another issue to be at least documented.
>
> RR> The test code currently asserts for German Umlaute and this needs
> RR> to be looked at further, but in all this doesn't look wrong to me.
>
> Sorry, which test code and where does it assert? I'm confused because I
> didn't check my new keyboard unit test in yet...
As I mentioned: in samples/text
Robert
Vadim Zeitlin
RR> I think I tested them all now
RR> "+*~" always yields "+" is I expected
I have really no idea why does it work for you and not for me... What is
your system?
RR> "#'" yields "#" without SHIFT and something else with SHIFT (the latter
RR> is wrong)
RR>
RR> "<>|" ",;" ".:" "-_" all yield the values wihout SHIFT and thus those
RR> which I expected
So you think that the key code should always be the unshifted character
produced by the key? I think I should be able to make MSW behave like this
but what do we do with the "ß/?" key then? There is no key code for "ß" so
should we use "?" for this one? But then what about "ü/Ü"?
RR> As I mentioned: in samples/text
I missed the mention of this, sorry. Could you test samples/keyboard too
please? Maybe something is wrong with the sample itself (e.g. it uses
Unicode character of the event instead of the key code maybe)?
Thanks,
VZ
Robert Roebling
> On Mon, 06 Sep 2010 19:07:36 +0200 Robert Roebling wrote:
>
> RR> I think I tested them all now
> RR> "+*~" always yields "+" is I expected
>
> I have really no idea why does it work for you and not for me...
> What is your system?
A totally boring openSUSE 10.3 - maybe it's a more recent
GTK+ version that causes your trouble.
>
> RR> "#'" yields "#" without SHIFT and something else with SHIFT (the latter
> RR> is wrong)
> RR>
> RR> "<>|" ",;" ".:" "-_" all yield the values wihout SHIFT and thus those
> RR> which I expected
>
> So you think that the key code should always be the unshifted character
> produced by the key?
Yes.
> I think I should be able to make MSW behave like this
OK.
> but what do we do with the "ß/?" key then? There is no key code for "ß" so
> should we use "?" for this one? But then what about "ü/Ü"?
I guess we shouldn't try to be smarter than the world and just send
some WXK_UNKNOWN or WXK_OTHER in these cases. We can support the
capitalized ASCII values and the most common extra signs found on
keyboards.
Robert
Robert Roebling
> Could you test samples/keyboard too please?
I get the same results there.
Robert
Vadim Zeitlin
RR> > On Mon, 06 Sep 2010 19:07:36 +0200 Robert Roebling wrote:
RR> >
RR> > RR> I think I tested them all now
RR> > RR> "+*~" always yields "+" is I expected
RR> >
RR> > I have really no idea why does it work for you and not for me...
RR> > What is your system?
RR>
RR> A totally boring openSUSE 10.3 - maybe it's a more recent
RR> GTK+ version that causes your trouble.
Still no idea about why am I getting completely different results, it's
just a mystery. A bit sad that nobody else could be bothered to test this.
RR> > So you think that the key code should always be the unshifted character
RR> > produced by the key?
RR>
RR> Yes.
Ok, I've implemented this for MSW. This actually made the code simpler as
we can just use the system MapVirtualKey() function instead of having to
define the key codes of all VK_OEM_XXX keys ourselves.
It now seems to work as expected in all the keyboard layouts I tried (EN,
RU, FR, DE) and as a nice side effect finally returns the more logical '='
key code for the '+/=' key even in the US layout.
RR> > but what do we do with the "ß/?" key then? There is no key code for "ß" so
RR> > should we use "?" for this one? But then what about "ü/Ü"?
RR>
RR> I guess we shouldn't try to be smarter than the world and just send
RR> some WXK_UNKNOWN or WXK_OTHER in these cases.
We send WXK_NONE (0) as the key code now and return the character code in
wxKeyEvent::m_uniChar.
I think this is about as good as it gets, if I wasn't still worried about
the crazy values I'm getting under Linux, I'd almost be happy. Ah, and if
OS X generated the correct key events too, of course...
Regards,
VZ