Questions about keyboard handling in notebook

[Brian Granger]

Hi all,

I am here in NYC working with Jason, Chris and Sylvain this month.
This past week we started to talk about and do design work on the
keyboard shortcuts/handling for the notebook. We have uncovered a
significant design choice we are going to have to make related to
this. It is difficult choice that is cluttered by a rats nest of
standards and partial implementations. I will try to do my best in
describing the top-level choice here.

tl;dr - we can make our keyboard shortcuts magically work for all
users, regardless of their keyboard layout, but users of US English
keyboards will be likely frustrated by an awkward case-dependent
syntax they haven't seen before.

We have two main options for handling keyboard events and mapping
those to keyboart shortcuts:

1) .keyCode

The keyCode attribute of keyboard events provides an integer number
that is associated with the physical key the user pressed. We
currently do all of our keyboard handling and shortcuts using keyCode.

Pros:

* It matches how users think about keyboard shortcuts. When they see
ctrl-shift-A, they know exactly which keys to press.
* The actual shortcut syntax is case insensitive because all of the
case of handled by the shift modifier. Thus ctrl-shift-a and
ctrl-shift-A are the same thing.

Cons:

* Browsers do not provide a JavaScript API that can map to the
characters the keyCode maps to on a given keyboard.
* We basically have to assume US English keyboard.
* Keyboard shortcuts won't work for users of international keyboards
out of the box. They can customize them, but it is really nasty.

2) .key

The .key attribute of keyboard events is a newer standard that
provides the *character* that the particular combination of keystrokes
produces. If a user presses "ctrl+shift+a" the .key attribute would be
"A".

Pros:

* Keyboard shortcuts will work out of the box for International users.
They may have to type a different combination of keystrokes to get the
character, but if they can type the char, they will get the shortcut.
*

Cons:

* Chrome has not implemented .key yet, so we will have to assume US
English layout on Chrome for now.
* If a user doesn't know how to type the character on the keyboard
layout they are using, they won't be able to issue the shortcut. They
would have to change the char.
* US English keyboard users will be confused. "ctrl+a" and "ctrl+A"
will be different shortcuts because all of the modifiers used to
produce the character are hidden and embedded in the char itself. It
is not possible to figure those modifiers out in an international
friendly manner. Another example, it would be "ctrl+_" rather than
"ctrl+shift+-"

Thoughts?

Cheers,

Brian




--
Brian E. Granger
Associate Professor of Physics and Data Science
Cal Poly State University, San Luis Obispo
@ellisonbg on Twitter and GitHub
bgra...@calpoly.edu and elli...@gmail.com

[Scott Sanderson]

 US English keyboard users will be confused. "ctrl+a" and "ctrl+A" 
will be different shortcuts because all of the modifiers used to 
produce the character are hidden and embedded in the char itself. 

This doesn't seem like especially confusing or surprising behavior.  This is more or less how Emacs handles keybindings, for example.   Modifier keys (CTRL, META, and SUPER) are written as prefixes, but the main key itself is case-sensitive, and SHIFT is not considered a modifier key.

For example, I have separate bindings in Python mode for "CTRL-c followed by lower-case n" and "CTRL-c followed by upper-case N":

(define-key python-mode-map (kbd "C-c n") 'nose-trace)

(define-key python-mode-map (kbd "C-c N") 'nosetests)

My vote would be for option 2.

[Matthias Bussonnier]

On Oct 10, 2015, at 16:55, Scott Sanderson <ssand...@quantopian.com> wrote:

 US English keyboard users will be confused. "ctrl+a" and "ctrl+A" 
will be different shortcuts because all of the modifiers used to 
produce the character are hidden and embedded in the char itself. 

This doesn't seem like especially confusing or surprising behavior.  This is more or less how Emacs handles keybindings, for example.   Modifier keys (CTRL, META, and SUPER) are written as prefixes, but the main key itself is case-sensitive, and SHIFT is not considered a modifier key.

For example, I have separate bindings in Python mode for "CTRL-c followed by lower-case n" and "CTRL-c followed by upper-case N":

(define-key python-mode-map (kbd "C-c n") 'nose-trace)

(define-key python-mode-map (kbd "C-c N") 'nosetests)

My vote would be for option 2.

I have no opposition to option2, either.

Though I’m curious to see it working on international keyboard without issues,

I really doubt that it will work out of the box, with all the edge cases. 

Though I will be **really** happy to see it working.

From memory the issues we encountered:

 ~ (titlda)  is **only** combining char on some french keyboard, which mean that ~ shortcut is actually Alt-N,Space press.

I’m not sure how this would handle this kind of key, and | (pipe) on mac-french is Alt-Shit-L with

Alt that do not register an even in some browser because of I don’t remember which native shortcut.

IIRC, attaching things to key instead of keypress was broken on some east-european

keyboard where it became impossible to to type - (minus) in a code cell without triggering a

split cell shortcut.

I’m also curious of what "assume US English layout” mean, in case where the physical mapping of the keyboard cannot match.

US UK and FR keyboard are for example **Physically** different in position and number of keys, so I’m not sure of what this would mean. 

I can dig up a some of my non-us computer if you need testing, an I know many people that would be happy to have better keyboard handling.

-- 

M

[Chris Colbert]

On Sat, Oct 10, 2015 at 8:09 PM, Matthias Bussonnier <bussonnie...@gmail.com> wrote:

I’m also curious of what "assume US English layout” mean, in case where the physical mapping of the keyboard cannot match.

US UK and FR keyboard are for example **Physically** different in position and number of keys, so I’m not sure of what this would mean. 

The current notebook is assuming US english keyboard layout via this mapping from `keyCode` to char: 

https://github.com/jupyter/notebook/blob/master/notebook/static/base/js/keyboard.js#L30 

What Brian was saying is that until Chrome finishes implementing `key`, we will have to continue to assume US english layout on Chrome. The Chrome team just recently promoted the feature from testing to experimental, so it should only be a short-lived limitation:

https://code.google.com/p/chromium/issues/detail?id=227231

[MinRK]

We talked about this same problem at length when doing the original keyboard shortcuts, mainly in that we use the keyDown event to get shortcuts on keys like Enter, but only the keyPress event has the right text value for dispatching the correct events. It would be *great* if keyDown can have the same text identity as keyPress, and we don't have to change what event we are looking for to fix this. I think this is strictly an improvement, and what I've always been trying to do, so +1 for option 2 from me.

-MinRK

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.
To post to this group, send email to jup...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CAPc_zRUeLtTkue9wi_XeFtiwEiKmj%2BKWpwubU0An2BVS%2BR3_8A%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Brian Granger]

Oh yeah, forgot about the keyDown/keyPress issue. Chris, has this
situation changed?

[Chris Colbert]

Keypress is deprecated in DOM 3 and also has some other cross browser issues. Keydown is the way to go.

To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CAH4pYpT_m%2BPMLeGAWgGmM_9TYn-Mt9nzdyWfCYLhu8RMxPbgTA%40mail.gmail.com.

[Brian Granger]

We had more good conversations about this today. Quick thoughts about
our thinking:

* Option 2 is preferred.
* Like the idea a list of shortcuts to indicate chording.
* Like the idea of using space " " as a separator for modifiers and
chars, so "Ctrl A" rather than "Ctrl-A" or "Ctrl+A". The reason for
this is that both - and + are chars so it is possible for users to be
confused about shortcuts like "Ctrl++" and harder to parse shortcuts
with -/+.

> https://groups.google.com/d/msgid/jupyter/CAPc_zRV5oST1bDsEWP0xox%2BCKU8v8mWgXwmNP21fKMwX0FyD%2Bw%40mail.gmail.com.

[Chris Colbert]

To follow up on Brian's point, the space to separate chars/modifiers is wrt the actual specification of the shortcut in code/json. We expect the UI *presentation* of the shortcut to be converted to a more pleasant form, whether it be icons, space-separated, whatever...

To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CAH4pYpThsi2-UM13kZVu6p-eTW5U8k93N59gHOho1N6jqYPNfw%40mail.gmail.com.

[Jason Grout]

FYI, to actually have a shortcut involving space, you would need to write it out: "Ctrl Space".

Jason

[Matthias Bussonnier]

> On Oct 12, 2015, at 12:39, Jason Grout <grout...@gmail.com> wrote:
>
> FYI, to actually have a shortcut involving space, you would need to write it out: "Ctrl Space".
>
> Jason

> On Oct 12, 2015, at 11:31, Chris Colbert <scco...@gmail.com> wrote:
>
> To follow up on Brian's point, the space to separate chars/modifiers is wrt the actual specification of the shortcut in code/json. We expect the UI *presentation* of the shortcut to be converted to a more pleasant form, whether it be icons, space-separated, whatever...

No objection.
--
M

[Chris Colbert]

It seems like I replied to the wrong thread the other day. Sorry about that. Here's a recap about the failed character-based keymaps:

So, the experiments with the `.key` attribute ended up being a failure. The browsers which do implement it, do so incorrectly and inconsistently across browsers and platforms (event amongst themselves). It's quite disappointing.

Our current thinking is to continue to use the original `.keyCode` approach and provide default support for US-English keyboards. The keymap manager will take an optional key code map which will map from `keyCode` to "key cap"* and vice-versa. This mapping will necessarily be specific to the browser, platform, and keyboard layout. 

We will make it simple to provide these mappings as plugins, so the user can configure the keymappings according to their particular configuration. We will rely on the community to generate these mappings for the various OS/browser/layout combinations, but we will provide a graphical tool to help with the generation.

* The "key cap" is the primary character printed on a physical keyboard key.

We've now finished the approach described in that quote. The rough summary of the system is as follows:

• The IKeybinding interface describes the user-defined key binding object. It contains a `sequence` which is an array of keystrokes, a CSS `selector` for scoping the key binding, and a handler function to invoke when the binding is matched.
• The IKeyboardLayout interface is used to abstract away the logic for mapping from a `keydown` event to the character printed on the physical keyboard key (the keycap). The current state of the browser keyboard event API is abysmal, and there is no good cross-browser/locale solution for reliably getting a keycap for a `keydown` event, which is crucial for a usable shortcut system. One semi-reliable bit of information we *can* get from the event is the `keyCode`, which is an implementation-dependent number which represents the physical key which was pressed independent of modifier state. Unfortunately, this code can and will vary between browsers and operating systems. Abstracting away the keyboard layout allows us to create shortcuts which will work for each user's machine, provided they are using a compatible layout. We provide a default layout which supports US English keyboards across the major browsers and operating systems. We will gladly accept community PRs for other international keyboard layouts. If/when the browser event APIs improve, we can implement a keyboard layout which uses these new APIs and which hopefully supports more users out-of-the-box.
• The KeymapManager is used to dispatch a `keydown` event to the registered keybindings. It contains the logic for matching the key sequences and CSS selectors, resolving ambiguous key bindings, and handling the timers for muli-keystroke chords. The manager consumes a keyboard layout in order to covert `keydown` events into keystrokes (among other things).

Major feature highlights:

• Support for user-defined keyboard layouts
• Support for multi-keystroke binding sequences. This is useful for complicated multi-key shortcuts `Ctrl-K Ctrl-P`, or Vim-like chords `D D`.
• Support for CSS selector-based scoping (ala Atom editor). This allows the user to declaratively specify when a keybinding should be enabled. When looking for a matching key binding, the bindings selector must match a node on the bubble path of the event. This allows the user to "scope" a key binding by adding/remove the relevant class/id/data from their DOM nodes.
• Supports US English keyboard layouts on all major platforms and browsers by default.

A very simple example is available in the Github repo.

We spent quite a bit of time discussing this system, and I'm fairly confident in the design of the API at this point. It should cover all of our current use cases and needs, and gives us plenty of room to grow. As always, questions/comments are welcome.

--
You received this message because you are subscribed to the Google Groups "Project Jupyter" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jupyter+u...@googlegroups.com.
To post to this group, send email to jup...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/DA8A24C3-D5D4-4318-B307-7C103FAF2982%40gmail.com.

[Chris Colbert]

Forgot an important highlight:

• Supported modifiers are `Accel`, `Alt`, `Cmd,`, `Ctrl`, and `Shift`. The `Accel` modifier is automatically translated to `Cmd` on Mac, and to `Ctrl` everywhere else. The `Cmd` modifier is *only* valid on Mac. This translation makes it easy to write one set of key bindings which work naturally on most platforms.

For complicated plugin-based apps, we expect the application layer to handle loading application, user, and OS-specific key bindings from JSON files. That is out of scope for this particular repo, but we *are* currently working towards such functionality in other repos, and it uses this keymap manager at its core.

[Brian Granger]

Thanks for the good summary Chris!

We had a number of long design discussions about this during this week
and I am really pleased with how it turned out. Even though we are
unable too use a character based approach, I think this approach will
lead to much better international keyboard support in the long run.

Cheers,

Brian