emacs bindings and cmd functionality for dead keys?

[Matt]

I have a keyboard layout where a and e are dead keys. Their
terminators are just the letters a and e.

When using TextEdit (or anything else, it seems), I get the following
behavior:

Emacs bindings:

If I type ctrl-a or ctrl-e, an ascii 1 or 5 (i.e. the actual invisible
ascii control character) gets inserted into my document. This is
annoying, because those characters are invisible, so I don't even
notice that they got inserted. They are easy to notice only in that
the cursor doesn't move when it goes back and forth across this
character in the document. Even ascertaining what characters were
being inserted into my document required writing a program to read and
print out the characters in the document. Preceding the ctrl-a or
ctrl-e with ctrl-q is the same as without the ctrl-q. (My q key is a
normal q key.) My keyboard layout shows the default ascii control
codes when the control key is down.

Cmd bindings:

If I type cmd-a (for "select all") then it acts as if I had not been
pressing cmd (i.e. it enters the dead key state showing a highlighted
"a" rather than selecting all). Same for cmd-e (for "enter
selection"). Can this be fixed using the modifiers drawer somehow? I
don't see how to change the "set" for the second row (the row for when
the command key is down) to distinguish it from the first row.

I have read 5.7.2 and 8.7 of the manual, but I do not understand how
dead keys relate to what is said there.

Thanks for any tips,
Matt

[John Brownie]

I'm not sure of all the issues here, never having been an emacs user and
so not being used to the emacs bindings.

One thing that you could try is to see whether the emacs bindings work
with a keyboard layout which is not a system one, but which is ordinary,
such as installing one of the ones from the Ukelele installation disk
image. If that doesn't work, then this would seem to be a situation
where the operating system treats system and user keyboard layouts
differently. If it does work, then it would indicate that it's something
in your keyboard layout that is at fault.

I suspect that your command-key combination problems arise from having
command map to the same outputs as keys without any modifiers. What
would most likely solve this would be to create a new modifier set for
command, probably making it a standard key map (QWERTY for most people).
Alternatively, some experimenting with the "Unlink" commands might
resolve the issue.

I hope that helps a bit. If not, please ask again and see if we can
solve things for you.

John
--
John Brownie, john_b...@sil.org or j.br...@sil.org.pg
Summer Institute of Linguistics | Mussau-Emira language, Mussau Is.
Ukarumpa, Eastern Highlands Province | New Ireland Province
Papua New Guinea | Papua New Guinea

[Matt]

I should have been more clear -- emacs itself works fine, not
suffering from the issue I describe, but I don't use it very often.

I am talking just about the bindings like ctrl-e for "go to end of
line", which work across most of OSX, for example in TextEdit or in
this browser window where I am typing this reply. There are over a
dozen of them, and I refer to them as "emacs" bindings only because
the correspondence between keys and actions derives historically from
how the keys behaved in emacs. You can see a list of these bindings
in the lower right corner of the http://www.danrodney.com/mac/ page.

My issue occurs because my keyboard uses regular unmodified alphabetic
characters, like "e", as dead keys. I do not know any system keyboard
with this feature, so I can't compare.

Your tip for fixing the command-key problem works perfectly, by the
way! Specifically, the command-key functionality works fine once I
set up the keyboard so that it provides a standard qwerty map when the
command key is pressed. This requires messing with the modifiers
drawer.

So my only remaining problem is that plain-alphabet dead keys cannot
have their control- version bound to anything in the
DefaultKeyBinding.dict file.

To reproduce my problem, set up "e" as a dead key. You can have its
terminator be the letter "e", and the dead key state can be an empty
keyboard, so that you can type normally. The only difference from
using a normal keyboard is that the "e" will be highlighted in orange
until you type the next character.

Now, using this keyboard, it seems to be the case that ctrl-e will
insert literally whatever the keyboard indicates for ctrl-e. Note
that this is completely abnormal behavior -- normally, pressing
control-this or control-that will either do nothing (beep) or perform
its defined emacs-style action. More specifically, it will perform
whatever action is defined in the DefaultKeyBinding.dict or
StandardKeyBinding.dict files (which is in fact where the system
defines the emacs-style bindings for us). Pressing control-something
certainly does not insert raw control characters into our text.

But when the plain (not control) "e" key is a dead key, then this
strange behavior occurs: The raw info from the keyboard's definition
of the control (not plain) "e" key is inserted into the document,
without ever passing through any of the *KeyBinding.dict files.

Examples:

For these examples, I set my DefaultKeyBindings.dict to contains the
following lines. (Which makes it very hard to type!) If you're not
familiar with the format, the string on the right doesn't have any
particular meaning to the system; it is simply inserted into the
document when the key on the left is received. These lines are not a
useful example to learn from -- the power of key bindings comes from
the fact that you can bind many different actions to keys besides
inserting text.
"e" = ("insertText:", "<plain e>");
"^e" = ("insertText:", "<ctrl-e>");
"~e" = ("insertText:", "<alt-e>");
"η" = ("insertText:", "<η>");
"^η" = ("insertText:", "<ctrl-η>");
"~η" = ("insertText:", "<alt-η>");
"€" = ("insertText:", "<€>");
"^€" = ("insertText:", "<ctrl-€>");
"~€" = ("insertText:", "<alt-€>");
"\U0005" = ("insertText:", "<enquiry>");
"^\U0005" = ("insertText:", "<ctrl-enquiry>");
"~\U0005" = ("insertText:", "<alt-enquiry>");

(note that \U0005 is a useless rare invisible ascii control code
character, "enquiry")

US keyboard:
plain e = "e", ctrl-e = "\U0005", alt-e = dead key for acute accent
behavior:
plain e = "<plain e>", ctrl-e = "<ctrl-e>", alt-e = dead key
behavior

my keyboard:
plain e = dead key for accented e, ctrl-e = "\U0005" (or "e"), alt-e
= "η"
behavior:
plain e = dead key behavior, ctrl-e = "\U0005" (or "e"), alt-e = "η"

Hungarian keyboard:
plain e = "e", ctrl-e = "\U0005", alt-e = "€"
behavior:
plain e = "<plain e>", ctrl-e = "<ctrl-e>", alt-e = "<alt-e>"

The thing that is weird:
Ctrl-e on my keyboard ignores the key bindings. My keyboard treats
ctrl-e just like every other keyboard does, but the behavior is
completely different. It is impossible for me to assign an action
(even the default OSX action) to ctrl-e. The difference stems from
the plain e being a dead key.

Why it is annoying:
Text navigation using the keyboard is very useful. I would like it
to work with my keyboard. But if my fingers try to navigate by
mistake, I wind up filling my document with invisible control code
characters. This doesn't happen with any other keyboard. My keyboard
doesn't have any invisible control code characters that the other
keyboards don't have. But with the other keyboards, these insidious
characters never get inserted into the text.

The Ukulele Manual (p. 66) says:
The way that the control key is handled appears to be this. When a key
stroke with the control key is sent to the system, if the option key
is not down, or if the quote key (normally ^q) has not just been sent,
the system reanalyses the key stroke. It does this by asking what
character would be generated by the same key stroke without the
control key.

The manual goes on to explain how you can compensate for this
weirdness via DefaultKeyBindings.dict. My experimentation shows that
this approach will only work when the non-control key is not a dead
key.

In fact the same process seems to apply for the alt (option) key as
well, considering the above data from the Hungarian keyboard: The
keyboard's definition for the unmodified key is sent, along with
modifier information, to the *KeyBinding.dict files. However, if the
keyboard has no definition for the unmodified key (for example because
it is a dead key), as is the case for my keyboard, then this binding
process can't do anything, and the keyboard's own definition of the
modified key is used.

As another comment, the Key Codes app reports absolutely no keys when
I press the dead "e" key on my keyboard. This contrasts with pressing
the dead key alt-e on the US keyboard, for which it reports that keys
= alt-e. This strangeness is in fact consistent with the process
described in the previous paragraph. The "keys" reported by Key Codes
is the same as what gets passed to the *KeyBinding.dict files.

At this point I believe my problem is unfixable. This is sad, because
the whole reason I made my own keyboard was to be able to type
diacritics *after* the letter, just like how we write them by hand.
But OSX seems to be unprepared for this approach.

I may try a pure DefaultKeyBinding.dict approach, since that can also
handle sequences of keys, but that approach will not allow switching
back and forth between DefaultKeyBinding.dict-defined keyboard
behaviors, whereas you can easily switch between actual keyboards. I
could define a keyboard to produce characters from an alphabet I
normally never use, and then a single DefaultKeyBinding.dict could
define one behavior for one keyboard and another behavior for the
other keyboard. A peculiar approach -- pressing the "a" key would
make that keyboard report an "alpha" key, and the
DefaultKeyBinding.dict file would then map the "alpha" back to "a", or
perhaps to a dead state waiting for diacritics to put on the "a",
without interfering with any ctrl-a functionality -- I think it would
work, but the dead keys (as defined in DefaultKeyBinding.dict) would
be invisible until completed, not even showing up in orange, and I
don't know of any way to set the equivalent of a terminator (to be
used if the next key pressed is not in the provided list), so the dead
key state would need to repeat the entire keyboard definition.

[Geke]

Dear Matt,

>the whole reason I made my own keyboard was to be able to type
>diacritics *after* the letter, just like how we write them by hand.

Then can't you do just that: type the letter, then the Unicode
"combining diacritical mark" for the accent?
Example: U+0061, U+0301 would give á
I expect most programs to replace the two-character sequence (letter
+accent) by the standard precomposed character, but some may leave it
unaltered.

Geke

[Matt]

Dear Geke,

Thanks for the idea!

If I change (for example) the single quote key to be this "combining
diacritical mark", then it works great when I want an accent, and in
fact it has super powers -- it can put an acute accent on anything!
(Regardless of whether that thing was just typed or entered long
ago.) I've never seen an accented 4 before, but I have now! It never
gives just a quote. Even when put on top of a space, it seems to be
distinct (regarding e.g. search and replace in TextEdit) from what
"alt-e space" produces on the US keyboard.

Maybe I'll make a keyboard where I give up a bunch of standard
keyboard keys so that I can make these accents, but it's a high price.

The approach of my keyboard was that the accenting characters are in
fact normal characters when not following a vowel, so no new keys need
to be learned beyond what is printed on the keys. The keyboard works
largely normally, but certain sequences combine into accented letters,
like e' or c, or u: or o". The keyboard can be explained and learned
by anyone immediately (all the accents I use look at least roughly
like some regular punctuation symbol), and also it is faster to type
this way (despite the fact that I am pretty used to the "alt-e =
acute" keyboard).

My only problem is that typing is so natural on this keyboard that I
forget that I'm not allowed to press ctrl-e or ctrl-a!

Matt

[Geke]

Hi Matt,

To be sure things work as you want, you have to check in the saved
file what characters get entered when you type a character followed by
a combining diacritical mark. The most basic behaviour would be just
ending up with two characters, exactly the character and the mark. I'm
not sure if that also happens if the text program is more Unicode-
savvy and replaces the two by the precomposed character, if it exists.
Maybe that replacement happens only onscreen (to give better
typography), maybe it gets saved too, I just don't know.
It's also possible that this whole replacing business is organized by
the OS, so as long as the text program is not too old, it would work
everywhere.

You can get just the combining mark to show by typing it after a non-
breaking space, U+00a0, which is usually on option+space. As here (I
hope it shows up):

 ́
Certainly, this is different from what alt-..., space produces,
because those are spacing accents. E.g., alt-e, space produces U+00b4,
but the combining mark is on U+0301.

> Maybe I'll make a keyboard where I give up a bunch of standard
> keyboard keys so that I can make these accents, but it's a high price.

Why not change alt-e etc from dead keys into normal ones, containing
the combining accents? Then you can type the letter followed by alt-e
to get the accented letter.
Or put them on ctrl-e etc.

[Christophe]

I want to create a dead key with the s key, but also to keep the s
output with the cmd modifier. The problem is I can't create a new
modifier set, for instance with "Unlink Modifier Set…" (Ukelele
2.1.6).

Thanks for answer,
Christophe

[Matt]

Hi Geke,

That's a nice idea, using unicode symbols to post-process characters
instead of dead keys to pre-process keystrokes.

But I still find it strange that achieving my goal of typing "j'e'tait
la`" to create "j'était là" works, but prevents OSX's system-wide
emacs key bindings from working. I see no way, through any
combination of keyboards, key bindings, and unicode, to achieve my
typing goal while having the emacs key bindings also work.

Matt

[Matt]

Hi Christpohe,

To make a new modifier set, first press the colorful "modifiers" key
in the toolbar to get the modifiers drawer to show up at the bottom.
Then, below the list of modified keyboards, you'll see a "+" button
and a "-" button. Using those you can slowly change the list of to be
whatever you want. I find the default set rather confusing, so I
replaced almost all of them, and it works fine. I didn't have much
luck trying to figure out the linking and unlinking, so in the end I
just ignored it.

Matt

[Christophe]

Hi Matt,

Thanks a lot! Now it works. However, I think the modifier keys
question is the most difficult, or a bit confused. I'm not sure the
two "unlink" commands do really work. By creating a new modifier set,
you have to not forget to set the command key down. If you don't, and
set it after, it won't work immediatly, and you have to close the
keyboard, and to open it again.

[Geke]

That's a great link, that Dan Rodney page!

If you put the accents (the "combining diacritic marks") on alt-e etc,
and change those keystrokes from dead ones into "live" ones, then the
emacs keybindings should work, no? That's at least what I understood
from your earlier description.
So then you could, for example, type á by typing "a", then "option-e".
And typing ctrl-e should jump to the end of paragraph.

Or you may want to put those accents on the digit keys (the top row)
like the French do. Then you could e.g. type "a", then "2" to get "á"
-- without having to take your hand off the keyboard to reach for the
Option key.

[Matt]

Hey, I finally got this to work!

I had to edit my .keylayout file directly, to get it into a form that
I don't think Ukulele can create. For this, the documentation at
http://developer.apple.com/library/mac/#technotes/tn2056/_index.html
was very helpful.

What I did was this:
. . .
<key code="0" action="a"/> <!-- 0 is the A key -->
. . .
<key code="39" action="acute"/> <!-- 39 is the ' key -->
. . .
<action id="a">
<when state="none" output="a" next="after a"/>
</action>
. . .
<action id="acute">
<when state="none" output="'"/>
<when state="after a" output="&#x0301;"/>
. . . <!-- other states can be added here -->
</action>
. . .
The idea is that the "a" is not a normal dead key. It immediately
produces the output "a" (which allows the key bindings to work), but
also puts the keyboard into a state I called "after a". There is no
terminator for this state (because the "a" was already produced).
When you press ' then one of two things happens: if it is in the
"after a" state (or "after e", etc.), then it produces a "combining
acute accent", but otherwise it produces its normal character (a
single quote mark). I also made the backslash key serve as a "no
accent" marker for accentable letters, so typing "c," produces "ç" but
typing "c\," produces "c,". I also set up the keyMaps so that
capslock (which I never use) turns off the accenting functionality of
the various accent keys ( ' ` ^ , ~ etc.), so it is very easy to turn
it on and off.

So, through a combination of the .keylayout file and unicode's
combining diacritics (thanks Geke for that idea!), I was finally able
to achieve my goal:
* I type the accent *after* the letter (just like how you would
write it with a pen).
* The key for the accent is the standard key that looks most like
the accent (easy to type and nothing to memorize).
* The key only accents appropriate letters, and otherwise has its
normal non-accenting function.
* Key bindings work normally (e.g. ctrl-a for start of line).

Matt