International Optimized Layout
Matthias Schult
As I wrote two weeks ago I have counted the occurrence of characters and character combinations in the Wikipedia in 65 languages using the Latin alphabet and I have counted special characters in some programming languages (C/C++, Java, perl, shell scripts, Python, HTML, css). The goal of course is to create key layouts for the AG to use in other languages. Since I still haven't got my AG and it's unlikely that it will arrive tomorrow, I won't have it this prolonged weekend (1st of May is a holiday). Still I would like to hear your views on my demands for creating such a layout:
A key layout should...
1. be as similar as possible for all languages
2. a) have the most common characters in the most convenient places
b) have the rarely used characters on the front
3. minimize hand switching while typing
4. a) try to let both Hands do an equal amount of work
b) try to allow as much one-handed usage as possible
5. replace [Print], [Scroll lock], [Pause] and [Caps lock] and possibly the duplicates of the [Shift]-keys by something more useful.
To 1)
It is utterly annoying having to type on a slightly different keyboard layout when typing in a foreign language. E.g. I'm used to typing on a German QWERTZ-layout. But when typing Swedish, the å and Å are very hard to reach, because they don't exist in German, so I have to switch to the Swedish QWERTY-layout frequently mistyping y and z (~0.7% of the Swedish language). Also sometimes one has to cope with a standard US-layout but German keyboard and has to go hunting for things like asterisk, brackets, dash, and so on. So it would be most convenient to have everything but the language's special characters in the same place. The problem is how different languages should be weighted on creating such a layout. E.g.: All equally? By Wikipedia size? By number of speakers (mother language or total?)? Language families weighted equally? Number of web pages on the internet? The differences are not too big though.
To 2a)
This one is quite obvious, but gets into conflict with (1). The most common characters are language dependent, but not as much as one might think. It only gets problematic with common special characters like ə in Azerbaijani, ë in Albanian, ä in Finnish and Tartar, ı in Tartar and Turkish, á in Hungarian, ð in Icelandic, ê and î in Kurdish, ā in Latvian, and so on (it seems like my sorting script has ripped apart some Vietnamese characters...).
It also conflicts with AGs not sending key codes for the shift keys. It will be possible to find a compromise, I guess, once I have received my AG and know which characters are available for the new layout.
To 2b)
That is for not having to memorize them.
To 3)
Since I was seriously planning on creating a keyboard similar to the AG for about a year I have been watching out for typing mistakes. There are tree types:
a) A missing letter
b) A letter to much
c) The wrong letter
d) permutations
There is not much to do against type (a). Type (b) and (c) can be improved by an improved layout; the asymmetric standard keyboard layout really asks for landing in between or on the wrong key.
Type (d) makes up more than 90% of all typing mistakes (that's an educated guess, I didn't keep a list). They always (100%) occur when switching hands, never when both letters are on the same hand! That's why Dvorak turns out to be a very bad idea -- it maximizes typing errors. A user-friendly layout would try to minimize hand-switches. Sadly character combinations are somewhat language dependent, even though enforcing (1) does not make matters too bad.
To 4a)
This is to avoid one hand hurting after typing for a long time, but it heavily collides with (3). You can have either few hand-changes or do an equal amount of typing left and right. The result when enforcing this point still won't be as bad for typing mistakes as QWERTY and still much better than Dvorak, but won't be much of an improvement either.
To 4b)
It would be very convenient to be able to turn a page or pick your nose without slowing down in typing. Together with (1) this will hardly be possible to archive for normal text, but it is worth thinking about making the "Number Pad" one handed, so you need only one hand to enter numbers and keep the other hand for pointing on the line at the paper you are copying of. Also one hand could be responsible for typing while the other hand does navigating (ctrl, Arrows, Page ↑/↓, Home, End, Tab,...), which is the direct opposite goal to 4a. This demand - surprise - coexists perfectly with (3).
To 5)
Will anyone die without those? If one Shift key is moved to the back side, there is no need for a second one. Sadly this is not possible if it doesn't send a key code.
I would welcome comments on the conflicts noted above, especially: How important is 4a? Also: How should the languages be weighted?
It would be absolutely great if someone could make a list of key codes (e.g. using xev).
I would like to put my results, character counts, and thoughts online somewhere. I checked, but my university doesn't give server space for student home pages, so can anybody recommend a place where I can put up a web page (best for free without too annoying ads)?
Have a good night,
Matthias
Didgers
looked like the American original. You could start by getting a hybrid
out of them. The philosophy of Dvorak Layout is to reduce the effort
required on a traditional keyboard. They may not have been thoroughly
optimized for their language, but I'll bet it's better than what was
derived from qwerty. Anyhow, there's only so many keys on the AG-5,
but there are so many international characters. It would be difficult
to get them all on there, and still retain a level of ease of use.
Autohotkey can only do so much.
2) Since different languages use different letters, some letters'
rarity will vary. Assuming you can even get all of the characters on
the AG-5, there would be at least one language that will have a steady
stream of front pecks. Probably ought to just settle for a layout
that's good for two or three different languages or good for a family
(Germanic, Romantic, Cyrillic, etc)
3) My only mistakes are missed / wrong keys. I don't really have too
much trouble with permutations, but that might just be me. I honestly
believe that Dvorak's alternating hands rthym helps against permutation
error. However, I don't have to do much international typing
(occasionally some French, which is just some accents), so maybe that's
an issue with typing in other languages. All I do is just use standard
keyboard symbols before/after letters and then just use search and
replace to polish it up (e\: => ë, i\^ => î).
4) If you want to type with one hand, you could try to base your AG-5's
layout on one of Dvorak's one handed typing layouts. Whatever layout,
you will be overwhelming the physical limits, which are already being
pushed due to all the international characters you're trying to type.
The AG-5 makes typing a lot less stressful, but it's not really much of
a speed thing. If you don't have time to stop and take a break for
whatever, you don't have time to learn the enhanced qwerty layout, let
alone some complex one you come up with.
5) Certain Shift / Lock keys are hard coded in the AG-5. With
Autohotkey, you can work with them to get what you want.
Conclusion) There's a million possibilities, ranging from a few letter
swaps to chording everything to the point of having an 8 finger chord
to produce a specific byte. Use chords of 5 to produce 31 different
characters if you insist on one hand. More or less, you should just
decide what you want for what you do. AG-5 layout based on American
Dvorak works great for me, especially since the learning curve is
easier (less keys moved around). However perfect it is for a Dvorak
fanboy, some would prefer to trudge through something else.
Matthias Schult
> 1) I looked at the various language specific Dvorak layouts, and most
> looked like the American original.
> You could start by getting a hybrid
> out of them. The philosophy of Dvorak Layout is to reduce the effort
> required on a traditional keyboard.
Sorry, it seems not to have become clear what I intended to do. I want to create a layout based on scientific research as much as possible. The reason is not only to get as close to the optimum as possible, but also to keep 1001 different layouts popping up everywhere. Starting with a layout optimized for a standard keyboard thus is not the way to go. The way to go is creating one from scratch.
Anyway, as an example I compared German and American Dvorak layout. They differ in 25.9% of an average German text and in 35.0% of an average English text -- even though both languages are Germanic languages. I wouldn't call that very similar...
The Dvorak layout is based on:
1) minimizing finger mileage: This does not apply to the AG since most keys are right under the fingers anyway
2) maximizing hand switches: Dvorak thought that would be faster, which I strongly doubt. There have been no serious scientific investigations about that which I am aware of. The only study I know shows a tendency that typing with adjacent fingers inwards (ring, middle, index finger in that order) is the fastest combination possible, but even that might be different if not typing on a flat board. On the other hand, hand switches are very error prone.
> They may not have been thoroughly
> optimized for their language, but I'll bet it's better than what was
> derived from qwerty. Anyhow, there's only so many keys on the AG-5,
> but there are so many international characters. It would be difficult
> to get them all on there, and still retain a level of ease of use.
> Autohotkey can only do so much.
That's a misunderstanding. I mean you don't have to be able to reach all international characters on every layout, just the language specific ones. But those letters should be the only serious difference between the layouts. I don't have MS Windows, so I don't need Autohotkey, but I'm confident that the resulting layout will be possible to implement in it.
> 2) Since different languages use different letters, some letters'
> rarity will vary. Assuming you can even get all of the characters on
> the AG-5, there would be at least one language that will have a steady
> stream of front pecks.
No, surprisingly. The 10 most common characters make up between 60% and 65% percent of a language for almost every language. And those include a,e,i,n,r,t and space for nearly all languages. The order of the 20 most frequent characters might be different from language to language, but the characters are basically the same. Things start getting different when looking at character combinations and digraphs.
> Probably ought to just settle for a layout
> that's good for two or three different languages or good for a family
> (Germanic, Romantic, Cyrillic, etc)
Does a romantic layout have l,o,v,e in the home row? :-) I won't make any Cyrillic layouts, because I don't speak any languages which use the Cyrillic alphabet myself. So I stick to the Latin alphabet. The 5 language families with the most speakers in languages using the latin alphabet are:
italic (French, Spanish, Italian, Portuguese, Romanian, ...)
germanic (English, German, Dutch, Swedish, Norwegian, Danish, Luxemburgish, Islandic, Afrikaans, ...)
malayo-polynesian (Indonesian, Tagalog, Javanese, Cebuano, ...)
turkic (Turkish, Tartar, Azerbaijani)
slavic (Polish, Czech, Slovac, Slovene, Serbo-Croatian, ...)
Though small, Finno-Ugric (Finnish, Hungarian, Estonian) is pretty far spread in the internet as well.
> 3) My only mistakes are missed / wrong keys. I don't really have too
> much trouble with permutations, but that might just be me. I honestly
> believe that Dvorak's alternating hands rthym helps against permutation
> error.
My observations are mainly based on e-mails I got, less on my own writing. It's also a conclusion I didn't expect. I expected missed or wrong keys to make up most of the typing error, but they clearly don't.
> 4) If you want to type with one hand, you could try to base your AG-5's
> layout on one of Dvorak's one handed typing layouts. Whatever layout,
> you will be overwhelming the physical limits, which are already being
> pushed due to all the international characters you're trying to type.
No, it's not about one-handed writing. My formulation probably wasn't so good. It's about what's more convenient for normal use.
I would reserve chording for stenographic typing, except maybe for writing language specific stuff. E.g. in German umlauts are resolved to ä→ae, ö→oe, ü→ue, so those chords would be kind of natural.
Well, ok. I hope my AG will arrive next week,
Matthias
ivanwfr
> really have too much trouble with permutations, but
> that might just be me. I honestly believe that Dvorak's
> alternating hands rthym helps against permutation error.
gotcha: rthym
Didgers
ivanwfr
ivanwfr
This is not good enough... Compromise is not the only way to go; with
some smart approach everyone could get an adapted configuration...
Customization, Layout and printed symbols:
We definitely need a programmable device! With its keys out of sight,
AG users can't hunt and peck anymore. This means that the default
printed symbols are much less of a problem than with other keyboards.
All this make the AG concept the prefect candidate for new layout
experimentation. Anyone with good customizing software could have his
own perfect layout uploaded into a firmware flash memory!
This does not mean that everyone is to devise an original layout! That
should be done seriously, quite like you are dealing with right now...
What scares me about this is that you can't change your mind about a
layout that will suck much of your time to learn! This is why I am not
currently playing with AutoHotkey. Even if Mike's layout is not the
best for me, I need to know that I am working on steady ground. Yet, I
am ready to relearn the AG6 layout if Mike make it happen, hopefully
with the help of studies like yours and the many comments they cause
like mine...
--
A programmable AG6, please!
--
Didgers
Did I spell that right? ;)
ivanwfr
To make this post less off topic, I'll add some of my thoughts more
or less related to the real subject...
[Quitting QWERTY and Dvorak]
This is one of the many challenges that go with ergonomic design; the
past is there to make every decision difficult. It is easy to get
trapped by one of the many reasons why things should not change:
- Is it all about not being afraid of having to relearn typing from
scratch?
- Not loosing those many years of practice?
- Would it help making the learning curve less painful if the new
looked as much like the old as possible?
These all are illusions... Being honest, we all know that it will be
painful!
[Muscle memory: Loosing fingers allocated letters relationship]
I fail to see how QWERTY or Dvorak could take place in devising on an
effective layout for this new device. Sure, each finger is acquainted
to its set of letters on a standard keyboard but I am not sure that it
is a good idea to build everything on that.
[QWERTY and Dvorak premises]
I rather believe it is time to turn the page once and for all, to get
rid of QWERTY and Dvorak...
Their originating premises are very poor considering modern ways and
means of studying ergonomic matters like this.
[Creating a layout from scratch]
I can't agree more because I really don't see how we can take advantage
from the experience of typing on a standard keyboard when using the
AlphaGrip. It is just impossible to be more wrong than QWERTY. Building
anything related to Dvorak layout would only be a compromise.
Yet, I would certainly like having one of these standard layouts
wrapped around the AG as Lee sees it here:
http://groups.google.com/group/alphagrip/msg/653e93a8f46cf40e
Didgers
effort into optimizing my layout. I'm really pleased with what I've
come up with, and I believe that it will suffice.
Lee's thing would be a nice keyboard for me. I could just use the
default Dvorak layout to switch it. But it would be such a large piece
of equipment, that you would have a lot of weight tiring you out. Too
bad there's no cheap, easy, all encompassing super solution.
Toby Dickenson
> The goal
> of course is to create key layouts for the AG to use in other languages.
You didnt make any comment on design method...... I have been planning a
similar exercise using this as a starting point:
http://www.visi.com/~pmk/evolved.html
> [permutation errors] always (100%) occur when switching hands, never
> when both letters are on the same hand! That's why Dvorak turns out
> to be a very bad idea -- it maximizes typing errors. A user-friendly
> layout would try to minimize hand-switches.
I am a keen dvorak user. I agree that the hand-alternation characteristic has
some effect on error rate. Certainly permutation errors make up a higher
proportion of my typing errors under dvorak (although it is not obvious that
the overall rate is higher).
However alternation does have a major effect on comfort, and for me that is a
more important factor than error rate. For example the word "monopoly" is all
right handed on qwerty. I think typing would be less comfortable if too many
words were like that.
Neither would it be good with alternation every letter. Dvorak places several
common digraphs (such as 'th') on adjacent keys under the same hand. Typing
"pith" (left-left-right-right) is more comfortable than
"late" (right-left-right-left)
It would be nice to have a quantitative method for estimating these various
factors (comfort, error rate, are there others?) for a specific layout.
--
Toby Dickenson
Didgers
Maybe if we had a million multilingual monkeys, and a million AG-5's
with a million different international layouts, in a million years they
could produce the perfect layout for each language(s).
That or we can recreate the evolving experiment. Need lots of samples
of what international people type so we have something to parse. Then,
lets jump straight to the third experiment once we get a lot of samples
of key pressing data. Let the computers parse and evolve the layouts.
Not as fun as a million monkeys, but quicker and less messy.
Matthias Schult
> You didnt make any comment on design method...... I have been planning a
> similar exercise using this as a starting point:
> http://www.visi.com/~pmk/evolved.html
That's exactly what I intended to do. Thanks for pointing out that web page. With the AG we have the advantage that for most keys the fingers don't have to move at all.
On the other hand I don't believe it is a good idea to optimize a layout for one specific language. Most people are not native English speakers and especially dealing with computers many will switch a lot between their mother tongue and English for example. There are language boarders everywhere too, just look at Europe.
> Neither would it be good with alternation every letter. Dvorak places several
> common digraphs (such as 'th') on adjacent keys under the same hand. Typing
> "pith" (left-left-right-right) is more comfortable than
> "late" (right-left-right-left)
Right, that's the problem. How to find a decent compromise between putting digraphs on one hand and equal hand usage?
an, en, and er are common digraphs in most languages, but a,e, and r make up 21% of all typing already (including space, punctuation, and special characters); and if we see that "e " is actually an internationally even more common combination and almost 14% of a text are spaces (16% in English), we soon have a problem. The 6 most common characters of a language (in English [Space], a, e, t, i, and o) make up 50% of all text. That seems to be universally valid.
Digraphs and trigraphs are also heavily language dependent. If you find a lot of "th" in a text, it's probably English. If you see a lot of "ch" and "ei" and "ie" and even "sch" it's German. And with "nh", "th", and "ch": Vietnamese.
First I'd try to find out which characters should be on the 24(?) home row keys. Then divide into left and right hand.
Right now I'd say we give a penalty for each hand switch and make a penalty for unequal hand usage. This penalty could be distributed like [a * exp(- x²/(1 - x²) ) * exp(- b * x²)]. As you said it would be very nice to have some reasoning for choosing the width and hight of the distribution.
[supplement:] I just came up with the following reasoning: The standard deviation of the distribution should be the average work distribution between the hands using a random letter distribution. It's not easy to calculate that, and I have no clue right now what the result will be.
For the test text I'd leave away all languages and dialects which are usually not used for writing:
luxembourgish_lb (?)
breton_br
walloon_wa
kurdish_ku (? written in arabic in Iran and Iraq; usage was restricted till 2002 in turkey, small part of Syria. So I guess the latin alphabet is usually not used for writing.)
alemannic_als
frisian_fy
sicilian_scn
asturian_ast
limburgish-li
low_saxon_nds
tatar_tt (? uses mostly the cyrillic alphabet)
min_nan_zh-min-nan (? usually written in chinese characters)
occitan_oc
neapolitan_nap
waray-waray_war (?)
aragonese_an
scots_sco
scots_gaelic_gd (?)
cornish_kw
corsican_co
Neglecting also artificial languages (Esperanto, Ido, Interlingua) and dead languages (Latin). Leaving:
language amount of text speakers language family
english_en 2845571442 470 germanic
spanish_es 269538836 362 italic
portuguese_pt 163812785 182 italic
indonesian_id 30817378 140 malayo-polynesian
french_fr 558925974 124 italic
german_de 1128049793 121 germanic
javanese_jv 1197566 76 malayo-polynesian
vietnamese_vi 24035758 67 mon-khmer
italian_it 260198406 63 italic
turkish_tr 39983689 59 turkic
polish_pl 287792245 44 slavic
romanian_ro 33147565 26 italic
serbo-croatian_sh 7494274 21 slavic
dutch_nl 266594403 21 germanic
azerbaijani_az 1276922 21 turkic
tagalog_tl 2649370 15 malayo-polynesian
hungarian_hu 65426119 15 finno-ugric
cebuano_ceb 863257 15 malayo-polynesian (?)
czeck_cs 57663394 12 slavic
swedish_sv 130327751 9 germanic
finnish_fi 93787778 6 finno-ugric
danish_da 43996924 5 germanic
norwegian_bokmal_no 74809291 4 germanic
lithuanian_lt 20014862 4 baltic
latvian_lv 5061632 1.6 baltic
estonian_et 24406323 1.1 finno-ugric
slovak_sk 32338079 slavic
haitian_ht 363879 italic
catalan_ca 47349432 italic
galician_gl 23843558 italic (?)
slovene_sl 29198388 slavic
faroese_fo 1172743 germanic
afrikaans_af 7916993 germanic (?)
norwegian_nynorsk_nn 15903664 germanic
irish_ga 3542912 celtic
bosnian_bs 12145269 slavic
islandic_is 9439568 germanic
croatian_hr 25977897 slavic
welsh_cy 3682368 celtic
basque_eu 4296053 isolated (?)
albanian_sq 5484695 isolated
I'm not sure about including some of them (marked with "?").
Should they be weighted differently?
If all goes well, I'll have a web space next weekend to make everything availiable online next weekend (when I'll hopefully have an AG as well.
Matthias
Toby Dickenson
On Monday 01 May 2006 00:09, Matthias Schult wrote:
> On Sun, 30 Apr 2006 09:34:05 +0100 Toby Dickenson <to...@tarind.com> wrote:
> > You didnt make any comment on design method...... I have been planning a
> > similar exercise using this as a starting point:
> > http://www.visi.com/~pmk/evolved.html
....to summarise the method described on that page: using a modern
evolutionary optimisation algorithm and real inter-key timing and letter
distribution data to generate a better layout.
> That's exactly what I intended to do. Thanks for pointing out that web
> page.
Ive had a few sleepless nights and spent some time on this approach. My goal
is to reproduce the comfortable feel of the dvorak layout on the AG5. I have
been focussing on the layout for the letter keys, comma, and dot. No chording
or anything similarly exotic. This email is a summary of my results so far.
Initial results were encouraging, but not entirely successful. My real goal is
to maximise comfort, however I dont know how to express that in a cost
function for the optimiser. Experiments with a cost function which estimates
typing speed failed to produce anything comfortable.
Separately I tried using Didgers' dvorak-based AG5 layout and found the home
row worked well, but the other letters did not. This suggested a hybrid
approach - using the home row exactly as Didgers' dvorak layout, and my
optimiser to reassign the non-home-row letter keys. Ive been using the
resulting layout for the last two weeks and, so far, this does feel pretty
good.
Layout cheatsheet attached, and linux/X11 layout files available on request.
Im aware that others are interested in this evolutionary optimisation
approach. I hope to polish and publish the current code sometime next month.
--
Toby Dickenson
