Hi,
Vivien and I have been trying to design an API to allow writing a
virtual keyboard in HTML. This API is very different from the IME API
from Google [1] that aims to re-use the system's IME in a web page. We
want an API to implement the system IME as a web app.
** Naive approach **
We began to draft an API that was a naive approach trying to solve the
two technical issues we had:
- the VKB app wasn't able to know when the focus was changing in other
apps and what kind of element it was;
- the VKB app wasn't able to send trusted key events such as they are
considered by the other apps as user' inputs.
To fix that, we had the idea to create a navigator.vkb object that would
get 'focuschange' events when the focus would be change in any app. The
event would carry information like the element name, type, value, etc.
In addition, navigator.vkb would have a sendKey() method to send a key
event as trusted.
However, that solution wasn't enough because it is not handling
composition events [2] and we clearly need that. For some languages but
also for suggestions that you can see in some phone's virtual keyboard.
Also, that solution wasn't taking into account elements that had to be
handled without text typing like some input types (date, for example) or
select elements.
** Handling special cases **
It's actually quite easy to handle special cases, we just need specific
functions for them:
- .setValue(DOMString value)
This function takes in parameter a string that would be set to the
focused element if it's an input element. This has to be used for
special situations where the value had to be chosen amongst a list
(type=month) or a widget (type=date, time, etc.). If the value passed in
parameter isn't valid (in the term of HTML5 Forms Validation), the value
will simply be ignored by the input element.
- .setSelectedOption(long idx)
This function takes in parameter the option index that has to be
selected on the focused select element. If the focused element isn't a
select element, it will be ignored. As if the index isn't valid.
- .setSelectedOptions(Array<int> idxs)
Same as before but for select elements that allow multiple selected options.
** Solution 1: composition handled by the VKB **
The main issue with composition is when should it be started? Should we
start a composition when selecting the middle of a word, for example? As
far as we understood, there is no reason why we couldn't. So we thought
we could let the app decide whether or not begin a composition event.
To do that, we need to pass the selectionStart and selectionEnd values
to the VKB but also inform it when the selection changes inside the
currently focused element with a new event named 'selectionchange'. That
event could carry some information like element's value, new selection
values, etc.
Basically the selection handling could be done with three methods:
- .startComposition(int begin, int end)
Parameters will define the text that is affected by the composition.
Calling that method would fire 'compositionstart' with the appropriate
values in |data| to the focused element.
- .updateComposition(DOMString text)
This method should be used after calling startComposition() (calling it
in other situations would fire) and would update the text being
composed. For example, updateComposition('F') should be called if the
text field was empty and the user typed 'F'. updateComposition('Fo')
should be called if the user then typed 'o'. Calling that method would
send a 'compositionupdate' with the appropriate values in |data|, an
'input' event and maybe 'keydown', 'keypress' and 'keyup' events.
- .endComposition(DOMString text)
This method should be called when the composition has to stop. Whether
because a delimiter has been typed or because a suggested word has been
selected. Whatever the reason is, the text should be passed as a
parameter. It will send a 'compositionend' event with the appropriate
values in |data| and like for updateComposition() will fire other DOM
events.
In addition of those methods, we still need a sendKey() event that would
be used for delimiters, backspace and numbers but could also be used
when the VKB doesn't want to use composition. Calling sendKey() between
startComposition() and endComposition() calls would throw.
** Solution 2: composition handled by the platform **
This is very similar to solution 1 but without startComposition() and
endComposition(). Instead of those method, there would be
'compositionstart' and 'compositionend' events fired by the platform to
the VKB. When not in composition mode, the keyboard would have to call
sendKey() when a key would be pressed and, when informed that it should
now go to composition mode, updateComposition() would have to be used
exactly like in solution 1. The text being composed would be passed in
the 'compositionstart' event.
With that solution, sendKey() could whether create a new composition or
not, depending on the platform's decision but calling sendKey() between
'compositionstart' and 'compositionend' events would throw an exception.
** Feedback **
We are waiting for feedback to know which solution should be preferred
if any sound sensible.
Personally, I prefer the first one because it's more flexible and let
the VKB app to really be implemented as the developer wants.
[1]
http://dvcs.w3.org/hg/ime-api/raw-file/default/Overview.html
[2]
http://dev.w3.org/2006/webapi/DOM-Level-3-Events/html/DOM3-Events.html#events-compositionevents
Cheers,
--
Mounir