Hi All,
I sent this feedback in a separate thread, but just saw this one so I
figured it's better to send it here instead.
I just looked over the keyboard API and in general it looks great! I
have a few comments but in general it feels like something that is
very polished and thought through.
I take it that the API in the Wiki is the latest version? It seems
like it has a few updates compared to what's in the email below, so I
reviewed what's in the wiki.
I added a few minor comments to the wiki and marked them with [JS]. I
also had a few bigger comments below:
* InputContext for contenteditable
We need to define what constitutes an inputcontext when dealing with
contenteditable.
I think generally we should treat any separate block-level HTML
element as a separate inputcontext. That keeps things simpler in that
the keyboard API doesn't have to deal with the concept of paragraphs.
I.e. it can keep considering the inputcontext as a continuous piece of
text.
So I don't think that we need to look at what CSS is applied. Instead
simply looking at element names should be enough to determine where
the boarder between inputcontexts go.
* Race conditions between keyboard and app processes
Another issue that needs to be defined is what to do if the selection
changes between the point when we fire "onselectionchange", and the
keyboard app sends calls replaceSurroundingText or sendKey etc. I.e.
consider the following scenario:
1. The page focuses an <input> element
2. We bring up the keyboard and fire a oninputcontextchange event
3. The user clicks inside a misspelled word in the <input> element
4. We fire an onselectionchange event in the keyboard app
5. The keyboard app calls getText to investigate the text surrounding the cursor
6. We return a result to the keyboard app
7. The keyboard app realizes that the word is misspelled and displays
a suggested correction to the user.
8. The user clicks the suggestion and the keyboard app calls
replaceSurroundingText() to change the misspelled word.
9. Before the app receives the replaceSurroundingText command, the app
calls a function to move the cursor, either to move it to a new
inputcontext, or to move it inside the current inputcontext.
10. The app receives the replaceSurroundingText.
At this point, I think we need to detect that the text or cursor has
changed since the last time the keyboard app was notified. In this
case the replaceSurroundingText command should fail. I.e. no
modification should be done and we should make the Promise that was
returned report an error.
One way to do this is to in the app process and the keyboard process
keep a "generation-number". Any time the current inputcontext is
modified or any time cursor is moved we increase this
generation-number in the app process.
Whenever we send a onselectionchange, oninputcontextchange or
onsurroundingtextchange is sent from the app process to the keyboard
process we include the updated generation-number in the internal
notification from the app process to the keyboard process.
Whenever we actually fire the onselectionchange, oninputcontextchange
or onsurroundingtextchange events, we update the generation-number in
the keyboard process using the in the keyboard app.
Whenever we send a sendKey(), replaceSurroundingText() or other
modification from the keyboard process to the app process, we include
the keyboards current generation-number. When that notification is
processed by the app process we compare with the app process
generation-number. If the two are different we don't do the
modification and instead cause a failure.
If this is complicated to implement then maybe we can wait for 1.3 to fix this.
* Apply spell-fix automatically on lost focus.
When editing latin text, we sometimes want behavior similar to IME
composition characters. Consider the following scenario:
1. User starts messaging app and starts composing a message
2. User types "see you tomorro"
3. Keyboard shows as suggested spell fix "tomorrow", but doesn't
automatically fix the text yet since the user hasn't finished typing
the word.
4. User presses "send" button
At this point we need to give the keyboard the opportunity to fix the
spelling before the message is sent.
Same thing if the user clicked in a different text box or simply
closed the keyboard.
There are at least two ways we can fix this. Either we could let the
keyboard provide a range and a "replacement text". This is used so
that if the user moves focus, the text in the defined range is
replaced with the "replacement text". Note that this wouldn't affect
the selection, since in the example above, the selection is still
collapsed at the end of the text. But the replacement range consists
of the "tomorro".
Another way to fix it is to fire an event in the keyboard API whenever
focus is moved, but *before* the focus is moved. This would give the
keyboard app the ability to call replaceSurroundingText before the
cursor is moved or focus is lost.
However I'm worried that having to fire an event in a separate process
before the cursor is moved could be a performance problem.
Is this something that can be done with the existing API? I don't
quite understand how the setComposition/endComposition functions work
so maybe they can be used for this?
/ Jonas
On Tue, Jul 23, 2013 at 12:31 AM, Tim Chien <
timd...@mozilla.com> wrote:
> Hi all,
>
> I've work with Xulei to come up with the new Input Method API idl. I
> believe we have covered many of the use cases needed, so we would
> really love to have your prompt feedback.
>
> Basically methods to interact with the text field are categorized into
> 4 kind of use cases, see
>
>
https://wiki.mozilla.org/WebAPI/KeboardIME#Use_cases_for_each_of_the_methods
>
> 1) For a simple virtual keyboard action (send a character and key
> events w/ each user action), use sendKey().
> 2) For spellcheck, autocomplete etc, use surrounding text methods.
> 3) For cursor moment helper features, use setSelectionRange() and
> related attributes.
> 4) For Asian IMEs that sends characters and composition along with the
> composition events, use setComposition() and endComposition().
>
> One thing I find in disagreement with Xulei is methods that actually
> changes the text if the fields. Eventually we have kept the use case
> separation, but just like Xulei you might find the APIs are a bit
> redundant. We would need more pair of eyes for giving us feedback on
> limit the redundancy while keeping the clear separation. Thanks,
>
>
> On Tue, Jul 23, 2013 at 5:51 AM, Yuan Xulei <
xy...@mozilla.com> wrote:
>> void showInputMethodPicker showAll();
>>
>> // Ask the OS to switch away from the current active Keyboard app.
>> // OS should ignore this request if the app is currently not the active
>> one.
>> void switchToNextInputMethod next();
>>
>> // To know if the OS supports IME switching or not.
>> // Use case: let the keyboard app knows if it is necessary to show the
>> "IME switching"
>> // (globe) button. We have a use case that when there is only one IME
>> enabled, we
>> // should not show the globe icon.
>> boolean supportsSwitching();
>>
>> // Ask the OS to hide the current active Keyboard app. (was:
>> |removeFocus()|)
>> // OS should ignore this request if the app is currently not the active
>> one.
>> // The OS will void the current input context (if it exists).
>> // This method belong to |mgmt| because we would like to allow Keyboard
>> to access to
>> // this method w/o a input context.
>> void removeFocus hide();
>> partial interface Navigator {
>> readonly attribute InputMethodManager InputMethodManager;
>> readonly attribute InputMethodConnection InputMethodConnection;
>> };
>>
>> dictionary InputMethodInfo {
>> url: 'XXX',
>> name: 'IME name',
>> locale: 'en-US',
>> };
>>
>> // Manages the list of IMEs, enables/disables IME and switches to an IME.
>> interface InputMethodManager {
>> // Show a (system) menu for all enabled input methods that allow user
>> to select.
>> void showInputMethodPicker();
>>
>> // Get a list of all installed IMEs.
>> Promise<InputMethodInfo[]> getAll();
>>
>> // Get a list of all enabled IMEs.
>> Promise<InputMethodInfo[]> getEnabled();
>>
>> // Get input method info of the caller IME.
>> Promise<InputMethodInfo> getSelf();
>>
>> // Switch to the given IME.
>> // Privileged API. Only the system app and current IME can switch IME.
>> Promise<boolean> setCurrentInputMethod(InputMethodInfo info);
>>
>> // Switch to next IME.
>> // We may not need this method and use setCurrentInputMethod instead.
>> Promise<boolean> switchToNextInputMethod();
>>
>> // Get current IME.
>> Promise<InputMethodInfo> getCurrentInputMethod();
>>
>> // Enable an IME.
>> // Privileged API. Only the system app can eanble an IME.
>> Promise<boolean> enable(InputMethodInfo info);
>>
>> // Disable an IME.
>> // Privileged API. Only the system app can disable an IME.
>> Promise<boolean> disable(InputMethodInfo info);
>> };
>>
>> // The input context, which are attributes and information of current input
>> field.
>> dictionary InputMethodContext {
>> // This is used to specify the target of input field operations. This ID
>> becomes invalid as soon as user leaves input field and blur event is sent.
>> long contextId;
>>
>> // The tag name of input field, which is enum of "input", "textarea", or
>> "contenteditable"
>> DOMString name;
>> // The type of the input field, which is enum of text, number, password,
>> url, search and email.
>> DOMString type;
>> /*
>> * The input mode string.
>> *
https://bugzilla.mozilla.org/show_bug.cgi?id=796544
>> * It can be one of the following values:
>> * "none"
>> * "verbatim" - no capitalization, no word suggestions
>> * "latin" - word suggestions but no capitalization
>> * "latin-prose" - word suggestions and capitalization at the start of
>> sentences
>> * "latin-name" - word suggestions and capitalize each word
>> * "digit" - digits(0-9) only.
>> */
>> DOMString inputmode;
>> /*
>> * The primary language for the input field.
>> * It is the value of HTMLElement.lang.
>> * see
>>
http://www.whatwg.org/specs/web-apps/current-work/multipage/elements.html#htmlelement
>> */
>> DOMString lang;
>> };
>>
>> interface InputMethodConnection: EventTarget {
>> // It informs the IME that text input has started in an input field. If
>> the IME is activated, this event is sent when focus enters a new input
>> field. Otherwise this event is sent when user switches to the IME and
>> activates it.
>> attribute EventHandler onstart;
>>
>> // It informs the IME that text input has finished in last input field.
>> This event is sent when focus leaves current input field or user switches to
>> another IME.
>> attribute EventHandler onfinish;
>>
>> // This event is sent when the attributes of input context has changed,
>> such as type, but focus has not.
>> attribute EventHandler oncontextchange;
>>
>> /*
>> * This event is sent when the text around the cursor is changed, due to
>> either text
>> * editing or cursor movement. The text length is limited to 100
>> characters for each
>> * back and forth direction.
>> *
>> * The event handler function is specified as:
>> * @param beforeString Text before and including cursor position.
>> * @param afterString Text after and excluing cursor position.
>> * function(long contextId, DOMString beforeText, DOMString afterText) {
>> * ...
>> * }
>> */
>> attribute SurroundingTextChangeEventHandler onsurroundingtextchange;
>>
>> // User moves the cursor, changes the selection, or alters the composing
>> text length
>> attribute EventHandler onselectionchange;
>>
>> // TODO: maybe the parameters could be simpler?
>> Promise<boolean> sendKey(long contextId, long keyCode, long charCode,
>> long modifiers);
>> // Or Promise<boolean> sendKey(long contextId, KeyboardEvent event)
>>
>> /*
>> * Get the whole text content of the input field.
>> */
>> Promise<DOMString> getText(long contextId);
>>
>> /*
>> * Commit text to current input field and replace text around cursor
>> position. It will clear the current composition.
>> *
>> * @param text The string to be replaced with.
>> * @param offset The offset from the cursor position where replacing
>> starts. Defaults to 0.
>> * @param length The length of text to replace. Defaults to 0.
>> */
>> Promise<boolean> commitText(long contextId, DOMString text, [optional]
>> long offset, [optional] long length);
>>
>> /*
>> *
>> * Delete text around the cursor.
>> * @param offset The offset from the cursor position where deletion
>> starts.
>> * @param length The length of text to delete.
>> */
>> Promise<boolean> deleteSurroundingText(long offset, long length);
>>
>> // The start and stop position of the selection.
>> readonly attribute long selectionStart;
>> readonly attribute long selectionEnd;
>>
>> /*
>> * Set the selection range of the the editable text.
>> * Note: This method cannot be used to move the cursor during
>> composition. Calling this
>> * method will cancel composition.
>> * @param start The beginning of the selected text.
>> * @param length The length of the selected text.
>> *
>> * Note that the start position should be less or equal to the end
>> position.
>> * To move the cursor, set the start and end position to the same
>> value.
>> */
>> Promise<boolean> setSelectionRange(long contextId, long start, long
>> length);
>>
>> /*
>> * Set current composition. It will start or update composition.
>> * @param cursor Position in the text of the cursor.
>> */
>> Promise<boolean> setComposition(long contextId, DOMString text, long
>> cursor);
>>
>> /*