## This topic has been hidden because it was flagged for abuse.

Showing 1-28 of 28 messages
 This message has been hidden because it was flagged for abuse. Re: Proposal: mozKeyboard API for extending built-in keyboard and supporting 3rd-party Keyboard app Ben Francis 11/16/12 9:20 AM Hi, I was at a Career Fair at Cambridge University yesterday and was talking to an engineer from SwiftKey (swiftkey.net), a London-based startup who make an alternative keyboard for Android. Their advice was not to make our keyboard API anything like Android's! I have pointed them towards this thread to give them an opportunity to provide further feedback. Ben On Fri, Nov 9, 2012 at 7:20 AM, Yuan Xulei wrote: > Hi all, > > In order to enhance the virtual keyboard for more commonly used feature on > the Android and the iOS platforms, and also to support the 3rd-party > keyboard app after v1, we(Rudy, Tim and I) are trying to design a new > mozKeyboard API. > > I created a MoPad to list and discuss our proposal: >  https://etherpad.mozilla.org/**gaia-mozkeyboard-v3 > > The draft API of the our proposal are listed bellow, which is similar to > that of the android. > > == KeyboardManager == > > When user focuses an editable text field which requires the soft keyboard, > an "imestatechange" event will be fired. > > The keyboard app should listen to the  "inputmethodstatechane" events by >  the KeyboardManager#**oninputmethodstatechane attribute. Then it check >     // Check https://groups.google.com/**group/mozilla.dev.webapi/** > browse_thread/thread/**56cdfe1c6bfd34dcfordetails. > to the  "selectionchange" events by  the KeyboardManager#**onselectionchange > attribute. > An object of KeyboardSelectionChangeEvent will be passed when a > "selectionchange" event is fired. > > interface KeyboardSelectionChangeEvent: Event { > readonly attribute long oldSelectionStart; > readonly attribute long oldSelectionEnd; > readonly attribute long newSelectionStart; > readonly attribute long newSelectionEnd; > readonly attribute long composingTextStart; > readonly attribute long composingTextEnd; > }; > > sample code: > > // Called when the user starts or finishes editing an input field > navigator.mozKeyboard.**oninputmethodstatechane= function(event) { >     // Get the KeyboardConnection instance >     var conn = navigator.mozKeyboard.**connection; >     if (!conn) { >         // finishes editing > return; >     } >     // Send a string to user's input field >     if (conn.inputType == "text") { > conn.commitText("Hello world"); >     } else if (conn.inputType == "url") { > conn.commitText("http://www.**mozilla.org "); >     } else { > conn.commitText("123"); >     } > }; > > > Please let me know if you have any questions. > Thanks. > > Regards, > > Yuan Xulei > ______________________________**_________________ > dev-webapi mailing list > dev-w...@lists.mozilla.org > https://lists.mozilla.org/**listinfo/dev-webapi > -- Ben Francis http://tola.me.uk This message has been hidden because it was flagged for abuse. This message has been hidden because it was flagged for abuse. This message has been hidden because it was flagged for abuse. Re: Proposal: mozKeyboard API for extending built-in keyboard and supporting 3rd-party Keyboard app Salvador de la Puente González 2/6/13 7:37 AM Hello guys. I saw this bug referring to keyboard: https://bugzilla.mozilla.org/show_bug.cgi?id=838308 In short, any application can use mozKeyboard.onfocuschange and *setValue()* method to inject text in the target input, in any other application.  From my point of view, there is no problem to keep mozKeyboard open without permission but only if the sender of the *setValue()* message is in the same domain than the receiver. We should control these kind of situations when designing the new API. Of course, the sender could be the keyboard application which receives special permission to inject text to any other input, whatever its domain was. Cheers! ________________________________ Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo. This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at: http://www.tid.es/ES/PAGINAS/disclaimer.aspx This message has been hidden because it was flagged for abuse. This message has been hidden because it was flagged for abuse. Re: Proposal: mozKeyboard API for extending built-in keyboard and supporting 3rd-party Keyboard app Anne van Kesteren 4/15/13 2:34 AM On Mon, Apr 15, 2013 at 10:21 AM, Yuan Xulei wrote: > We're working on implementing the new mozKeyboard API: >   https://wiki.mozilla.org/WebAPI/KeboardIME FWIW, it seems Microsoft might have an interest similar to ours: http://lists.w3.org/Archives/Public/public-webapps/2013JanMar/1059.html We might also want to tell the WebApps list what we're working on to see if long term we can work out something that works the same everywhere. -- http://annevankesteren.nl/ Re: Proposal: mozKeyboard API for extending built-in keyboard and supporting 3rd-party Keyboard app Salvador de la Puente González 4/15/13 3:03 AM Hello people. In-line commentaries: Some changes are made according to people's advice and our requirements, including: 1. Combined commitText with deleteSurroundingText into replaceSurroundingText. INMHO, the name is very long but I understand is very explicative. I suggest some replacements but may be other people can suggest a better (shorter) wording:   *   alterText() - my favourite   *   changeText()   *   replaceText() When I was developing the keyboard I did not find any situation requiring changing the surrounding text but I can be different when developing suggestion or other IMEs. If so, the I have nothing to say but if not, I do not fully understand why to include these "surrounding" methods. May be we can use something similar to splice in Array methods. Some like: alterText(newText, fromIndex, toIndex, cursorRelative) - replace text between fromIndex to toIndex (not included) with newText. If cursorRelative is set to true fromIndex and toIndex are treated like beforeLength and afterLength. But do whatever you want with this proposal. ;) 2. Add ontextchange attribute to allow IME get notifications when the text content has been changed. What kind of information is included in this event? When this event is triggered? If I commit text from the keyboard, is it triggered as well? 3. Combined returnKeyType with returnKeyLabel into returnKey. I would keep type and label separated, may be in the same attribute but I think this should be an object (or interface) with different fields for type and label. Remember, if you are here in Madrid and want to talk remember I'm in the 6th floor. Cheers. Re: Proposal: mozKeyboard API for extending built-in keyboard and supporting 3rd-party Keyboard app Ehsan Akhgari 4/15/13 7:24 AM Hi Yuan, Here's my feedback based on which I believe is the latest version of the proposal (please correct me if I'm wrong.) 1. InputMethodManager is not defined anywhere as far as I can see. 2. oninputmethodstatechange, onselectionchange and ontextchange should be EventHandlers (and InputMethodConnection should be an EventTarget). 3. As Jonas mentioned in the bug, I think that we should make sure that the API doesn't make assumptions about the text edit control. Specifically, setText for example will not be useful for a contentEditable element (and probably not even for textareas).  If you need an API to clear the text, you should add that, but the interaction with multi-line text editing in contentEditable/textarea needs to be thought out. 4. returnKey's value can be a localized string, right?  In that case, I don't think that special casing the behavior for "done" is the right thing to do, we should just add another boolean attribute to expose what happens when pressing the return key.  Also, why is returnKey readonly? 5. What is the use case for the beforeLength and afterLength arguments to replaceSurroundingText? 6. Do we want to expose some information about text change events, such as the before/after text, the offset of the text inside the editable field, etc.?  Same question for onselectionchange. 7. selectionStart and selectionEnd need to be unsigned long. 8. Can you please rewrite the proposal in Web IDL, and drop the moz prefixes? Thanks! Ehsan This message has been hidden because it was flagged for abuse. This message has been hidden because it was flagged for abuse. Re: Proposal: mozKeyboard API for extending built-in keyboard and supporting 3rd-party Keyboard app Salvador de la Puente González 4/15/13 10:02 AM Hello Yuan >>> 2. Add ontextchange attribute to allow IME get notifications when >>> the text content has been changed. >> What kind of information is included in this event? >     Nothing. It should include the changed text, but for > contentEditable element, the text may be huge. Thus it requires IME to > invokes getText  method to get the changed text. Here we can add just the information about if this event was triggered by this session or not. In order to distinguish between IME operations or external action. My suggest is to add 'external' flag set to true if the event is not triggered by the edit session. That's all. ________________________________ This message has been hidden because it was flagged for abuse. Re: Proposal: mozKeyboard API for extending built-in keyboard and supporting 3rd-party Keyboard app Salvador de la Puente González 4/15/13 11:10 AM Even better. I always can trigger a custom event If I want to listen for my own modifications in this fashion. Re: Proposal: mozKeyboard API for extending built-in keyboard and supporting 3rd-party Keyboard app Ehsan Akhgari 4/15/13 12:23 PM On 2013-04-15 12:27 PM, Yuan Xulei wrote: >> 4. returnKey's value can be a localized string, right? >       Yes. >> In that case, I don't think that special casing the behavior for >> "done" is the right thing to do, we should just add another boolean >> attribute to expose what happens when pressing the return key.  Also, >> why is returnKey readonly? >      The behaviour and title of the return key are specified by the > input field and the IME can't change it. For example, a search box could > set its return key to 'search' with the code  returnKey='search' />. OK, but your proposal says: If the value is "next", when pressing the enter key, the focus should be move to the next input field. I don't think this is a good idea since the web app might set returnKey to the translation of "next" in the target locale, and that would break the above behavior. Also, is the returnKey attribute spec'ed somewhere?  How is one supposed to specify it for contentEditable/designMode? >> 5. What is the use case for the beforeLength and afterLength arguments >> to replaceSurroundingText? > To correct a word, user would replace the word before or around the cursor. Wouldn't it be better to have a replaceWord(index) API instead, which will replace the word that |index| would fall into? I can see some complications with using this API for large text boxes that are getting modified by both the keyboard and content script. Tracking the correct offsets doesn't seem very easy in that case. >> 6. Do we want to expose some information about text change events, >> such as the before/after text, the offset of the text inside the >> editable field, etc.? > Currently no information will be included. The before/after text and > text offset will be useful for making suggestion and auto-correction, > may be we could expose some of them without degrading the system > performance. OK. Cheers, Ehsan This message has been hidden because it was flagged for abuse. Re: Proposal: mozKeyboard API for extending built-in keyboard and supporting 3rd-party Keyboard app Mounir Lamouri 4/17/13 3:12 AM To remove any confusion, the Keyboard IME API and the IME API from WebApps are totally different. They do not support the same use cases. The former is to create virtual keyboards/IME using the Web Platform and the later, as far as I understand it, is to show the system's IME in web applications using canvas. For example, a textfield in a game. -- Mounir Re: Proposal: mozKeyboard API for extending built-in keyboard and supporting 3rd-party Keyboard app Mounir Lamouri 4/17/13 9:52 AM Hi, First of all, thank you for working on this Yuan. The API looks generally good, but I have a few comments. I think it would be good to clearly mention as a UC (Use Case) that this API is there for dealing with text but not to deal with other form controls like . >> >> I am not sure what InputMethodManager is there for given that it is not >> defined. Is that intentional? > > Yes, it will defined later. Please, define at least what would be the purpose. >> We should not expose it to >> other platforms anyway, right? > > Why not? The desktop and mobile browser may benefit from the API. We can > provide virtual keyboard to other platforms as well. I doubt any other platforms will use that API any time soon. Pushing it to other platforms will increase the exposure of this API. We should keep it inside Firefox OS to not harm the Web. >> I'm not a big fan of the 'inputmethodstatechange' event and the >> 'inputStarted' attribute. First of all, the event name is way too long >> and developers don't like long name. 'statechange' would be enough. > > Shorter and simple event name will be better, but the name should tell > the developer that it is an IME event. This event is sent on an object named inputMethodConnection implementing the InputMethodConnection interface. I believe it should be clear the event is related to 'inputmethod'. >> Should we simply >> have some kind of 'start'/'change'/'finish' events? I actually wonder >> how the Keyboard application is expected to be created. Shouldn't we >> make the 'start' event a system message and maybe the 'finish' event >> should not exist because we might expect the application to be killed at >> that point? Though, I guess it might be platform dependant. > The life-cycle of the Keyboard application is: > > 1. create > | > 2. start <----- > | | > 3. change focus an input field > | | > 4. finish ----- > | > 5. destroy > > The Keyboard manager will create the keyboard app, which will run as a > normal app and receive the load event on start-up. When user focus an > input field, the keyboard app starts text input. During the input, the > content or attributes of input field may be changed by external > actions(for example, JS code to clear the text), the keyboard app should > get notification about the changes. After user finishes input of current > input field, the keyboard app will wait for the user to focus another > input field. > > So there needs a least two types of events: one fired when user switches > between input fields, and the other fired when the current input field > has changed. I'm not sure if you are agreeing or disagreeing with my proposal. It seems that 'start', 'modify' and 'finish' would work correctly here, wouldn't they? Also, 'inputStarted' could be replaced by 'enabled' maybe? Basically, the keyboard application would know if it is currently being in use. > Besides the load and unload events, we may need additional 'start', > 'change' and 'finish' events. To be honest, I wonder if the start event is really needed. When the keyboard application is started, it will be obviously used for a field so a 'start' event is useless here. If the keyboard got dismissed and is re-open again without being actually killed, we might as well just send a 'modify' event. Another option could be to use a system message. I'm not sure those are great ideas but probably worth thinking about. >> It seems that replaceSurroundingText could be replaced by replaceWord >> unless there are some UCs I'm missing. Inserting some text maybe? But I >> don't see in which case we want to insert more that one character at a >> time without replacing the current word? > > For CJK, the words are not separated by spaces. Each sentence could be > viewed as a word. replaceWord doesn't work for CJK IME. > And for Latin, when doing suggestions, we often need to insert some text > to complete a word, for example, input 'En' and get the suggestion of > 'English'. For the latin case, you could simply do replaceWord('English') and that would replace 'En' with 'English'. For the CJK case, I do not know anything about that but I wonder how the keyboard application would be able to know that something is a word if the UA can't event know that. Are CJK users editing their previously typed stuff and except to have auto-correction? or that is something only latin users expect? >> I feel that setComposingText UCs can be handled by replaceWord. If >> replaceWord takes by the default the word at the cursor position and we >> consider that in the following case "foo|" the cursur is position on the >> word "foo", wouldn't setComposingText be exactly like replaceWord? > > Yes, replaceWord or replaceSurroundingText can be used to add a string > to the input field as well. But the setComposingText is not used to add > string to the input field. It is used to handle composition events for > CJK IMEs. For example, to input the Chinese word '我', we will input the > composing text 'wo' first, then convert it to corresponding Chinese > character. Also sometimes we use composing text to help us input special > characters, such as mathematical symbols. For example, the composing > text of Σ is '\sum'. You can find more about composing text in the > section "2.1 Compose" of > https://dvcs.w3.org/hg/ime-api/raw-file/default/Overview.html#composer-section. First use case: - the user types 'wo'; - the keyboard application calls replaceWord('我') which is going to replace the current word ('wo') with '我'. Second use case: - the user types '\sum'; - the keyboard application calls replaceWord('Σ') which is going to replace the current word ('\sum') with 'Σ'. Why wouldn't that work? >> Why do we have getText() instead of always exposing the text inside the >> field? Is that to improve performance? > > Yes, to avoid transfer large text between apps, sine for the content > editable element, the text content may be several KB. But I think we > could change it to a sync method without callback. I think we should keep that async. If the concern is to save performance, we shouldn't make that synchronous. Also, we should probably use: Future getText(start, end); >> What is the UC for removeFocus() all the keyboards I have seen are >> dismissed by the system, not by themselves. Is there a particular UI in >> mind? > > removeFocus could support a hide button for the keyboard. If the > keyboard support full screen editing, it will need a hide button to > cancel editing. I've never seen such UI. On Android, I'm able to dismiss a virtual keyboard from the system UI but I never saw a virtual keyboard offering a button to dismiss itself even when they are fullscreen. Do you have specific UIs in mind? Are we planning to implement that in our UI? >> advanceFocus() and rewindFocus() are interesting because they seem to >> apply on the entire document but all the keyboards I have used would >> make that feature work in a form, not in the document. Is that >> applicable in the document on purpose? > > Yes, applicable in the document, but the app itself could the focus be > restricted in the forms. I didn't catch that. >> Also, this would skip some form >> elements, which I believe a user wouldn't want. > > Yes, non-text field will be skipped. Those methods just follow the > behaviour of that on android and iOS. I haven't seen that behaviour on Android. I doubt the iOS behaviour is that great. I think the form behaviour is way better but we could also design the API such as a keyboard can implement both. >> Finally, returnKey will not be added as an attribute on form controls. I >> do not think this is needed and people will reject that proposal if the >> use case is a Keyboard API. Actually, what can't we do with the >> information we already have? If type='search', you can use 'search', if >> there is no form control after, you can use 'done' or 'go' (you might >> need to know if the form is submittable for that though). I do not see >> what 'send' is there for. >> Anyway, the idea is that we should give enough context to the keyboard >> so it can show its own UI based on those information. > > It will complicate for the keyboard to infer the return key meaning by > the context. Take the awesome bar of firefox browser for example, if > user inputs a keyword, the return key will be 'search'; if user inputs a > URL, the return key will be 'go'. Keybard can hardly determine the > return keyboard type by the content of the awesome bar. > > 'send' is used for sending a mail, SMS, instance message and so on. The problem is that first of all, you can't expect web pages to use that magic returnKey attribute just because the Keyboard API uses it. If I recall correctly, Firefox Android tried to use this and switched back to guess the label based on the content. Second of all, standardizing this API is going to be hard. If we have to rely on a new magic attribute in HTML, this is going to be even harder. The HTML specification will never accept that change. If we keep that work self-contained, we increase the chances to have it standardized. If that happen, we would have a stronger point if we go to the HTML WG and ask for a new attribute for the virtual keyboards. Cheers, -- Mounir This message has been hidden because it was flagged for abuse. Re: Proposal: mozKeyboard API for extending built-in keyboard and supporting 3rd-party Keyboard app Salvador de la Puente González 4/29/13 12:34 AM On 28/04/13 20:46, Yuan Xulei wrote: On 04/19/2013 05:50 PM, Mounir Lamouri wrote: On 19/04/13 00:35, Yuan Xulei wrote: On 04/18/2013 12:52 AM, Mounir Lamouri wrote: Hi, First of all, thank you for working on this Yuan. The API looks generally good, but I have a few comments. I think it would be good to clearly mention as a UC (Use Case) that this API is there for dealing with text but not to deal with other form controls like
More topics »