How to access pop-ups & options menu from soft IME

[Jorge Silva]

I am working on an IME (Tekla) for remote switch access, so powered
wheelchair users and other people with disabilities can access the
Android platform (see video here: http://www.youtube.com/watch?v=r5K-HXk-LYU)

The Tekla IME is a fork of the LatinIME source (at the
"android-2.0_r1" tag) from:
http://bit.ly/euiMRI

As part of the accessibility features, Tekla adds a small UI
Navigation keyboard that sends DPAD events allowing users to navigate
the UI:
http://bit.ly/hblRTN

This all works like a charm for the most part, with the notable
exception that when some pop-ups (http://bit.ly/dH4kJ2) and the
options menu (http://bit.ly/eFUeFQ) appear, two really bad things
happen:
1. the soft IME is hidden in the background, and
2. even when the IME remains active, it sends the key events to the
parent activity instead of the foreground pop-up or menu layer.

All this means that tekla users are effectively locked out of the
phone and will need help from a temporarily-able person to get out of
this "locked out" state.

In a related issue, the list of suggestions attached to the search
function in a few apps also causes the first issue mentioned above
(i.e., it hides the soft IME), however, the key events are sent
correctly. Problem is, Tekla users can't see what they are doing
because the list forces itself above the soft IME once it gets focus
(http://bit.ly/gRnNhU).

Both issues are reported downstream here:
https://bugs.launchpad.net/meadl/+bug/617060
https://bugs.launchpad.net/meadl/+bug/677303

Any insights on a solution or workaround for these issues will be
appreciated. Please let me know if I should file this as a bug to
Android.

cheers!
Jorge

[Jorge Silva]

Diane H, do you have any insights on issue #2 above?

> "even when the IME remains active, it sends the key events to the
> parent activity instead of the foreground pop-up or menu layer."

Could it be solved by injecting key events? The hardware keyboard
behaves correctly, so I am guessing there must be a way. Currently we
are using :

private void keyDownUp(int keyEventCode) {
getCurrentInputConnection().sendKeyEvent(
new KeyEvent(KeyEvent.ACTION_DOWN, keyEventCode));
getCurrentInputConnection().sendKeyEvent(
new KeyEvent(KeyEvent.ACTION_UP, keyEventCode));
}

cheers!

> Both issues are reported downstream here:https://bugs.launchpad.net/meadl/+bug/617060https://bugs.launchpad.net/meadl/+bug/677303

[Dianne Hackborn]

No, an application can't inject events into another application's window.

I thought I already replied to this question elsewhere saying that this is how the input framework is designed and there isn't a way I know to do what you are requesting?

--
You received this message because you are subscribed to the Google Groups "android-platform" group.
To post to this group, send email to android-...@googlegroups.com.
To unsubscribe from this group, send email to android-platfo...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/android-platform?hl=en.

--
Dianne Hackborn
Android framework engineer
hac...@android.com

Note: please don't send private questions to me, as I don't have time to provide private support, and so won't reply to such e-mails.  All such questions should be posted on public forums, where I and others can see and answer them.

[Dianne Hackborn]

(Btw, this is pretty fundamental to the user interaction, as you will see that things like the menu window are displayed on top of the IME, so the UI design is intrinsically around them not getting interaction from the IME.)

[Jorge Silva]

No, an application can't inject events into another application's window.

Isn't that what an IME does? I think we are already doing that, but maybe I am confused.
 

I thought I already replied to this question elsewhere saying that this is how the input framework is designed and there isn't a way I know to do what you are requesting?

yes, through TV, thanks for the comment... I just thought there might have been a slim chance I had not explained the issue properly

(Btw, this is pretty fundamental to the user interaction, as you will see that things like the menu window are displayed on top of the IME, so the UI design is intrinsically around them not getting interaction from the IME.)

It seemed to me that was the case from the beginning, however,  two things still make me think there should be a way to do it:
1. The hardware keyboard does the right thing (ie. sends events to the pop-up or menu instead of the parent activity), and
2. pop-ups that contain a text field do not block the soft IME

so any further insights, even if it is just a gut feeling, will be appreciated. I knew it was a long shot since no phone is designed with alternative access in mind. Android is the closest to achieving that though... and the wheelchair industry is pretty substantial (~2 billion for Invacare alone: http://en.wikipedia.org/wiki/Invacare), so it is not just a couple of people looking for this.