Command-Control system to navigate through a smartphone by voice

[Bruno Lima]

Hello everyone,

I'm quadriplegic so, as my master degree work, I decided to developed a system to control a smartphone by voice.  My idea consists in "change" the way the user interact with the system by allow him use voice instead touch. I have been working on that for the last year and I advanced a lot. Now I have a speech recognition system based on sphinx (I'd like to use Google Voice API but I couldn't find out how to limit the vocabulary) that work very well and a command-control, both on a single Service, with a few commands, such as:

• Go Home
• Louder
• Quieter
• Set Mode (Normal, Silent, Vibrate)

Those were the only ones I could find over the internet. Now I got stuck on some obstacles:

1. (Un)Lock Screen: I found a few solutions but none worked;
2. Go Back: First I tried to use Intents and flags but didn't workout (I don't know if I did it wrong or what). Then I tried to use the ActivityManager to get the current activity and finish it. But didn't work.
3. Swipe and Scroll: I wish there is something similar to AudioManager that allow me to perform those actions programmatically. If not, how can I do that?

Press/Hold: My idea is to enumerate with badges each clickable then execute the click event based on the number.

Ps.: Should I break these topics in 4 threads?

Ps2.: Sorry about my English.

Well, that's all. I really appreciate any help.

[ch...@christopherprice.net]

Very impressive!

Personally, I think the best approach may be to use Android Wear as a remote. The Android Wear services would let you wake the device via Bluetooth (via voice... Android Wear can always listen for "Ok Google" and you can then run app commands from the Google prompt).

#1 - Again, I'd suggest Android wear. You can set the Lock screen to none - that will remove the lock screen completely. That leaves the display wake itself.

Programmatically, you have choices... especially if you can build AOSP for the device. If you can get that done, you can add display sleep controls (such as never). If the device is mounted to a wheelchair or mobility device, a battery could then keep it powered, and always on. Developer options also gives you Stay Awake when charging on most devices. Or, you could use something as simple as Tasker to set WakeLock on, that would bypass the need to bake a firmware.

#2 & #3

So, from early on in Android, an input keyevent can pass along the back button. Swipes though, you can't recreate very easily. You can, however, code an Android keyboard service that creates an input device - basically allowing you to insert such inputs across all apps. Ironically, an Android Keyboard adaptation may be your best bet there - albeit, a cumbersome solution.

Hope that gets you started. A significant other of mine spent a long time coding Android visionless UX concepts - won a few hackathons that way.

Hope I can be of more help, but like I mentioned I think Android Wear could actually turn into a major universal-access tool for Android phones and tablets.

Christopher Price

ConsoleOS.com