Python No Hands video on showmedo

[John Graves]

A 90-second video presented at Kiwi PyCon 2009 over the weekend in
Christchurch, New Zealand is now available on showmedo at

http://showmedo.com/videotutorials/video?name=8980000&fromSeriesID=898

It demonstrates programming and running a Python program entirely
through voice control (no mouse or keyboard). If you've ever wanted
to talk to your computer instead of (or in addition to) typing, now
you can! Check it out.

[Kyran Dale]

very cool - will do,

Kyran

 

[Gasto]

Cool.

On Nov 9, 5:53 am, Kyran Dale <kyran.d...@gmail.com> wrote:

[Tim Bower]

How'd you do dat? It looks like a related interface to the speech
recognition with MS Word. I played with that some once, but
eventually got frustrated with it. I wanted to try Dragon Naturally
Speaking, but I wasn't sure that my PC was up to its requirements, so
I held onto my money.

I have a friend that is completely blind and is constantly frustrated
trying to use a computer. Is this something that I could use to help
her put together some custom user interfaces to simple tasks that she
needs to do?

--
Tim Bower
Assistant Professor
Computer Systems Technology
Kansas State University at Salina

[John Graves]

Tim -

Specifically, I have Windows Vista on my laptop. All by itself, the
Windows speech recognition engine (under Control Panel | Speech
Recognition) takes you through a tutorial, trains the system to
understand your voice and then gives you the ability to open and
control applications as well as dictate into a text editor.

To create the specialized voice macros used to write the Hello World
program in the Python No Hands video, I downloaded Python 2.5 from
http://www.python.org/ and dragonfly from

http://code.google.com/p/dragonfly/

and then modified the _multiedit.py program from

http://dragonfly-modules.googlecode.com/svn/trunk/command-modules/documentation/mod-_multiedit.html

to include my customized voice commands:

"insert comment": Text("# \n# \n# \n# \n# \n# \n# \n# \n# \n")+Key("up:5, end"),
"insert function": Text("def ():\n\t\"\"")+Key("up, left:2"),
"insert class": Text("class ():\n\t\"\"")+Key("up"),
"doc add": Key("c-end, enter:2, backspace"),
"with": Text("()")+ Key("left"),
"quote": Text("\"\"")+ Key("left"),

Finally, WHILE running _multiedit.py under Python 2.5, I opened a
second IDLE window and used the speech recognition engine to drive the
keystrokes.

This past weekend at the Kiwi PyCon conference, I moved to the next
step: getting the computer to TALK BACK and await further voice input.
Here is the code I used to do that:

# A voice mirror
# (c) Copyright 2009 by John Graves
# using dragonfly by Christo Butcher
# Licensed under the LGPL, see <http://www.gnu.org/licenses/>

import pkg_resources
import dragonfly
from dragonfly import *

e = dragonfly.get_engine()
e.speak("Hello")

grammar = Grammar("voice mirror")

class SpeakRule(CompoundRule):
spec = "<text>"
extras = [Dictation("text")]

def _process_recognition(self, node, extras):
e = dragonfly.get_engine()
e.speak("You said %s!" % " ".join(node.words()))
e.speak("Say something more intelligent.")

grammar.add_rule(SpeakRule()) # Add the top-level rule.
grammar.load() # Load the grammar.

BUT -- before you even start on following along -- please give a
thought to the potential of our working together to make this
technology really helpful and useful. As you say Tim, you have a
blind friend. I know a guy who fell down some stairs, broke his neck
and lost the use of his hands and feet. Voice-based interfaces would
be extremely helpful to these folks, but I think everyone else could
benefit as well. Think of the 1+ billion cell phones being sold every
year and what it would mean if there were really useful
phone-and-find-out services. I'm just starting work on a PhD on the
open source development of these voice-enabled interfaces so it is my
full-time job now to coordinate and study a project in this area.
Please let me know if you'd like to be involved -- whether as a
developer or just a tester. Thanks in advance and enjoy talking!

Cheers,

John Graves

[Tim Bower]

That's neat, thanks for sharing it. I'll save your message to try
sometime. Python seems like the perfect environment for gluing
together speech recognition and text-to-speech capabilities to make
simple applications that fill a need, especially for people with
certain handicaps.

Actually, I'm just starting another major project myself. It is not
"official" yet, so I need to hold off on announcing what it is, but
down the road a bit, I certainly will want to try out some of the
speech capabilities. Do you have a blog or web page where we can
follow your progress?

Best wishes,
Tim

[John Graves]

Tim -

For now, I'm posting to http://bit.ly/JohnGravesWiki. Don't have a
feed set up yet, but I'll be working on that soon.

Thanks for your interest. Next up on my to do list today is bringing
in video input. There was a presentation on how to do this at Kiwi
Python 2009:

http://www.slideshare.net/hardbyte/python-in-computer-vision

I think when the computer can listen, talk and SEE and (to a limited
extent) RECOGNIZE and perhaps GESTURE BACK, we're going to have an
entirely different kind of interaction.

Cheers,

John

[John Graves]

Tim -

For a blind or seeing impaired (or even dyslexic) person, I just saw
this Intel Reader device mentioned in the NY Times:

http://www.intel.com/healthcare/reader/index.htm
http://bits.blogs.nytimes.com/2009/11/10/intel-sells-its-own-reader-for-the-healthcare-market/

While it might not helpful interacting with a computer, it certainly
seems like it would be helpful interacting with anything else in
print. Point, shoot and listen.

John

On 11/10/09, Tim Bower <woods...@gmail.com> wrote:
>