Testing?

[Jon Poploskie]

Hello,

Given the number of people on the board who seem to have some history with
speech apps and the MS folks, I'm wondering if someone can provide some
guidance on testing. It seems very difficult to have reliable, repeatable
tests in this environment (speech, not just the SASDK/MSS environment).
Unit testing seems to be the general programming answer to fast, repeatable
testing, but I can't find any good solutions to "unit testing" vui's. It's
also very difficult in this scenario because so much stuff happens on the
client-side, and RunSpeech is responsible for so much of the application's
behavior. So it seems that we're left testing at a higher level, i.e.
assuming that if we can have repeatable "functional" tests that things
should be ok at the lower level (strikes me as somewhat dangerous, but...)
Then we run into the problem of how to have reliable, repeatable (and
"fast") functional/load/etc. testing for a speech app.

So, does anyone have any comments about these assumptions? Are there
products out there that do a good job with these sorts of things? The only
thing I've been able to dig up so far is Empirix Hammer, any comments on
that? Will it even work on this platform?

Thank you very much for your help,
Jon

[david burton[@MS]]

Yep, testing is tough.

Empirix does have products that can help with performance-like testing and
regressions in application functionality. And it can be used to drive a DTMF
and/or Speech input application. In the past, you would prepend/append dtmf
tones to each system prompt and then use dtmf recogntion to track the
application progress. Empirix has a new, higher end system that has
continuous speech rec built in, so you might also try that route.

For simple testing, the logplayer tool in the SDK can be useful. It allows
you to record a path through a dialog (using text input) and then replay it
and see if the same result is generated.

Finally, you could build an MSS-based test application, using the SDK, to
verify the production application you are building for public consumption.
You would drive your application with real recorded speech stored in a
promptDB delivered over a live phone line and use MSS to recognize the
production application's responses. This potentially could be done with only
one MSS system that is running two applications.

In summary,

1. use logplayer to buld text-driven tests of small functional units
2. Use Hammer, MSS, or other home-brew telephony-based tools to drive and
listen to your application's responses.

--david

"Jon Poploskie" <n...@spam.com> wrote in message
news:uI$t$RiTEH...@TK2MSFTNGP12.phx.gbl...