UniMRCP C# Wrapper Example Application for Nuance ASR - Unable to detect speech START-OF-INPUT

[Nader Hijazi]

Hi all, Vali,

Unfortunately, the c# wrapper only comes with the class UniSynth which is a sample application for TTS.

I have created a sample recognition (ASR) test application by streaming an audio file, by using similar objects used in UniSynth but replacing the channel and stream UniMRCPRecognizerChannel, UniMRCPStreamRx, and UniMRCPAudioTermination to build it, but the Speech Server does not throw any START-OF-INPUT event.
It times out throwing the event RECOGNIZER_RECOGNITION_COMPLETE with a completion reason: 002 no-input-timeout

I wanted to have a proof of concept that the c# wrapper can do MRCP recognition, in order to incorporate it in our solution. No luck so far.
I have looked into the C code in UniMRCP for the demo_recog to see if there is any conceptual difference with the C# wrapper but I couldn't find any.
Also, I have tried to swig the demo_recog code to create a C# Wrapper but No Luck also.

Could you kindly give me a hand?

Here is an overview of what I did:
----------------------------------

- For the configuration and profiles, I believe I have everything in place (tested the same configuration in UniMRCP and it works fine for both TTS and Recognition)

- For the Audio file, I am using the same file provided by the UniMRCP demo_recog sample (one-8kHz.pcm) which works with the demo

- For the scenario I am following scenario
@subsection sec-recog1 Simple Recognition
* <pre>
* C>S: Send RECOGNIZE message with Content-Type header and body with grammar.
* UniMRCPClientChannel::OnAdd
* C<S: Receive IN-PROGRESS response.
* UniMRCPRecognizerChannel::OnMessageReceive
* (Client now sends audio and/or DTMF to recognize.) UniMRCPStreamRx::ReadFrame
* C<S: May receive START-OF-INPUT event.
* UniMRCPRecognizerChannel::OnMessageReceive
* C<S: Receive RECOGNITION-COMPLETE event.
* UniMRCPRecognizerChannel::OnMessageReceive
* </pre>

- In UniMRCPStreamRx, I have overridden the constructor and ReadFrame() to read an audio file and call SetData() in order to stream the audio to the speech server as follows:

public class RecogStream : UniMRCPStreamRx
{
public bool streaming = false;
// Buffer to copy audio data to
private byte[] buf = null;
private FileStream f = null;

public RecogStream(string filename)
{
f = new FileStream(filename, FileMode.Open, FileAccess.Read);
}

public override bool ReadFrame()
{
if (streaming)
{
if (f != null)
{
uint frameSize = GetDataSize();

buf = new byte[frameSize];
//
int readcount = f.Read(buf, 0, Convert.ToInt32(frameSize));
if (readcount > 0)
{
//Console.Write("{0} - ReadFrame [{1} bytes]", this, readcount);
SetData(buf, (uint)readcount);
if (readcount != frameSize)
{
Console.Write("- Done reading file: Last ReadFrame [{1} bytes]", this, readcount);
streaming = false;
//SendDTMF('2');
}
}
else
{
streaming = false;
}
}
}
return true;
}
--
This e-mail (including any attachments) is for the sole use of the intended
recipient(s) and may contain information that is confidential and/or
protected by legal privilege. Any unauthorized review, use, copy,
disclosure or distribution of this e-mail is strictly prohibited. If you
are not the intended recipient, please notify Mitel immediately and destroy
all copies of this e-mail. Mitel does not accept any liability for breach
of security, error or virus that may result from the transmission of this
message.

[Vali]

Hi Nader,

UniMRCP C# wrapper can be indeed used for speech recognition, we use it in a larger project.

Your app did not even write "{0} - ReadFrame [{1} bytes]", right? Probably you missed a detail in modifying the synth example. Looking at its sources, what should be modified (I hope I do not miss something):

UniMRCPStreamTx -> UniMRCPStreamRx
WriteFrame -> ReadFrame
GetData -> SetData
OnOpenStreamTx -> OnOpenStreamRx  (here return the instance of your stream)
UniMRCPSynthesizerChannel -> UniMRCPRecognizerChannel

Please, try to check it again and if it does not help, send me the sources, I will try to debug it.

Cheers,
- Vali


Dne pondělí, 18. srpna 2014 21:58:18 UTC+2 Nader Hijazi napsal(a):

[Vali]

Hi Nader,

let us continue our discussion in public.

To conclude our private communication: The wrapper can be indeed used for recognition with Nuance server. There were some problems with RECOGNIZE message contents and headers, but were easily sorted out.

To avoid crashes, destroy objects manually, do not rely on garbage collection, e.g.

mChannelASR.Dispose();
mSession.Dispose();

To answer your other questions:

I am envisioning listening to source audio calls and setting the data frames in the StreamRx as we go. I will see how this goes...

If you receive audio as a RTP stream or can make the audio source to do so, you should use media bypass. It is not implemented in the wrapper yet, but maybe I will need it soon myself, so I would implement it.

How do you think the DTMF works in the stream? Does SendDTMF work, and what do I have to set?

DTMF generation and detection works. Just set up the generator/detector by UniMRCPAudioTermination::EnableDTMFGenerator() or EnableDTMFDetector() respectively. Inband means generating the actual tones, outband RFC 2833.

What would the StreamRxBuffered be useful for?

 

Is a tread-safe output buffer. You add audio chunks and DTMF digits to it and they are transmitted automatically. You do not need to override the ReadFrame() method.

Cheers,
- Vali

Dne úterý, 19. srpna 2014 10:14:52 UTC+2 Vali napsal(a):

[Sakthikumar Mca]

Hi Friends,

I am new to this. I am facing problem the umc application in the project. I am connecting the unimrcp client with nuance server. The connection opened and the mrcp communication is working fine. But when i give command "run recog" in the umc console. I always receiving "no-input-timeout".

Can you help me to resolve this problem?