voice recognition on asterisk and unimrcp

[Alexander Butenko]

Hello,

Im trying to implement voice driven IVR and stuck with a problem that as soon as call hits SpeechBackground() or MRCPRecog() bargein comes in and blocking playback of the message.

Is there is something im missing? Thanks

Asterisk 11.13.1 and UniMRCP Server 1.3.0 with quite default configuration.

My test dialplans are:

exten => s,1,Answer
same=n,SpeechCreate
same=n,SpeechLoadGrammar(digits,/etc/asterisk/grammars/digits.jsgf)
same=n,SpeechProcessingSound(/usr/share/asterisk/sounds/en/beep.gsm)
same=n,SpeechStart
same=n,SpeechBackground(custom/000010)
same=n,Noop(${SPEECH(status)} ${SPEECH(spoke)} ${SPEECH(results)} ${SPEECH_SCORE(result number)} ${SPEECH_TEXT(result number)} ${SPEECH_GRAMMAR(result number)} )


exten => s,1,MRCPRecog(builtin:grammar/digits,t=3000&f=custom/000010)
same=n,Noop(1, status: ${RECOG_STATUS}, completion-cause: ${RECOG_COMPLETION_CAUSE}, result: ${RECOG_RESULT})

[David Taieb]

Dont know if relevant here but I had the same problem for days with a virtual machine until I found out I stupidly forgot to mute the line in and since I was playing the prompt on speaker, line in would hear the prompt and think caller was speaking. Maybe try with a headset and make sure sound lines are separated

Good luck

--
You received this message because you are subscribed to the Google Groups "UniMRCP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unimrcp+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alexander Butenko]

I were trying to dial in via couple cells/skype and a softphone via 2 sip trunks. So seems thats not related unfortunately.

Alex

[Arsen Chaloyan]

You didn't mention the recognition engine plugged into the UniMRCP server. If demo_recog is in use, then be aware that it uses a primitive energy-based voice activity detector, which may trigger a false START-OF-INPUT event. Of course, you can tune corresponding parameters, but ultimately I'd go for a smarter VAD.

--

Arsen Chaloyan
Author of UniMRCP
http://www.unimrcp.org

[Alexander Butenko]

Thank you very much for this tip. Its appeared that i havent got whole idea. I have installed older version of unimrcp and pocketsphinx and this is working as it should now.
As I see you have removed pocketsphinx backend support in favor of other engines. Can you suggest better opensource alternative?

Thank you, Alex

[Arsen Chaloyan]

Glad it helped, Alex. I had to discontinue the PocketSphinx plugin not in favor of other engines but because of lack of support in its open source development.

[Ernest Woźniak]

Hi Arsen,

I am having exactly the same problem, i.e. SpeechBackground doesn't play the complete prompt message and starts recognition. I am also using pocketsphinx, unimrcp and asterisk. The installation is recent, from binaries, so I believe I have the most, up to date versions. My setup is on Ubuntu 16.4 under VirtualBox. I am doing a call from my mobile using Zopier, which uses SIP protocol. If I do the call from the same PC where my VirtualBox is installed, using X-Lite (also SIP), this problem doesn't occur.

I don't fully understand your suggestion. I was trying to tweak some parameters in umspocketsphinx.xml file. Didn't work. I don't follow your suggestion with "smarter VAD". Does it mean I would have to install your unimrcp-pocketsphinx plugin from sources, and change the code by considering smarter VAD? If yes, then could you recommend any open VAD API and give a hint on where the change would have to be done?

Thank you,
Ernest

[Arsen Chaloyan]

Hi Ernest,

The post you refer to is dated back in 2015. At that time, the commercial PocketSphinx plugin did not even exist. In fact, it shares very little with the open source version. The VAD modules used in the two also differ.

So, not sure I follow you right. If you can provide some logs from working and not working cases, I'll take a closer look. But you may need to disable barge-in on Asterisk.

--
You received this message because you are subscribed to the Google Groups "UniMRCP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unimrcp+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ernest Woźniak]

Hi Arsen,

thank you for your prompt reply!

I know that the post is from 2015, and I know that unimrcp-pocketsphinx has changed and is now commercial, because recently I was requesting a trial licence :)

Nevertheless this problem persists at least in my case. In fact I've had the same issue with unimrcp-pocketsphinx plugin that I've installed from sources (non commercial version), and I've thought that usage of the most up to date versions will solve my problem. This seems not to be the case for me.

I send you 4 files:

1. no_problem_x_lite_asterisk.log -> this is console output of asterisk when making call from x-lite and there is no problem. Later on there are some errors referring to the problem with database connection but this has nothing to do with unimrcp-pocketsphinx so please don't bother with them.
2. no_problem_x_lite_unimrcpserver.log -> similar as above, just that it is console output of unimrcpserver
3. problem_mobile_zopier_asterisk.log -> console output of asterisk when making call from my mobile using zoiper
4. problem_mobile_zoiper_unimrcpserver.log -> similar as above but related to unimrcpserver console output.

I also provide two links to videos that I've made:

1. https://www.youtube.com/watch?v=33ZO8b8Akeo - NO problem -> I make a call from X-Lite, I get the prompt (SpeechBackground), I provide my answer, I get the prompt with question if my answer is the recognized number "10" in this case, I say "yes", and then it continues but it is not relevant.
2. https://www.youtube.com/watch?v=xY-nZ1ouDzI - PROBLEM -> I make a call from my mobile with zoiper, I get the prompt (SpeechBackground), but it is being interrupted (you can compare it with the previous video), and dialplan goes to the prompt with question if my answer is the recognized number "2" which is of course incorrect because I didn't even have the chance to provide my answer.

If there are other logs which could be more useful, please let me know. At the moment, the only solution which I see and which works for me, is to replace:

same => n,SpeechBackground(custom/pl/Komendy/podaj-linie,5)

with

same => n,Playback(custom/pl/Komendy/podaj-linie)

same => n,SpeechBackground(,5) 

Thank you,

Ernest

[Arsen Chaloyan]

Hi Ernest,

Thanks for such complete details. The more complete data is provided, the shorter is the path to resolution.

The following log statements extracted from problem_mobile_zoiper_unimrcpserver.log indicate that the recognition normally started.

2019-03-12 22:14:34:563438 [INFO]   Send MRCPv2 Data 192.168.0.12:1544 <-> 192.168.0.12:40622 [83 bytes]
MRCP/2.0 83 6 200 IN-PROGRESS
Channel-Identifier: 4f8f94cfe0654ad9@speechrecog

And then in 2 sec, the MRCP session was closed by the client, which is not the intended behavior.

2019-03-12 22:14:36:255375 [INFO]   TCP/MRCPv2 Peer Disconnected 192.168.0.12:1544 <-> 192.168.0.12:40622
2019-03-12 22:14:36:256059 [INFO]   Receive SIP Event [nua_i_bye] Status 200 Session Terminated [SIP-Agent-1]
2019-03-12 22:14:36:256077 [INFO]   Receive SIP Event [nua_i_state] Status 200 Session Terminated [SIP-Agent-1]

I doubt the problem has any relevance to VAD, though.

Moving to the corresponding Asterisk logs, I found the same IN-PROGRESS response

[Mar 12 22:14:34] NOTICE[22040]: src/mrcp_client_connection.c:633  Receive MRCPv2 Data 192.168.0.12:40622 <-> 192.168.0.12:1544 [83 bytes]
MRCP/2.0 83 6 200 IN-PROGRESS
Channel-Identifier: 4f8f94cfe0654ad9@speechrecog

The above is OK. However, what I see straight below makes me believe this might be root cause of the problem.

       > 0x22e5700 -- Strict RTP learning complete - Locking on source address 192.168.0.3:51990

Then comes the session termination, which is also observed in the server logs.

[Mar 12 22:14:36] NOTICE[22068][C-00000001]: res_speech_unimrcp.c:291 uni_recog_destroy: (RSU-0) Destroy speech resource

Next, I would make a network capture to see what exactly is sent to the MRCP server in this case. You can use tcpdump for this purpose.

It is not clear to me why playing back the prompt separately helps, as you mentioned. This might be some problem inside the res_speech module. Also, what is the sampling rate used by your mobile network. I recall some problems in res_speech using 16 kHz.

Have you tried the same with app_unimrcp and MRCPRecog in particular.

[Ernest Woźniak]

Hello Arsen,

thank you very much for your effort on investigating this problem.

I have some interesting findings:

In my case the problem depends on the SIP phone that I am using on my mobile, i.e.:

1. IPhone 5s with Zoiper Lite v3.21 - problem exists

2. Samsung S7 with Zoiper IAX SIP VOIP v 2.7.12 - problem exists

3. IPhone 5s with VaxPhone v8.6.0.2 - problem does NOT exist

4. Samsung S7 with MizuDroid - initial problem does not exist, except that the initial prompt is cut at the beginning (better than cut at the end as this influences recognition ;)) - see asterisk logs

   -- Executing [631@from-internal:1] Answer("SIP/6001-0000000a", "") in new stack

[Mar 26 08:58:12] WARNING[4927][C-0000000b]: res_rtp_asterisk.c:6627 ast_rtp_read: RTP Read too short

[Mar 26 08:58:12] WARNING[4927][C-0000000b]: res_rtp_asterisk.c:6627 ast_rtp_read: RTP Read too short

[Mar 26 08:58:12] WARNING[4927][C-0000000b]: res_rtp_asterisk.c:6627 ast_rtp_read: RTP Read too short

[Mar 26 08:58:12] WARNING[4927][C-0000000b]: res_rtp_asterisk.c:6627 ast_rtp_read: RTP Read too short

    -- Executing [631@from-internal:2] SpeechCreate("SIP/6001-0000000a", "") in new stack

Also, what is interesting is that for the problematic cases, first execution of SpeechBackground is causing the problem, i.e. the prompt is interrupted and recognition doesn't give proper result, however if dialplan proceeds further where there are other calls to SpeechBackground, these work fine.

I also provide you with log from tcpdump for the faulty case nr 2. 

I am using 8kHz speech recog model.

And I haven't tried with MRCPRecog but I can still try to see whether it changes something.

Do you think that we should further investigate that or simply assume that these are problems with the clients?

Ultimately I would like to make phone calls from the mobile without using SIP client, but using GSM network, hence that's what interests me the most. I just need to figure out first how to do this, and I can provide information how this works.

Thank you Arsen!

Ernest

[Arsen Chaloyan]

Hello Ernest,

While it is not clear to me what the differences are between the SIP phones, I would be very confident that the problem is in the handling of incoming SIP/RTP leg but not MRCP one.

If you could provide a network capture in pcap format, I would still take a look. But you may get a better help in troubleshooting by posting this issue to the Asterisk community.