Freeswitch: no-input-timeout -- How to capture completion cause

[Wilmar Pérez]

Hi all, 

In the context of SR, I am trying to find out what is the best way to detect when users do not enter (i.e. say) anything.

So, I am doing the simple following test with LUA:

------------------------------------------------

session:execute("play_and_detect_speech" " say: Please say something

detect:unimrcp:uni2 {start-input-timers=false}builtin:speech/transcribe")

 if (session:getVariable('detect_speech_result') ~= nil) then
                freeswitch.consoleLog("NOTICE", "User said something"\n");
     else
                 freeswitch.consoleLog("NOTICE", "User is shy"\n");
end

------------------------------------------------  

The problem is that I am always "getting something".   Looking at the logs I can see that no input was detected but I guess that is being assigned to the detect_speech_result session variable (although I could not get a print out of it)

Question is: how can I access the completion-cause result from in my script?

Furthermore, what I am doing, is it correct or is there a better approach?

Thank you!

--

--------------------------------------------------------

Wilmar Pérez 

[Vahagn Kocharyan]

Hi 

you must check  for completion-cause in result 

Your_result.find('completion-cause', 'completion-cause', 1, true)

--
You received this message because you are subscribed to the Google Groups "UniMRCP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unimrcp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/unimrcp/CAKEO2%3DvUhY%2BwZcduVgGo_srGcyVSUCVpwaZud%2Bf7mC%3Dnb0K5bA%40mail.gmail.com.

[Wilmar Pérez]

Hi Vahagn,

Thanks very much for your reply.  That put me in the right path but I am not very ecpeience with LUA yet,  What is the reference to this function?:

Your_result.find('completion-cause', 'completion-cause', 1, true)  

I cannot find the documentation around it. How does it work?  If I do something like:

detected_speech = session:getVariable('detect_speech_result')

cause = detected_speech.find('completion-cause', 'completion-cause', 2, true)

Am i expecting  cause to be true if completion-cause = 2 and false otherwise?

I've been playing around with it but I have not been able to catch the right logic.

Thank you again!

Wilmar

To view this discussion on the web visit https://groups.google.com/d/msgid/unimrcp/CAG1i3SKBnrMpi8fC6CoN2SerX3VDQZFOY3E7WjsYX8njyaffPg%40mail.gmail.com.

--

--------------------------------------------------------

Wilmar Pérez 

[Vahagn Kocharyan]

HI Wilmar

you can see lua usage 

https://freeswitch.org/confluence/display/FREESWITCH/Serving+Configuration+with+Lua

i am not familiar with lua..in javascript you can check like this Result.indexOf("Completion-Cause:") == -1)

for lua test this. i am not sure but it must work

session:execute("play_and_detect_speech", "say:Welcome detect:unimrcp:ums-v2 {start-input-timers=false}builtin:speech/transcribe");

print("ERR", session:getVariable("detect_speech_result").length)

detect_speech = session:getVariable("detect_speech_result")

if (session:getVariable('detect_speech_result') ~= nil) then

    freeswitch.consoleLog("NOTICE", "User said something");

elseif (detect_speech.find('completion-cause', 'completion-cause', 1, true)) then

    freeswitch.consoleLog("NOTICE", "User said something");

else

     freeswitch.consoleLog("NOTICE", "User is shy");

end

thanks

Vahagn

To view this discussion on the web visit https://groups.google.com/d/msgid/unimrcp/CAKEO2%3Ds96JXYmwQcJ-Ld%3D72RQuUoCOJGnnKxXSz5RxZXFVZWiw%40mail.gmail.com.

[Wilmar Pérez]

Hi Vahagn,

Thanks again for your kind reply and taking the time to suggest the script.  At the end the only way I found to make it work consistently was kind of uggly but until I find a more elegant solution it will have to do:

detected_speech = session:getVariable('detect_speech_result')

if(detected_speech ~= "Completion-Cause: 001" and detected_speech ~= "Completion-Cause: 002") then
   freeswitch.consoleLog("NOTICE", "User said: " .. detected_speech ..  "\n"); -- This is an XML object
elseif(detected_speech == "Completion-Cause: 001") then
   freeswitch.consoleLog("NOTICE", "Recognizer got something but it does not know what to do with it: no-match \n");
elseif(detected_speech == "Completion-Cause: 002") then

   freeswitch.consoleLog("NOTICE", "User is shy \n");
end

Tha leaves me with around 14 more cases I should handle: https://www.lumenvox.com/knowledgebase/index.php?/article/AA-01632/0/MRCP-Event-Messages.html  

Comparing against nil does not really seem to help a lot because detected_speech is always populated (at least this is what I believe happens) by either the XML construct or a corresponding error.

Anyway, it does what it need it to do, I have been testing all scenarios without it breaking.  Thanks a bunch for your help!

Best,

Wilmar.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/unimrcp/CAG1i3SJk1mYsY9iHCyJp8yJpB_Q3FVZ2%3D75p-idHzYcUGjC9AQ%40mail.gmail.com.

--

--------------------------------------------------------

Wilmar Pérez 

[Sean DiSanti]

Here's how I do it in javascript, this is a snippet from a pretty heavily modified SpeechTools.jm from the pizza demo:
if (body.indexOf('Completion-Cause:') == 0){
                                          var code = body.substring(18,21);

                                          switch(code) {
                                            case '002':
                                              // no input
                                              console_log('NOTICE', 'No speech detected');
                                              session.setVariable('asr_speechresult', '');
                                              rv.push('N');
                                              break;
                                            case '001':
                                              // no match
                                              var etel_no_match_mirror_n = session.getVariable('etel_no_match_mirror_n');
                                              if (etel_no_match_mirror_n  == "1"){
                                                console_log('NOTICE', 'Treating no-match like no-input');
                                                session.setVariable('asr_speechresult', '');
                                                rv.push('N');
                                              } else {
                                                console_log('NOTICE', 'Input received but not recognized');
                                                session.setVariable('asr_speechresult', '__NR__');
                                                rv.push('crosstalk');
                                              }
                                              break;
                                            case '006':
                                              // Error
                                              console_log('CRIT', 'UniMRCP returned a recognizer error');
                                              session.setVariable('asr_speechresult', '(RECOGNIZER ERROR)');
                                              session.execute("transfer",  "entry XML asr_error_transfer");
                                              break;
                                            default:
                                              console_log('CRIT', session.getVariable('uuid') +' Unexpected completion code: ' + code);
                                              session.setVariable('asr_speechresult', '(CODE ' +  code +')');
                                              rv.push('crosstalk');
                                          }

                                          var mrcpCompletionCode = code;
                                          session.setVariable('asr_mrcp_code', mrcpCompletionCode);

                                          return rv;
                                        } else {
                                          var mrcpCompletionCode = '000';  // 000 = Success
                                        }

                                        session.setVariable('asr_mrcp_code', mrcpCompletionCode);

To view this discussion on the web visit https://groups.google.com/d/msgid/unimrcp/CAKEO2%3DuOvROKiLpVug3DCUedi%2BJoh5ZME0RK%3DjrFXQDyWVxQkg%40mail.gmail.com.

[Wilmar Pérez]

Thanks very much Sean,

I can see that in a way you also check individual codes. So,  I am not that lost! 

I will have a look to the Pizza demo as well, I keep forgetting it exists.

Best,

Wilmar

To view this discussion on the web visit https://groups.google.com/d/msgid/unimrcp/CAANGHxvzGeE-f4h%3DUdNPWnRSzN1hLKzhn35Q%2BkC9gH-iLP33kA%40mail.gmail.com.

--

--------------------------------------------------------

Wilmar Pérez 

[Sean DiSanti]

No problem. Yeah we don't check for every code, just the ones that we've seen really. My first ASR prototype was based on the pizza demo, and then moving forward I super generalized it down and refactored it down. Now for each ASR interaction, i have an extension that sets some variables, for the sound, and what extension to jump to depending on the classification of the speech result. Then when there is a speech result, it's curl'd to a web service that classifies based on regex rules or a trained model (depending on the program) and returns positive/negative/crosstalk etc, which then sends to the next extension dynamically based on destinations specified in the dialplan variables.

I'm including a simplified example of a single interaction in the dialplan. We make a couple other calls to lua scripts to set custom variables in our call data, but I've removed those. The classifier which is called from the etel_asr.js sets the next extension to transfer to based on the variables set above it. I've added comments to explain the variables and flow.

 <!-- offer -->
        <extension name="offer">
            <condition field="destination_number" expression="^offer$">
                <action application="set" data="group_name=RegxMatch" /> <!-- this is a regex  based program, this lets the classifier know not to use the trained model -->
                <action application="set" data="asr_context=offer" /> <!-- current context, matches the extension expression -->
                <action application="set" data="top_sound=offer" />  <!-- sound file to play (/usr/share/freeswitch/sounds/en/us/callie/custom/this.wav) -->
                <action application="set" data="asr_positive=qualifier_o" /> <!-- extension to transfer to if classifier says speech was positive -->
                <action application="set" data="asr_crosstalk=dnq_pretran" /> <!-- or crosstalk -->
                <action application="set" data="asr_negative=dnq_pretran" /> <!-- or negative -->
                <action application="set" data="etel_gather=1" /> <!-- this is to let the javascript know whether to do a play and detect, or just a play -->
                <action application="javascript" data="etel_asr.js"/> <!-- This is the javascript that uses the variables above to play a sound and detect speech, then classify the result -->
                <action application="transfer" data="${asr_context} XML ${current_dialplan}" /> <!-- this variable is set by the javascript to whatever the next interaction will be -->
            </condition>
        </extension>

To view this discussion on the web visit https://groups.google.com/d/msgid/unimrcp/CAKEO2%3DtrUCzBuRsQH4sC__uN1kMeFwh1cuXK6RwG5qP1eH9Ozg%40mail.gmail.com.

[Wilmar Pérez]

Awesome Sean!. Very generous of you to provide all those details. With all this information being so difficult to find (or difficult to grasp for me) there is nothing like someone's experience to get a good push!

Thanks again!

To view this discussion on the web visit https://groups.google.com/d/msgid/unimrcp/CAANGHxvN3xk3nJzm3tL%2Bz%3DqfgCsSL6pCvkYQ%3DbGT7P3R6yfm3g%40mail.gmail.com.

--

--------------------------------------------------------

Wilmar Pérez 

[Sean DiSanti]

No problem, I started at 0 almost a year ago exactly and remember how hard it was to find or understand some of the documentation, let alone adjust the implementation to match our needs. I'm glad to be able to save others some of that pain.

To view this discussion on the web visit https://groups.google.com/d/msgid/unimrcp/CAKEO2%3DuYtq9gXEoaPHB-7PXjSe_nHu%3DTHaxRjCOi0AyEOD%3D3yw%40mail.gmail.com.

[Wilmar Pérez]

Yes, it is sort of a nightmare. So many choices and not enough information on what path to follow and exactly how to go about it.  One simple case: choose how to code: dialplan, LUA, python, JAVA, etc, ESL?  I decided to go for LUA for it was the only one I could find some information but I am not 100% convinced..... anyway.... decisions....

Thanks again!

To view this discussion on the web visit https://groups.google.com/d/msgid/unimrcp/CAANGHxvWKK-9LMvTi1CfJdqxYNkmQNkhzyruQAfa4tQzuW71rQ%40mail.gmail.com.

--

--------------------------------------------------------

Wilmar Pérez