Asterisk client SynthAndRecog timers Yandex SR

216 views
Skip to first unread message

Роман Милованов

unread,
Aug 27, 2020, 3:14:52 AM8/27/20
to UniMRCP

Problem with the SynthAndRecog command in the Yandex plugin

The problem is that if the keyword for synthesis is relatively large, then recognition doesn't work after the synthesis ends.



This is where recognition works after the synthesis is complete:

SynthAndRecog("Привет, как дела. погодка сегодня отличная.",builtin:speech/transcribe,t=30000&sit=0&sint=3000&sct=400&b=1&nit=3000&ct=0.7&spl=ru-RU&sl=0.8&uer=0&vn=oksana&pr=1)



Recognition doesn't work here after the synthesis is complete:

SynthAndRecog("хорошо, следующий интент не открытый. как твои дела? Я вот сейчас проверяю прерывания и я очень надеюсь что они будут работать хорошо. А вообще сегодня очень хорошая погода. хочет ся пойти погулять.",builtin:speech/transcribe,t=30000&sit=0&sint=3000&sct=400&b=1&nit=3000&ct=0.7&spl=ru-RU&sl=0.8&uer=0&vn=oksana&pr=1)



There is a suspicion that such and such a timer does not start

I have used all the timers, and I only start these timers when the synthesis ends:
sit=0

But still when a person starts talking the system for some reason for some timeout shuts down and does not start recognition.


we now have a new version of the plugin as it says here:

https://groups.google.com/g/unimrcp/c/UKaGI7KK1yI



Has anyone fixed this?

Роман Милованов

unread,
Aug 27, 2020, 6:16:39 AM8/27/20
to UniMRCP
MRCP/2.0 393 RECOGNIZE 1
Channel-Identifier: 3fa640dbbd984594@speechrecog
Content-Type: text/uri-list
Cancel-If-Queue: false
Start-Input-Timers: false
Recognition-Timeout: 30000
Confidence-Threshold: 0.7
Speech-Incomplete-Timeout: 3000
Speech-Language: ru-RU
No-Input-Timeout: 3000
Sensitivity-Level: 0.9
Speech-Complete-Timeout: 400
Content-Length: 25





2020-08-27 08:02:48:059005 [NOTICE] Speech Complete <5fb36c53f66741f4@yandexss>
2020-08-27 08:02:48:059256 [INFO]   Process SPEAK-COMPLETE Event <5fb36c53f66741f4@speechsynth> [1]
2020-08-27 08:02:48:059266 [NOTICE] State Transition SPEAKING -> IDLE <5fb36c53f66741f4@speechsynth>
2020-08-27 08:02:48:059319 [INFO]   Send MRCPv2 Data 10.11.11.10:1544 <-> 10.11.0.7:46966 [122 bytes]
MRCP/2.0 122 SPEAK-COMPLETE 1 COMPLETE
Channel-Identifier: 5fb36c53f66741f4@speechsynth
Completion-Cause: 000 normal




2020-08-27 08:02:50:718926 [INFO]   Speech Detector State Transition NO-INPUT -> IN-PROGRESS [11940 ms] <3fa640dbbd984594>
2020-08-27 08:02:50:718997 [INFO]   Start Input Timer [30000 ms] <3fa640dbbd984594>
2020-08-27 08:02:50:719286 [WARN]   Failed to Send Input <3fa640dbbd984594@yandexsr>
2020-08-27 08:02:50:719309 [INFO]   Received Response: status [0] result-count [0] <3fa640dbbd984594@yandexsr>
2020-08-27 08:02:50:729135 [INFO]   Input Complete [stopped] size=9600 bytes, dur=610 ms <3fa640dbbd984594@yandexsr>
2020-08-27 08:02:50:729412 [INFO]   Process RECOGNITION-COMPLETE Event <3fa640dbbd984594@speechrecog> [1]
2020-08-27 08:02:50:729428 [INFO]   State Transition RECOGNIZING -> RECOGNIZED <3fa640dbbd984594@speechrecog>
2020-08-27 08:02:50:729471 [INFO]   Send MRCPv2 Data 10.11.11.10:1544 <-> 10.11.0.7:46962 [138 bytes]
MRCP/2.0 138 RECOGNITION-COMPLETE 1 COMPLETE
Channel-Identifier: 3fa640dbbd984594@speechrecog
Completion-Cause: 006 recognizer-error

четверг, 27 августа 2020 г., 11:14:52 UTC+4 пользователь Роман Милованов написал:

Роман Милованов

unread,
Aug 27, 2020, 6:16:39 AM8/27/20
to UniMRCP
четверг, 27 августа 2020 г. в 11:14:52 UTC+4, Роман Милованов:

Роман Милованов

unread,
Aug 27, 2020, 6:16:40 AM8/27/20
to UniMRCP
headers recog in LOG:


MRCP/2.0 393 RECOGNIZE 1
Channel-Identifier: f4ad047faa684ab4@speechrecog
Content-Type: text/uri-list
Cancel-If-Queue: false
Start-Input-Timers: false
Sensitivity-Level: 0.9
Speech-Complete-Timeout: 400
Recognition-Timeout: 30000
Confidence-Threshold: 0.7
Speech-Incomplete-Timeout: 3000
Speech-Language: ru-RU
No-Input-Timeout: 3000
Content-Length: 25




LOG:

2020-08-27 07:48:10:328990 [NOTICE] Speech Complete <a5292022d4e54b92@yandexss>
2020-08-27 07:48:10:329208 [INFO]   Process SPEAK-COMPLETE Event <a5292022d4e54b92@speechsynth> [1]
2020-08-27 07:48:10:329218 [NOTICE] State Transition SPEAKING -> IDLE <a5292022d4e54b92@speechsynth>
2020-08-27 07:48:10:329264 [INFO]   Send MRCPv2 Data 10.11.11.10:1544 <-> 10.11.0.7:46760 [122 bytes]
MRCP/2.0 122 SPEAK-COMPLETE 1 COMPLETE
Channel-Identifier: a5292022d4e54b92@speechsynth
Completion-Cause: 000 normal


2020-08-27 07:48:12:598939 [INFO]   Speech Detector State Transition NO-INPUT -> IN-PROGRESS [12680 ms] <f4ad047faa684ab4>
2020-08-27 07:48:12:598977 [INFO]   Start Input Timer [30000 ms] <f4ad047faa684ab4>
2020-08-27 07:48:12:599551 [WARN]   Failed to Send Input <f4ad047faa684ab4@yandexsr>
2020-08-27 07:48:12:599577 [INFO]   Received Response: status [0] result-count [0] <f4ad047faa684ab4@yandexsr>
2020-08-27 07:48:12:609044 [INFO]   Input Complete [stopped] size=9600 bytes, dur=610 ms <f4ad047faa684ab4@yandexsr>
2020-08-27 07:48:12:609380 [INFO]   Process RECOGNITION-COMPLETE Event <f4ad047faa684ab4@speechrecog> [1]
2020-08-27 07:48:12:609394 [INFO]   State Transition RECOGNIZING -> RECOGNIZED <f4ad047faa684ab4@speechrecog>
2020-08-27 07:48:12:609429 [INFO]   Send MRCPv2 Data 10.11.11.10:1544 <-> 10.11.0.7:46756 [138 bytes]
MRCP/2.0 138 RECOGNITION-COMPLETE 1 COMPLETE
Channel-Identifier: f4ad047faa684ab4@speechrecog
Completion-Cause: 006 recognizer-error


четверг, 27 августа 2020 г. в 11:14:52 UTC+4, Роман Милованов:

Problem with the SynthAndRecog command in the Yandex plugin

Мария Ветошкина

unread,
Aug 28, 2020, 9:50:42 AM8/28/20
to UniMRCP
The same version, the same issue =(

четверг, 27 августа 2020 г. в 15:16:40 UTC+5, romarios...@gmail.com:

Arsen Chaloyan

unread,
Aug 29, 2020, 6:20:07 PM8/29/20
to UniMRCP
Well, I can make a similar statement too. This is the same problem which Yandex fixed back in May 2019.

See this post and refer to Jira #UNI-50 if you happen to bring this issue to the attention of Yandex.

--
You received this message because you are subscribed to the Google Groups "UniMRCP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unimrcp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/unimrcp/33d00f53-c77a-4758-8dcd-897b9ea1b1ddn%40googlegroups.com.


--
Arsen Chaloyan
Author of UniMRCP
http://www.unimrcp.org

Роман Милованов

unread,
Aug 30, 2020, 1:25:02 AM8/30/20
to UniMRCP
Yes indeed I have an error in the log. But we also tried similar ones to your plugin, and everything worked with the same request. they probably have a different start to the session. But I will write to Yandex in this case. And I will clarify how others did this plugin.

2020-08-30 05:18:35:158704 [WARN]   gRPC Status: Chunks have to be sent with at least one per 5 seconds frequency <443167b04f004b2f@yandexsr>



LOG:

MRCP/2.0 138 RECOGNITION-COMPLETE 1 COMPLETE
Channel-Identifier: 443167b04f004b2f@speechrecog
Completion-Cause: 006 recognizer-error


2020-08-30 05:18:35:158704 [WARN]   gRPC Status: Chunks have to be sent with at least one per 5 seconds frequency <443167b04f004b2f@yandexsr>
2020-08-30 05:18:35:164713 [INFO]   TCP/MRCPv2 Peer Disconnected 10.11.11.10:1544 <-> 10.11.0.7:37494
2020-08-30 05:18:35:165933 [INFO]   Receive SIP Event [nua_i_bye] Status 200 Session Terminated [SIP-Agent-1]
2020-08-30 05:18:35:165952 [INFO]   Receive SIP Event [nua_i_state] Status 200 Session Terminated [SIP-Agent-1]

воскресенье, 30 августа 2020 г. в 02:20:07 UTC+4, Arsen Chaloyan:

Роман Милованов

unread,
Aug 30, 2020, 11:24:03 AM8/30/20
to UniMRCP
Arsen what's the problem with re-opening the session if the current timer is not completed yet?

воскресенье, 30 августа 2020 г., 2:20:07 UTC+4 пользователь Arsen Chaloyan написал:
Well, I can make a similar statement too. This is the same problem which Yandex fixed back in May 2019.

See this post and refer to Jira #UNI-50 if you happen to bring this issue to the attention of Yandex.

To unsubscribe from this group and stop receiving emails from it, send an email to uni...@googlegroups.com.

Arsen Chaloyan

unread,
Sep 4, 2020, 9:41:38 PM9/4/20
to UniMRCP
Hi Roman,

I have just had the chance to look back at this problem hoping that Yandex may have fixed the behavior, but no, the problem unfortunately still persists, and I do not know what the root cause is.

Anyway, there is no good way to fix the issue from the client/plugin side. Or more specifically, the problem can be fixed by the client but only at the expense of the response time.

The gRPC method AsyncStreamingRecognize is called as soon as the RECOGNIZE request is received. And this is done on purpose in order to save some time on async initialization and not wait for a response when the caller starts speaking. The same routine does work with Google as intended.

I'll consider implementing an option to call AsyncStreamingRecognize late when the streaming is about to start but not when RECOGNIZE is received. This unavoidably will add a delay in the overall processing, but it is better to respond late rather than fail. Please note that no even estimated timeline can be provided at this time.



To unsubscribe from this group and stop receiving emails from it, send an email to unimrcp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/unimrcp/3e8e6444-edf6-4ac9-8ebe-1b07c409a741o%40googlegroups.com.

Роман Милованов

unread,
Sep 5, 2020, 12:24:23 AM9/5/20
to UniMRCP
Arsen greetings. We found a way around this error. This is of course the crutches but seems to work. We just break phrases into a few small fragments.

суббота, 5 сентября 2020 г., 5:41:38 UTC+4 пользователь Arsen Chaloyan написал:

Мария Ветошкина

unread,
Sep 7, 2020, 12:39:37 AM9/7/20
to uni...@googlegroups.com
Hi Arsen.
Yandex team replied that it was the expected behavior of the service: if no data was sent within 5 seconds, the service considered the session to be over and remote side has forgotten to close it.
The same 006 error happens if an abonent is keeping silent more than 5-10 seconds.
I still didn't find any workaround.



вс, 30 авг. 2020 г. в 03:20, Arsen Chaloyan <acha...@gmail.com>:
You received this message because you are subscribed to a topic in the Google Groups "UniMRCP" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/unimrcp/4_lH8Lx2PwA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to unimrcp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/unimrcp/CAC5Usc%2ByULwnU9ELFWM-rHVKwHdLS_CVTG0Kp1PJA%2BnKR673UA%40mail.gmail.com.

Мария Ветошкина

unread,
Sep 8, 2020, 12:45:10 AM9/8/20
to uni...@googlegroups.com
Yandex team also recommended sending “silence” for recognition to save the connection opened.

пн, 7 сент. 2020 г. в 09:39, Мария Ветошкина <maria....@gmail.com>:

Arsen Chaloyan

unread,
Sep 17, 2020, 8:51:46 PM9/17/20
to UniMRCP
Hi Maria,

Thanks for coordinating the responses back and forth. I am providing attached an extracted log of Jira #Uni-50 for information only.

Instead of debating any further regarding possible approaches or the origin of the problem, I'd rather save everyone's time by tackling the problem in the plugin.

You may download and install an early dev RPM as follow


This version allows placing a gRPC call at a later stage when audio activity is detected. There is a global configuration parameter which can be set in umsyandexsr.xml. The parameter defaults to false, if not specified. You would need to set it to true.

   <streaming-recognition
      late-grpc-call="true"

Furthermore, the behavior can be controlled per recognition request as follows

builtin:speech/transcribe?late-grcp-call=true

Hope this helps.


To view this discussion on the web visit https://groups.google.com/d/msgid/unimrcp/CAF51ix_kiKQycLSGkpySxRj%2BF-_hNEXijfSM%2B1KEsp3wc9ujxQ%40mail.gmail.com.
UNI-50.pdf

Мария Ветошкина

unread,
Sep 22, 2020, 12:45:00 AM9/22/20
to UniMRCP
Arsen, thanks a lot! It works.

пятница, 18 сентября 2020 г. в 05:51:46 UTC+5, Arsen Chaloyan:

Роман Милованов

unread,
Sep 22, 2020, 2:16:00 AM9/22/20
to UniMRCP
can I get a deb package?

пятница, 18 сентября 2020 г., 4:51:46 UTC+4 пользователь Arsen Chaloyan написал:
Hi Maria,

Thanks for coordinating the responses back and forth. I am providing attached an extracted log of Jira #Uni-50 for information only.

Instead of debating any further regarding possible approaches or the origin of the problem, I'd rather save everyone's time by tackling the problem in the plugin.

You may download and install an early dev RPM as follow


This version allows placing a gRPC call at a later stage when audio activity is detected. There is a global configuration parameter which can be set in umsyandexsr.xml. The parameter defaults to false, if not specified. You would need to set it to true.

   <streaming-recognition
      late-grpc-call="true"

Furthermore, the behavior can be controlled per recognition request as follows

builtin:speech/transcribe?late-grcp-call=true

Hope this helps.


On Mon, Sep 7, 2020 at 9:45 PM Мария Ветошкина <maria...@gmail.com> wrote:
Yandex team also recommended sending “silence” for recognition to save the connection opened.

пн, 7 сент. 2020 г. в 09:39, Мария Ветошкина <maria...@gmail.com>:
To unsubscribe from this group and stop receiving emails from it, send an email to uni...@googlegroups.com.


--
Arsen Chaloyan
Author of UniMRCP
http://www.unimrcp.org

--
You received this message because you are subscribed to a topic in the Google Groups "UniMRCP" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/unimrcp/4_lH8Lx2PwA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to uni...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "UniMRCP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to uni...@googlegroups.com.

Роман Милованов

unread,
Sep 22, 2020, 8:56:36 AM9/22/20
to UniMRCP
works only if you specify a string in the umsyandexsr file.xml in the dialplan this doesn't work



пятница, 18 сентября 2020 г., 4:51:46 UTC+4 пользователь Arsen Chaloyan написал:
Hi Maria,

Thanks for coordinating the responses back and forth. I am providing attached an extracted log of Jira #Uni-50 for information only.

Instead of debating any further regarding possible approaches or the origin of the problem, I'd rather save everyone's time by tackling the problem in the plugin.

You may download and install an early dev RPM as follow


This version allows placing a gRPC call at a later stage when audio activity is detected. There is a global configuration parameter which can be set in umsyandexsr.xml. The parameter defaults to false, if not specified. You would need to set it to true.

   <streaming-recognition
      late-grpc-call="true"

Furthermore, the behavior can be controlled per recognition request as follows

builtin:speech/transcribe?late-grcp-call=true

Hope this helps.

On Mon, Sep 7, 2020 at 9:45 PM Мария Ветошкина <maria...@gmail.com> wrote:
Yandex team also recommended sending “silence” for recognition to save the connection opened.

пн, 7 сент. 2020 г. в 09:39, Мария Ветошкина <maria...@gmail.com>:
To unsubscribe from this group and stop receiving emails from it, send an email to uni...@googlegroups.com.


--
Arsen Chaloyan
Author of UniMRCP
http://www.unimrcp.org

--
You received this message because you are subscribed to a topic in the Google Groups "UniMRCP" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/unimrcp/4_lH8Lx2PwA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to uni...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "UniMRCP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to uni...@googlegroups.com.

Arsen Chaloyan

unread,
Sep 26, 2020, 3:08:46 PM9/26/20
to UniMRCP
Hi Roman,

You should be able to set the parameter both globally and per recognition request. In my previous post, though, I misspelled "grpc" as "grcp". Use "late-grpc-call" consistently.

Similar development deb package can be downloaded for xenial and bionic as follows:

To unsubscribe from this group and stop receiving emails from it, send an email to unimrcp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/unimrcp/59394a3b-faac-4a36-a1e1-81cfa0b5e8ebo%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages