[UniMRCP] get the codec frames

[Anthony Masse]

Hi,

I add 2 "mpf codec capabilities" : PCMA and PCMU.

How can I retreive the codec in the "recog_stream_write" function (or "recog_stream_open" function). The "stream->tx_descriptor->payload_type" is always RTP_PT_UNKNOWN

Thanks

Anthony

--
You received this message because you are subscribed to the Google Groups "UniMRCP" group.
To post to this group, send email to uni...@googlegroups.com.
To unsubscribe from this group, send email to unimrcp+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/unimrcp?hl=en.

[Anthony Masse]

In fact, it's more complicated :

If I defined in my plugin one codec capability : PCMU

-> If I receive PCMU, I will have

2010-05-07 11:51:42:018359 02164 [NOTICE] [ASR Plugin] Channel stream open 0

2010-05-07 11:51:42:018359 02164 [INFO]   Source->[PCMU/8000/1]->Bridge->[PCMU/8000/1]->Sink

2010-05-07 11:51:42:018359 02164 [NOTICE] [ASR Plugin] Channel stream write 0

=> OK :  0 (ie : PCMU) in the "Channel stream open" and "Channel stream write" methods

-> if I receive PCMA, I will have 

2010-05-07 11:43:12:421348 02164 [NOTICE] [ASR Plugin] Channel stream open 128

2010-05-07 11:43:12:421348 02164 [INFO]   Source->[PCMA/8000/1]->Decoder->[LPCM/8000/1]->Bridge->[LPCM/8000/1]->Encoder->[PCMU/8000/1]->Sink

2010-05-07 11:43:12:421348 02164 [NOTICE] [ASR Plugin] Channel stream write 128

=> RTP_PT_UNKNOWN

------

If I defined in my plugin two codec capability : PCMU and PMCA

-> If I receive PCMU, I will have

2010-05-07 11:51:42:018359 02164 [NOTICE] [ASR Plugin] Channel stream open 0

2010-05-07 11:51:42:018359 02164 [INFO]   Source->[PCMU/8000/1]->Bridge->[PCMU/8000/1]->Sink

2010-05-07 11:51:42:018359 02164 [NOTICE] [ASR Plugin] Channel stream write 0

=> OK

-> if I receive PCMA, I will have 

2010-05-07 11:56:56:817249 05500 [NOTICE] [ASR Plugin] Channel stream open 8

2010-05-07 11:56:56:817249 05500 [INFO]   Source->[PCMA/8000/1]->Bridge->[PCMA/8000/1]->Sink

2010-05-07 11:56:56:817249 05500 [NOTICE] [ASR Plugin] Channel stream write 8

=> OK

Anthony

2010/5/7 Anthony Masse <amasse...@gmail.com>

[Anthony Masse]

There is a mistake in my previous mail.

the case 3 is not possible :

If I defined in my plugin two codec capability : PCMA and PMCU

-> If I receive PCMU, I will have

=> the result is :

2010-05-07 14:05:33:941188 04696 [INFO]   Source->[PCMU/8000/1]->Decoder->[LPCM/8000/1]->Bridge->[LPCM/8000/1]->Encoder->[PCMA/8000/1]->Sink

and payload_type is sert to 128.

Anthony

In fact, I don't understand why you can set severals codec capabilities because the first is always chosen.

Ps: I'm going to check Patrick mail to re-find any informations about that

Regards

[Arsen Chaloyan]

Hi Anthony,

I've just basically tested all the mentioned cases and everything seems to be working as intended, so you might be misinterpreted the behavior.

The idea is simple enough. You provide your capabilities upon channel creation, those capabilities are considered during offer/answer and the selected codec descriptor can be retrieved when channel is finally opened.

Typically, codec cannot be identified by RTP payload type. Don't forget about dynamic payload types.
The identifier is the codec name. Also, sampling rate and channel count must be considered. RTP payload type is actually set and used in RTP receiver and transmitter only. LPCM has no static payload type assigned (128 - undefined).

MPF frame contains no primary information about the codec type, it just carries the data. Rx and Tx descriptors of the stream (stream->rx_descriptor or tx_descriptor) should be used to identify the codec. Helper functions are available to retrieve these descriptors through channel object.
mrcp_engine_source_stream_codec_get()
mrcp_engine_sink_stream_codec_get()


Let me know in case of any questions.
Regards,

--
Arsen Chaloyan
The author of UniMRCP
http://www.unimrcp.org

[Anthony Masse]

1. In the "recog_engine_channel_create" method, I add two codec capabilities :

mpf_codec_capabilities_add(

&capabilities->codecs,

MPF_SAMPLE_RATE_8000,

"PCMU");

mpf_codec_capabilities_add(

&capabilities->codecs,

MPF_SAMPLE_RATE_8000,

"PCMA");

2. My MRCP client sends the following SETUP :

SETUP rtsp://192.168.1.34:1554/media/speechrecognizer RTSP/1.0

CSeq: 0

Transport: RTP/AVP;unicast;client_port=13350-13351

Content-Type: application/sdp

Content-Length: 230

v=0

o=telisma 1273240811 1273240811 IN IP4 192.168.1.34

s=telisma MRCP Client

c=IN IP4 192.168.1.34

t=0 0

m=audio 13350 RTP/AVP 8 101

a=rtpmap:8 pcma/8000/1

a=rtpmap:101 telephone-event/8000

a=fmtp:101 0-15

a=sendonly

3. The uniMrcp returns :

Source->[PCMA/8000/1]->Decoder->[LPCM/8000/1]->Bridge->[LPCM/8000/1]->Encoder->[PCMU/8000/1]->Sink

when I call "mrcp_engine_source_stream_codec_get" in the channel stream open / write" methods, the function returns NULL

when I call "mrcp_engine_sink_stream_codec_get" in the channel stream open / write" methods, the function returns a valid pointer and the coded is 128.

4. The SETUP result is : 

RTSP/1.0 200 OK

CSeq: 0

Transport: RTP/AVP;unicast;client_port=13350-13351;server_port=5000-5001

Session: 2b93b28abec29246

Content-Type: application/sdp

Content-Length: 195

v=0

o=UniMRCPServer 0 0 IN IP4 192.168.1.34

s=-

c=IN IP4 192.168.1.34

t=0 0

m=audio 5000 RTP/AVP 8 101

a=rtpmap:8 PCMA/8000

a=rtpmap:101 telephone-event/8000

a=fmtp:101 0-15

a=recvonly

---------------------------------------------------------------------

=> Two issue for me :

1. How can I do to knwon the codec of the datas sent to my plugin (via "channel stream write" method)

2. Why uniMRCP doesnt' retur/find) :

             Source->[PCMA/8000/1]-> Bridge -> [PCMA/8000/1]->Sink

My solution today : I only add the PCMU capabilities and set my ASR parameter to PCMU. But I lose the original datas from MRCP client and the uniMRCP take time with the  Decoder/Encoder.

Thanks

Anthony

2010/5/7 Arsen Chaloyan <acha...@gmail.com>

[Arsen Chaloyan]

On Fri, May 7, 2010 at 7:11 PM, Anthony Masse <amasse...@gmail.com> wrote:

1. In the "recog_engine_channel_create" method, I add two codec capabilities :

mpf_codec_capabilities_add(

&capabilities->codecs,

MPF_SAMPLE_RATE_8000,

"PCMU");

mpf_codec_capabilities_add(

&capabilities->codecs,

MPF_SAMPLE_RATE_8000,

"PCMA");

OK

2. My MRCP client sends the following SETUP :

SETUP rtsp://192.168.1.34:1554/media/speechrecognizer RTSP/1.0

CSeq: 0

Transport: RTP/AVP;unicast;client_port=13350-13351

Content-Type: application/sdp

Content-Length: 230

v=0

o=telisma 1273240811 1273240811 IN IP4 192.168.1.34

s=telisma MRCP Client

c=IN IP4 192.168.1.34

t=0 0

m=audio 13350 RTP/AVP 8 101

a=rtpmap:8 pcma/8000/1

a=rtpmap:101 telephone-event/8000

a=fmtp:101 0-15

a=sendonly

OK

3. The uniMrcp returns :

Source->[PCMA/8000/1]->Decoder->[LPCM/8000/1]->Bridge->[LPCM/8000/1]->Encoder->[PCMU/8000/1]->Sink

when I call "mrcp_engine_source_stream_codec_get" in the channel stream open / write" methods, the function returns NULL

when I call "mrcp_engine_sink_stream_codec_get" in the channel stream open / write" methods, the function returns a valid pointer and the coded is 128.

Haven't I just mentioned that numeric payload type is not an identifier? It's initialized only for or in RTP receiver and transmitter.
You should use codec name (descriptor->name), which is properly set as far as I can see.

4. The SETUP result is : 

RTSP/1.0 200 OK

CSeq: 0

Transport: RTP/AVP;unicast;client_port=13350-13351;server_port=5000-5001

Session: 2b93b28abec29246

Content-Type: application/sdp

Content-Length: 195

v=0

o=UniMRCPServer 0 0 IN IP4 192.168.1.34

s=-

c=IN IP4 192.168.1.34

t=0 0

m=audio 5000 RTP/AVP 8 101

a=rtpmap:8 PCMA/8000

a=rtpmap:101 telephone-event/8000

a=fmtp:101 0-15

a=recvonly

OK

---------------------------------------------------------------------

=> Two issue for me :

1. How can I do to knwon the codec of the datas sent to my plugin (via "channel stream write" method)

 above.

2. Why uniMRCP doesnt' retur/find) :

             Source->[PCMA/8000/1]-> Bridge -> [PCMA/8000/1]->Sink

When I initially added and announced this stuff, I mentioned that the algorithm finds the first match (or first available media path), it doesn't look for the best available option. If I'm not mistaken this was discussed in the details with Patrick by that time.

Is it feasible to implement the best matching algorithm? Of course, yes, but ideally, it should be optional to me, as this codec matching stuff may unlikely consume some additional CPU time, depending on the actual number of codecs participating in the o/a.

Is it a priority? No.

My solution today : I only add the PCMU capabilities and set my ASR parameter to PCMU. But I lose the original datas from MRCP client and the uniMRCP take time with the  Decoder/Encoder.

Actually, you may reliably add both PCMU and PCMA. The only note is that best matching algorithm.

[Anthony Masse]

Arsen,

1. Thanks, now I can read the codec name and it's right.

2. About the matching algorithm. In my point of view, there are two thinks :

     - find the best matching algorithm

and /or

     - just check if the input codec is included in the capabilities codec

But I don't check the code to read/understand the mechanism.