Kurento AMR-WB Support

[mehmet ozisik]

• Operating system (Ubuntu 16.04, 18.04, etc.) -->  Ubuntu 18.04
  
• Installation method (apt-get, Docker, AWS, build from sources) --> apt-get
•  Kurento version --> 6.14.0

I am dealing with AMR-WB relay scnaerio.

A---------> Kurento --------------->B

User A sends AMR-WB packet to B via RtpEndpoint. But I get below error in Kurento :

2020-08-04T01:55:29,838710 27736 0x00007f857d17e700    info GST_PADS                  gstpad.c:2462 gst_pad_link_full()  link between agnosticbin_tee3:src_1 and kmsparsetreebin1:ghost1 failed: no common format

I guess this is blocking the AMR-WB audio packets to be relayed as is to B.

My SDP is like below :

     "v=0\r\n"
            "o=ksession 3804937533 3804937533 IN IP4 10.154.22.20\r\n"
            "s=ksession Session\r\n"
            "c=IN IP4 192.168.1.100\r\n"
            "t=0 0\r\n"
            "m=audio 6000 RTP/AVP 96\r\n"
            "a=rtpmap:96 AMR-WB/16000/1\r\n"
            "a=fmtp:96 encoding-params=1;crc=0;robust-sorting=0;interleaving=0;mode-change-period=1;mode-change-capability=2;mode-change-neighbor=0;max-red=0;octet-align=1\r\n"

When I change "a=rtpmap:96 AMR-WB/16000\r\n" to "a=rtpmap:96 AMR/8000r\n" AMR audio packets packets are relayed to B.

May Kurento codes are blocking something due to a miss definiton for AMR-WB.

Also, I have added below lines into the SdpEndpoint.conf.json:

        {"name": "AMR-WB/16000/1"},
        {"name": "AMR/16000"},
        {"name": "AMR/16000/1"}

I would like to ask you that can you advise on a way for the solution?

PS: Attached full detailed log taken when client starts to send AMR-WB audio packets.

[Juan Navarro]

I think it's probably because only AMR is defined in the list of codec caps in the file kms-core/src/gst-plugins/commons/kmsagnosticcaps.h

This is not 100% sure because I have not worked on adding codec support, but maybe adding the correct caps for AMR-WB it would make Kurento accept that type of audio.

If you want to tinker with it, looks like you'd need to find out what is the caps name that GStreamer gives to AMR-WB (*probably* something like "audio/AMR-WB"), and add it to the list.

--
Juan Navarro
Kurento maintainer & developer
@j1elo at GitHub, Twitter

--
You received this message because you are subscribed to the Google Groups "kurento" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/78bad547-4d9c-435d-a80c-09d7f7abf901o%40googlegroups.com.

[Juan Navarro]

I had a quick look at it (with the GStreamer command "gst-inspect-1.5 amrwbdec"), and it looks the line to add would something like this:

"audio/AMR-WB,rate=16000,channels=1;" \

That should add the capabilities that match those of the "amrwbdec" GStreamer decoder for ARM-WB. As I said this was a quick look so I'm not sure if later on in the pipeline, the correct decoder would be chosen automatically when this codec is present at the input. But this is a good step for a first approach.

To build your customized version of kms-core, the easiest method is using the kurento-buildpackage Docker image: https://doc-kurento.readthedocs.io/en/latest/dev/dev_guide.html#kurento-buildpackage-docker-image

--
Juan Navarro
Kurento maintainer & developer
@j1elo at GitHub, Twitter

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/f33deb5b-a70b-2e34-373c-359467b365a6%40gmx.es.

[mehmet ozisik]

Thanks for the answer.

Actually I had added this line "audio/AMR-WB,rate=16000,channels=1;" \  into kmsagnosticcaps.h as you have mentioned and rebuilt. Reason was the same. 

Interestingly the KMS_AGNOSTIC_FORMATS_AUDIO_CAPS was being printed in the logs somewhere but AMR-WB was not in it although it exists within compiled libs. voamrwbenc also was being imported according to startup logs of Kurento.

I had suspected that there is a dependency to gstreamer within Kurento codes and deleting it with some reason. 

To unsubscribe from this group and stop receiving emails from it, send an email to kur...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/78bad547-4d9c-435d-a80c-09d7f7abf901o%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "kurento" group.

To unsubscribe from this group and stop receiving emails from it, send an email to kur...@googlegroups.com.

[PIYUSH BADKUL]

Hey, in our case of Transcoding from PCMA to AMR-WB, we hear a audio for a starting time of 5 seconds and then a total silence is observed.

I am getting warning message in kurento 

0:00:27.154301569  4831 0x14d704002720 WARN         rtpsynchronizer kmsrtpsynchronizer.c:561:kms_rtp_synchronizer_process_rtp_buffer_mapped:<KmsRtpSynchronizer@0x14d72c020d80> [Sorted mode] Fix PTS not increasing monotonically, SSRC: 3037524492, seq: 526, rtp_ts: 127015436, ext_ts: 127015436, last: 0:00:13.743311941, current: 0:00:07.723311941, fixed = last: 0:00:13.743311941
0:00:27.154397304  4831 0x14d704002720 WARN                kmsutils kmsutils.c:1479:kms_utils_depayloader_adjust_pts_out:<rtppcmudepay0> Fix PTS not strictly increasing, last: 0:00:13.755311941, current: 0:00:13.743311941, fixed = last + 1: 0:00:13.756311941
0:00:27.174221849  4831 0x14d704002720 LOG          rtpsynchronizer kmsrtpsynchronizer.c:425:kms_rtp_synchronizer_process_rtp_buffer_mapped:<KmsRtpSynchronizer@0x14d72c020d80> RTP SSRC: 3037524492, Seq: 527
0:00:27.174277267  4831 0x14d704002720 WARN         rtpsynchronizer kmsrtpsynchronizer.c:549:kms_rtp_synchronizer_process_rtp_buffer_mapped:<KmsRtpSynchronizer@0x14d72c020d80> Function Name :kms_rtp_synchronizer_process_rtp_buffer_mapped  File Name : /root/PIYUSH/kms-omni-build/kms-core/src/gst-plugins/commons/kmsrtpsynchronizer.c  Line Number: 549
0:00:27.174299732  4831 0x14d704002720 WARN         rtpsynchronizer kmsrtpsynchronizer.c:561:kms_rtp_synchronizer_process_rtp_buffer_mapped:<KmsRtpSynchronizer@0x14d72c020d80> [Sorted mode] Fix PTS not increasing monotonically, SSRC: 3037524492, seq: 527, rtp_ts: 127015596, ext_ts: 127015596, last: 0:00:13.743311941, current: 0:00:07.743311941, fixed = last: 0:00:13.743311941
0:00:27.174394041  4831 0x14d704002720 WARN                kmsutils kmsutils.c:1479:kms_utils_depayloader_adjust_pts_out:<rtppcmudepay0> Fix PTS not strictly increasing, last: 0:00:13.756311941, current: 0:00:13.743311941, fixed = last + 1: 0:00:13.757311941
0:00:27.174525946  4831 0x14d704002720 WARN           audioresample gstaudioresample.c:1009:gst_audio_resample_check_discont:<audioresample1> encountered timestamp discontinuity of 304 samples = 0:00:00.038000000
0:00:27.194253540  4831 0x14d704002720 LOG          rtpsynchronizer kmsrtpsynchronizer.c:425:kms_rtp_synchronizer_process_rtp_buffer_mapped:<KmsRtpSynchronizer@0x14d72c020d80> RTP SSRC: 3037524492, Seq: 528
0:00:27.194325675  4831 0x14d704002720 WARN         rtpsynchronizer kmsrtpsynchronizer.c:549:kms_rtp_synchronizer_process_rtp_buffer_mapped:<KmsRtpSynchronizer@0x14d72c020d80> Function Name :kms_rtp_synchronizer_process_rtp_buffer_mapped  File Name : /root/PIYUSH/kms-omni-build/kms-core/src/gst-plugins/commons/kmsrtpsynchronizer.c  Line Number: 549
0:00:27.194360607  4831 0x14d704002720 WARN         rtpsynchronizer kmsrtpsynchronizer.c:561:kms_rtp_synchronizer_process_rtp_buffer_mapped:<KmsRtpSynchronizer@0x14d72c020d80> [Sorted mode] Fix PTS not increasing monotonically, SSRC: 3037524492, seq: 528, rtp_ts: 127015756, ext_ts: 127015756, last: 0:00:13.743311941, current: 0:00:07.763311941, fixed = last: 0:00:13.743311941

I am not getting any Error messages and just a print of timestamp discontinuity in the interval of RTP packets

Something like this 

mple0> encountered timestamp discontinuity of 960 samples = 0:00:00.060000000

0:00:26.934555639  4831 0x14d704002720 WARN           audioresample gstaudioresample.c:1009:gst_audio_resample_check_discont:<audioresample1> encountered timestamp discontinuity of 304 samples = 0:00:00.038000000

0:00:26.974570708  4831 0x14d704002720 WARN           audioresample gstaudioresample.c:1009:gst_audio_resample_check_discont:<audioresample1> encountered timestamp discontinuity of 304 samples = 0:00:00.038000000

0:00:27.014521594  4831 0x14d704002720 WARN           audioresample gstaudioresample.c:1009:gst_audio_resample_check_discont:<audioresample1> encountered timestamp discontinuity of 304 samples = 0:00:00.038000000

0:00:27.054561629  4831 0x14d704002720 WARN           audioresample gstaudioresample.c:1009:gst_audio_resample_check_discont:<audioresample1> encountered timestamp discontinuity of 304 samples = 0:00:00.038000000

0:00:27.094575209  4831 0x14d704002720 WARN           audioresample gstaudioresample.c:1009:gst_audio_resample_check_discont:<audioresample1> encountered timestamp discontinuity of 304 samples = 0:00:00.038000000

0:00:27.122462722  4831 0x14d7100018f0 WARN           audioresample gstaudioresample.c:1009:gst_audio_resample_check_discont:<audioresample0> encountered timestamp discontinuity of 640 samples = 0:00:00.040000000

0:00:27.134523890  4831 0x14d704002720 WARN           audioresample gstaudioresample.c:1009:gst_audio_resample_check_discont:<audioresample1> encountered timestamp discontinuity of 304 samples = 0:00:00.038000000

0:00:27.174525946  4831 0x14d704002720 WARN           audioresample gstaudioresample.c:1009:gst_audio_resample_check_discont:<audioresample1> encountered timestamp discontinuity of 304 samples = 0:00:00.038000000

0:00:27.214529149  4831 0x14d704002720 WARN           audioresample gstaudioresample.c:1009:gst_audio_resample_check_discont:<audioresample1> encountered timestamp discontinuity of 304 samples = 0:00:00.038000000

0:00:27.254541540  4831 0x14d704002720 WARN           audioresample gstaudioresample.c:1009:gst_audio_resample_check_discont:<audioresample1> encountered timestamp discontinuity of 304 samples = 0:00:00.038000000

I just want to ask whether is this a bug at gstreamer?

I have reason to believe so because the source is sending the correct packets as per RFC verified by me Twice.

https://bugzilla.gnome.org/show_bug.cgi?id=726579

The above link talks about the same issue regarding the timestamp discontinuity but in the case of Opus Codec. Can this be the same for AMR-WB codec?

Just want your feedback.

This is completely random and difficult to produce, maybe that is the reason people have avoided it?

Regards

[PIYUSH BADKUL]

Hey,
It would be very helpful if you could just spare a minute or two and tell me whether there can be a possiblty of bug in gstreamer or not depending on the information provided above. I do not want you to disturb your schedule and make time for this bug.

I will try to fix it. All I want to know whether I am thinking in correct direction or not.

The above logs provided occur only in case of one hardphone and for the rest of the phones, transcoding works.

[Juan Navarro]

About the "encountered timestamp discontinuity of 304 samples" message, I doubt it is a GStreamer bug. It would be a good idea to inspect traffic with Wireshark to understand where this timestamp issues are coming from, or at least have a clear idea if the issue comes from the sending device (which might be jumping in its sequence numbers or in its timestamps), the network (which might be losing packets) or Kurento itself (which might be unsuccessfully trying to recover from some packet loss)

[Juan Navarro]

I know that the WebRtcEndpoint does indeed filter out the available video codecs, to work only with the handful ones that are standard (VP8, H264, OPUS, and I think PCMU). But I think RtpEndpoint didn't do such filtering. However if this addition in kmsagnosticcaps.h is not making it work for you, right now I'm not really sure of where could the problem be.

You could extend your logging levels to include those from Transcoding:
https://doc-kurento.readthedocs.io/en/latest/features/logging.html#suggested-levels

maybe they show some useful information that explains why the caps are not being accepted.

There is also a GStreamer category that can be enabled in the logs, GST_CAPS:4 (or 5), but that's pretty crazy because it prints caps details for ALL of GStreamer elements inside KMS, so you better use it with the simplest of pipelines to reduce the number of elements to the minimum.

Please let me know if you find anything! I'd be happy to accept a PR change that adds support for AMR-WB.

To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kurento/5e2d22d3-c428-400f-85cc-4e0a11b2e28eo%40googlegroups.com.

[jesper rask]

Did you find a solution to this problem? Im in the same situation...