H264 and AAC on RtpEndpoint

[Joana Lourinho Gomes]

Hello!

I have a stream being fed to KMS via WebRtc and sent to a RtpEndpoint. I've been working on my RTP client to process the H264 video stream, and that's working, apart some quality issues. I now added the AAC audio stream and, although I get the bitrates on the client side and VLC claims to be decoding the audio, there is no sound. I've tried various permutations on the SDP, and nothing seems to work.

This is the SDP I have right now:

v=0

o=- 2 2 IN IP4 192.168.174.102

s=

t=0 0

c=IN IP4 192.168.174.102

m=video 24230 RTP/AVP 100

a=rtpmap:100 H264/90000
a=recvonly\n

m=audio 25230 RTP/AVP 105

a=recvonly

a=rtpmap:105 MPEG4-GENERIC/32000/2

a=fmtp:105 streamtype=5;profile-level-id=7;mode=AAC-hbr;SizeLength=13;IndexLength=3;IndexDeltaLength=3;config=1290

Any ideas on what I might be missing? Thanks!

Joana Gomes

[Ivan Gracia]

Hi Johanna,

I'm not 100% sure we are supporting that audio. Did it work with any other kind of audio?

Ivan Gracia

--
You received this message because you are subscribed to the Google Groups "kurento" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kurento+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Max Dörfler]

Hi Ivan, hi Joana,

are there any updates on using aac with Kurento because I'm currently also trying to do same thing. For that matter I only got it to work with opus/pcmu. I've seen AMR in the sdp_pattern.txt but couldn't make it work.

The Documentation says

Automatic media transcodification between any of the codecs supported by GStreamer including VP8, H.264, H.263, AMR, OPUS, Speex, G.711, etc.

which should include aac, right?

Thanks! Regards,
Max

[Ivan Gracia]

Hi Max,

Please find answers inline.

Cheers,

Ivan Gracia

On Mon, Oct 19, 2015 at 3:00 PM, Max Dörfler <max.do...@googlemail.com> wrote:

Hi Ivan, hi Joana,

are there any updates on using aac with Kurento because I'm currently also trying to do same thing. For that matter I only got it to work with opus/pcmu. I've seen AMR in the sdp_pattern.txt but couldn't make it work.

The Documentation says

Automatic media transcodification between any of the codecs supported by GStreamer including VP8, H.264, H.263, AMR, OPUS, Speex, G.711, etc.

which should include aac, right?

That means that there will be automatic media transcodification between media elements inside KMS, not that you can use any of those in any transmission protocol. For now, we only support VP8 (thoroughly tested) and H264 (not tested very much) It probably wouldn't be that hard, but it implies spending a lot of time doing tests, adapting to the different SDP parameters added by some of those codecs... and the use case seems very reduced for now.

[Xuanah]

Hello!

Sorry for the (very) late reply! We did find a solution, which included editing and recompiling some files on KMV. What we did was adding the formats we needed to the files where the available ones are listed, both .json and .cpp/.h files. I don't have the specific list of changes we made with me, but I'll do my best to give you a step-by-step guide on how we did it. 

I'll return to this as soon as I can. I'm very sorry for not being more helpful at the moment, but I really lack the time!

Best regards,

Joana Lourinho Gomes

--
You received this message because you are subscribed to a topic in the Google Groups "kurento" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kurento/PzX_-0EqNak/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kurento+u...@googlegroups.com.

[Ivan Gracia]

Hey Joana,

That's great news! It would be great if you could submit a pull request, so we can put that in the next KMS release :-)

Cheers,

Ivan Gracia

[Xuanah]

Ahah good morning Ivan!

Will do! I'll send you my bill afterwards... :P

Joking! Have a great day!

Jo

[Ivan Gracia]

😂 Send it and I'll clip it with the rest... you know how bad the banks are here in Spain right now? :-P

Now, seriously, that's a great contribution you have there. It would be a pity to miss it!

Have a great day too!

Ivan Gracia

[Max Dörfler]

Hi Joana,

thanks for your reply, thats really great to hear. I'm looking forward to your solution/guide/pull request. :)

Regards
Max

[nicolo....@gruntpage.com]

Hi Joana,

I'm searching in a way to do the same thing (RTPEndopoint with H264 anc aac codecs).

Did you already uploaded the list of changes?

Thank you very much for your help

Bye

[nicolo]

So brielfy:

the input stream is VP8/opus with WebrtcEndpoint and I want to have H264/aac in a RtpEndpoint.

video transcoding from VP8 to H264 is OK

For the audio:

kmsdectreebin.c is correctly founding the decoder opusdec and there are no errors.

kmsenctreebin.c does not implement aac encoder, so I have implemented it (attached file), but there is still no sound in output.

I have added the following lines (copyng from what is alredy done with other codecs):

case AAC:
      return "voaac";
case MP3:
      return "lamemp3";

case AAC:
    {
      GST_WARNING_OBJECT (encoder, "mod aac config");
      g_object_set (G_OBJECT (encoder), "bitrate", 42100, NULL);
      break;
    }
case MP3:
    {
      GST_WARNING_OBJECT (encoder, "mod mp3 config");
      g_object_set (G_OBJECT (encoder), "bitrate", 40, NULL);
      break;
    }

else if (g_str_has_prefix (name, "voaacenc")) {
    self->priv->enc_type = AAC;
    GST_WARNING_OBJECT (self, "mod voaacenc");

 } else if (g_str_has_prefix (name, "lamemp3enc")) {
    self->priv->enc_type = MP3;
    GST_WARNING_OBJECT (self, "mod lamemp3enc");
  

It seems to be working (the encoder voaacenc is found and started), but there are no sound at the output.

My SDP is the follwoing:

v=0

o=- 2 2 IN IP4 172.16.101.100

s=

t=0 0

c=IN IP4 172.16.101.100
m=video 24230 RTP/AVP 100

a=rtpmap:100 H264/90000
a=sendonly

m=audio 25230 RTP/AVP 105

a=sendonly

a=rtpmap:105 MPEG4-GENERIC/32000/2

a=fmtp:105 streamtype=5;profile-level-id=7;mode=AAC-hbr;SizeLength=13;IndexLength=3;IndexDeltaLength=3;config=1290

Also with MP3 encoder is not working

v=0

o=- 2 2 IN IP4 172.16.101.100

s=

t=0 0

c=IN IP4 172.16.101.100
m=video 24230 RTP/AVP 100

a=rtpmap:100 H264/90000
a=sendonly

m=audio 25230 RTP/AVP 105

a=sendonly

a=rtpmap:105 MP4-LATM/90000

I have also modified  /etc/kurento/modules/kurento/SdpEndpoint.conf.json adding AAC and MP3 audio codecs.

How can I debug this? I really don't know where the problem can be, so if the problem is the decoder or the encoder.

[Nicolò Paganin]

I found that SDP answer of KMS it's incomplete:

This is the SDP request from wowza:

v=0
o=- 0 3641150070 IN IP4 127.0.0.1
s=Wowza Media Server
c=IN IP4 127.0.0.1
t=0 0
m=audio 25230 RTP/AVP 105
a=rtpmap:105 MPEG4-GENERIC/11025/1
a=ftmp:105 profile-level-id=1;mode=AAC-hbr;sizelength=13;indexlength=3;indexdeltalength=3;config=1490
a=sendonly
a=ssrc:3610119879 cname:user3527564918@host-d993465d
m=video 45752 RTP/AVP 101
a=rtpmap:101 H264/90000
a=rtcp-fb:101 nack
a=rtcp-fb:101 nack pli
a=rtcp-fb:101 ccm fi
a=sendonly
a=ssrc:276641451 cname:user3527564918@host-d993465d

and the following is the SDP answer of KMS:

v=0
o=- 3663385910 3663385910 IN IP4 172.17.0.1
s=Kurento Media Server
c=IN IP4 172.17.0.1
t=0 0
m=audio 51912 RTP/AVP 105
a=rtpmap:105 MPEG4-GENERIC/11025/1
a=recvonly
a=ssrc:903263233 cname:user2184203206@host-d9dd811b
a=mid:audio0
m=video 20610 RTP/AVP 101
a=rtpmap:101 H264/90000
a=recvonly
a=ssrc:1465579540 cname:user2184203206@host-d9dd811b
a=mid:video0

as you can see, the answer is incomplete (ftmp paramenter is missing). Maybe ftmp parameter is not implemented in the sdp manager of KMS?

Also, if I change the Wowza SDP request with recvonly and not sendonly the KMS SDP answer is this one:

v=0
o=- 3663384784 3663384784 IN IP4 172.17.0.1
s=Kurento Media Server
c=IN IP4 172.17.0.1
t=0 0
m=audio 26582 RTP/AVP
a=sendonly
a=ssrc:2531363156 cname:user4210729167@host-aad2bb17
a=mid:audio0
m=video 9416 RTP/AVP 101
a=rtpmap:101 H264/90000
a=sendonly

so, also the payload ID is missing

Do you have any advice?

I think this feature could be helpful also for other users...