video room > rtp forward > ffmpeg > rtsp server, but with h264, not vp8

[Dan Ionescu]

Hello dears, 

So this is what I am looking to accomplish:

• video room > rtp forward > ffmpeg > rtsp server, but with h264, not vp8

I managed to do this using vp8 codec, however, when configuring the video room plugin with the h264 video codec, something goes amiss. The reason I need to switch from the vp8 codec to the h264 one, I think it is obvious, is the huge cpu consumption needed to transcode with ffmpeg each vp8 stream to h264. If I would be able to get h264 from the rtp forward, I would copy further the video stream and only transcode the audio one.

As I am still at the beginning with understanding how everything should fall in place, please let me know your thoughts,  any suggestion is welcome.

I think the culprit might be in the SDP:

v=0

o=- 0 0 IN IP4 127.0.0.1

s=RTP Video

c=IN IP4 127.0.0.1

t=0 0

a=tool:libavformat 56.15.102

m=audio 11004 RTP/AVP 111

a=rtpmap:111 OPUS/48000/2

m=video 11006 RTP/AVP 96

a=rtpmap:96 H264/90000

[Lorenzo Miniero]

Probably something broken in your SDP. I don't see a fmtp attribute for video, for instance. Check older posts for examples.

L.

[vishalr...@gmail.com]

We use exact same scenario you have mentioned i.e. video room > rtp forward > ffmpeg > rtsp server (NGINX)

Without any issue it worked perfectly so far.

Where are you failing?

I would troubleshoot in below manner

1. Check if RTP is coming through after RTP forward. Use tcpdump or similar tool to check

2. Check if correct codec ID is coming through i.e. 111 for OPUS and 96 for H264 in RTP packets.

3. See errors in FFMPEG. Usually there are errors if it does not understand media coming from Janus. Here is FFMPEG comand i used to push RTMP to NGINX. I converted OPUS to AAC

4. Also make sure if key frame is sent frequently i.e. fir_freq in videoroom config, otherwise FFMPEG wont detect H264

ffmpeg -analyzeduration 300M -probesize 300M -protocol_whitelist file,udp,rtp -i testsdp.sdp -c:v copy -c:a aac -ar 16k -ac 1 -preset ultrafast -tune zerolatency -f flv rtmp://a.b.c.d/myapp/test

[Dan Ionescu]

Thank you for the reply. I went through all I could find in group and on github. I would really appreciate a working example for h264.
Thank you.

🖖 Live long and prosper,
👤 Dan Ionescu
📱sent from iPhone

> On 4 Sep 2020, at 13:08, vishalr...@gmail.com <vishalr...@gmail.com> wrote:
>

[vishalr...@gmail.com]

Here is SDP i am using which is working for me

v=0
o=- 0 0 IN IP4 127.0.0.1

s=No Name

c=IN IP4 127.0.0.1
t=0 0

a=tool:libavformat 55.2.100
m=audio 5502 TCP 111
a=rtpmap:111 opus/48000
m=video 5504 RTP/AVP 96
a=rtpmap:96 H264/90000

[Dan Ionescu]

a=tool:libavformat 55.2.100

This is what I think I was doing wrong. I will give it a spin right away. Thank you so much for your insights.

🖖 Live long and prosper,
👤 Dan Ionescu
📱sent from iPhone

> On 4 Sep 2020, at 13:13, vishalr...@gmail.com <vishalr...@gmail.com> wrote:
>
> a=tool:libavformat 55.2.100

[dvision]

This is my room config in the jcfg file:

room-1234: {

        description = "Demo Room"

        #pin = 1234

        secret = "adminpwd"

        publishers = 6

        bitrate = 1000000000

        fir_freq = 1

        audiocodec = "opus"

        videocodec = "h264"

        h264_profile = "42e01f"

        record = false

        #rec_dir = "/path/to/recordings-folder"

}

The odd thing is that if I switch to vp8 in the jcfg config and in the sdp file it works fine. Is there any more places where I need to do changes? Thank you.

[Mirko Brankovic]

and your rtp_forward?

--
You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/8fedb307-be7b-41e8-9283-e29df3719fb2n%40googlegroups.com.

--

Regards,

Mirko

[dvision]

My RTP forward request is:

        var rtpmsg = { "request": "rtp_forward", publisher_id: myid, room: 1234, "audio_port": audioport, "audiopt": 111, "video_port": videoport, "videopt": 96, "host": ndihost, "secret": "adminpwd" };

        sfutest.send({"message": rtpmsg, success: function(result) {

                if(result) {

                        audio_stream_id = result["rtp_stream"]["audio_stream_id"];

                        video_stream_id = result["rtp_stream"]["video_stream_id"];

                }

        }});

Thank you.

[dvision]

The SDP I am using as suggested by Vishal earlier is:

v=0

o=- 0 0 IN IP4 172.16.1.245

s=RTP Video

c=IN IP4 172.16.1.245

t=0 0

a=tool:libavformat 55.2.101

m=audio 11004 RTP/AVP 111

a=rtpmap:111 OPUS/48000/2

m=video 11006 RTP/AVP 96

a=rtpmap:96 H264/90000

[dvision]

Hi Vishal, 

is the fir_freq = 1 the right value?

Thank you.

[dvision]

I am going to try with a vanilla version of the videoroom demo and get back to you.

Anyway I want to thank you for your suggestions so far.

Regards.

[dvision]

Hey Vishal,

I can see the traffic incoming in the network monitor as I am building this as a PoC for the moment. 

The ffmpeg complains:

[sdp @ 0x3d93d80] Could not find codec parameters for stream 1 (Video: h264, none): unspecified size

Consider increasing the value for the 'analyzeduration' and 'probesize' options

Input #0, sdp, from 'janus1.sdp':

  Metadata:

    title           : RTP Video

  Duration: N/A, bitrate: N/A

    Stream #0:0: Audio: opus, 48000 Hz, stereo, fltp

    Stream #0:1: Video: h264, none, 90k tbr, 90k tbn, 180k tbc

Stream mapping:

  Stream #0:0 -> #0:0 (opus (native) -> aac (native))

Press [q] to stop, [?] for help

Finishing stream 0:0 without any data written to it.

Output #0, rtsp, to 'rtsp://localhost/test':

  Metadata:

    title           : RTP Video

    encoder         : Lavf58.20.100

    Stream #0:0: Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s

    Metadata:

      encoder         : Lavc58.35.100 aac

size=N/A time=00:00:00.00 bitrate=N/A speed=   0x

video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

On Friday, 4 September 2020 at 13:07:56 UTC+3 vishalr...@gmail.com wrote:

[vishalr...@gmail.com]

Looks like SDP and RTP not matching.

a=tool:libavformat 55.2.101

See if version is correct here. If you see video packets in network, it has something to do with FFMPEG and not JANUS probably.

[dvision]

I tried also with 

a=tool:libavformat 55.2.100

[dvision]

I just finished the trial with the vanilla video room, but to the same result, so let me wrap up all the information:

Janus and demo room version: 0.10.4 - latest release on github

ffmpeg version: 4.1.6

Browser: Google Chrome 85.0.4183.83 official build 64-bit windows 10 / macosx catalina

janus video room jcfg:

room-1234: {

        description = "Demo Room"

        #pin = 1234

        secret = "adminpwd"

        publishers = 6

        bitrate = 1000000000

        fir_freq = 1

        audiocodec = "opus"

        videocodec = "h264"

        h264_profile = "42e01f"

        record = false

        #rec_dir = "/path/to/recordings-folder"

}

janus.sdp file:

v=0

o=- 0 0 IN IP4 127.0.0.1

s=RTP Video

c=IN IP4 127.0.0.1

t=0 0

a=tool:libavformat 55.2.100

m=audio 11004 RTP/AVP 111

a=rtpmap:111 OPUS/48000/2

m=video 11006 RTP/AVP 96

a=rtpmap:96 H264/90000

ffmpeg command and output:

ubuntu@node1:~$ ffmpeg -hide_banner -protocol_whitelist file,udp,rtp -probesize 300M -analyzeduration 300M -i janus1.sdp video.mp4

[sdp @ 0x3709c40] Could not find codec parameters for stream 1 (Video: h264, none): unspecified size

Consider increasing the value for the 'analyzeduration' and 'probesize' options

Input #0, sdp, from 'janus1.sdp':

  Metadata:

    title           : RTP Video

  Duration: N/A, start: 0.000000, bitrate: N/A

    Stream #0:0: Audio: opus, 48000 Hz, stereo, fltp

    Stream #0:1: Video: h264, none, 90k tbr, 90k tbn, 180k tbc

File 'video.mp4' already exists. Overwrite ? [y/N] y

Stream mapping:

  Stream #0:0 -> #0:0 (opus (native) -> aac (native))

Press [q] to stop, [?] for help

Output #0, mp4, to 'video.mp4':

  Metadata:

    title           : RTP Video

    encoder         : Lavf58.20.100

    Stream #0:0: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s

    Metadata:

      encoder         : Lavc58.35.100 aac

size=    1280kB time=00:02:57.98 bitrate=  58.9kbits/s speed=12.7x

As I said earlier, I see incoming traffic on the host that I am using for ffmpeg, in network monitor, when I forward the rtp stream. Although, I don't know how to test the validity of the stream(s), but only with ffmpeg or ffplay in conjuction with the sdp file. 

Thank you again in advance for anything that could help me break through.

[dvision]

I can also confirm that the stream encoded by the browser app is h264

[Mirko Brankovic]

I don't see video profile in sdp

--
You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/149d8c5a-aa1c-4b68-b3be-1ab69e4db4d4n%40googlegroups.com.

[dvision]

Hy Mirko, 

Thank you for your reply. 

I am also trying with the following sdp, but to no success:

v=0

o=- 0 0 IN IP4 127.0.0.1

s=RTP Video

c=IN IP4 127.0.0.1

t=0 0

a=tool:libavformat 55.2.100

m=audio 11004 RTP/AVP 111

a=rtpmap:111 OPUS/48000/2

m=video 11006 RTP/AVP 96

a=rtpmap:96 H264/90000

a=fmtp:96 packetization-mode=1;profile-level-id=42e01f

[dvision]

Hi Mirko, 

Did you get to see my previous response. Do you think the SDP is correct?

Thank you.

[Mirko Brankovic]

hey yes indeed, i read it but I don't have a good suggestion.

I would try gstreamer and also capturing output of streams into file and examine in wireshark maybe

To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/9ffcf5e6-8839-45b4-912e-bc4574612752n%40googlegroups.com.

--

Regards,

Mirko

[Avital Yachin]

Below is a working sample of Videroom --> RTMP using ffmpeg and H.264.

Points of interests:

1. fir_freq = 2 (can try also with 1, but not 10)
2. video_pt = 102 
3. audio_pt = 111
4. packetization-mode=1 
5. Must trasncode opus to aac in the ffmpeg command to generate RTMP output 

 

Room Config:

room-1234: {

        description = "Demo Room"

        secret = "password"

        publishers = 6

        bitrate = 128000

        fir_freq = 2

        audiocodec = "opus"

        videocodec = "h264"

        record = false

}

Forwarder:

{"body":{"request":"rtp_forward","publisher_id":8640298420261539,"room":1234,"host":"127.0.0.1","audio_port":50555,"video_port":51555,"video_pt":102,"audio_pt":111,"secret":"verysecret"},"janus":"message","transaction":"123"}

SDP (h264.sdp):

v=0

o=- 0 0 IN IP4 127.0.0.1

s=RTP Video

c=IN IP4 127.0.0.1

t=0 0

a=tool:libavformat 58.2.102

m=audio 50555 RTP/AVP 111

a=rtpmap:111 OPUS/48000/2

m=video 51555 RTP/AVP 102

a=rtpmap:102 H264/90000

a=fmtp:102 packetization-mode=1

FFMPEG:

ffmpeg -protocol_whitelist rtmp,udp,rtp,file -f sdp -buffer_size 2M -rtbufsize 50M -i h264.sdp -y -map v:0 -map 0:a? -flags +global_header -c:v copy -c:a aac -f flv rtmp://127.0.0.1:1935/push/my-stream-key

[Lorenzo Miniero]

fir_freq=2 is an awful idea. When you create forwarders the VideoRoom automatically asks for a keyframe anyway, and there are other (documented) ways to programmatically ask for more.

Lorenzo

[Avital Yachin]

Thanks Lorenzo, which value would you recommend for the fir_freq?

[Alessandro Toppi]

10 is a good value. In case you need more frequent KFs lower it to 5.