Better webrtc H264 Profile support for decoding/playback

[Anand Setlur]

I am trying to play  using the janus streaming plugin some H264 content from youtube(https://www.youtube.com/watch?v=9cQT4urTlXM) "without transcoding" .

I am not successful yet. I am using the nice tutorial by https://flashphoner.com/how-to-grab-a-video-from-youtube-and-share-it-via-webrtc-in-real-time/ 

youtube-dl --list-formats https://www.youtube.com/watch?v=9cQT4urTlXM

...

22           mp4        1280x720   hd720 , avc1.64001F, mp4a.40.2@192k (best)


In https://bugs.chromium.org/p/webrtc/issues/detail?id=8317 , a statement is made that

"the decoder can actually handle decoding High, Main, and Baseline, we can add support for all of these profiles."

I was not able to get the decoder to play this  content that uses Main 3.1 profile

    creation_time   : 2016-08-23T12:21:06.000000Z

  Duration: 00:29:59.99, start: 0.000000, bitrate: N/A

    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 288 kb/s, 30 fps, 30 tbr, 90k tbn, 60 tbc (default)

fmtp:96 packetization-mode=1; sprop-parameter-sets=Z01AH+iAKALdgIgAAAMACAAAAwHgeMGIkA==,aOvvIA==; profile-level-id=4D401F

I am however able to play different content from a Chinese RTSP camera that claims to be using High Profile 5.1 without any transcoding .

In both cases, I only tell the browser that it is packetization-mode=1 and deliberately omit telling the browser the profile-level-id, else it chokes when setting RemoteSDPDescription since it still only publicly claims to support constrained baseline profile.

So my question is, how does one know what H264 profiles can the OpenH264 decoder officially handle without the need to transcode using ffmpeg to some lesser profile? I'd rather not transcode when I can avoid it but need to know definitively which profiles the browser can really handle instead of finding out after the fact.

I understand the encoder still continues to encode using lesser profiles and will be less capable than the decoder

I'd greatly appreciate someone shedding light on what decoder profiles are supported or in the works.

[Anand Setlur]

The chinese RTSP camera advertised the following H264 profile which my janus streaming plugin(https://janus.conf.meetecho.com/streamingtest.html)  successfully played without any transcoding

fmtp:96 packetization-mode=1; sprop-parameter-sets=Z2RAKawsqAeAIn5U,aO44gA==; profile-level-id=644029

[Anand Setlur]

When I try to get additional details as to why the h264 stream is not being rendered in the browser, I get no additional logs. Am I invoking the logging command right? How can I get the most detailed H264 decoder logs to drill into why the stream is not being rendered from the browser if I launch it on the command line ? Do I need to recompile chrome for it just to see detailed webrtc-logs? I hope not.

/usr/bin/google-chrome-unstable --enable-logging --vmodule/webrtc/*=2,*/libjingle/*=2,*=-2

[Stepan Salenikovich]

On Friday, December 8, 2017 at 10:52:57 AM UTC-5, Anand Setlur wrote:

When I try to get additional details as to why the h264 stream is not being rendered in the browser, I get no additional logs. Am I invoking the logging command right? How can I get the most detailed H264 decoder logs to drill into why the stream is not being rendered from the browser if I launch it on the command line ? Do I need to recompile chrome for it just to see detailed webrtc-logs? I hope not.

/usr/bin/google-chrome-unstable --enable-logging --vmodule/webrtc/*=2,*/libjingle/*=2,*=-2

You need to add  --enable-logging=stderr if you want to see the logs in the console, otherwise I think they go into a file. See: https://www.chromium.org/for-testers/enable-logging

[Anand Setlur]

Thanks Stepan. With --enable-logging=stderr I see a lot more webrtc output but still no smoking gun error and I see "No decodable frame in 200 ms" with no obvious errors. I probably need a higher logging level to see the H264 parsing/decoding output

It is curious that the browser assumed an initial assignment of dynamic payload numbers and the modified them. I am using chrome63

Changing recv codecs from {VideoCodec[96:VP8], VideoCodec[98:VP9], VideoCodec[100:H264], VideoCodec[102:H264]} to {VideoCodec[96:H264]}

[1:20:1208/133146.317921:INFO:webrtcvideoengine.cc(544)] CreateChannel. Options: VideoOptions {}

[1:20:1208/133146.318006:INFO:channel.cc(178)] Created channel for video

[1:20:1208/133146.318038:INFO:channel.cc(334)] Setting RTP Transport for video on video transport 0xc6709ed3200

[1:20:1208/133146.318077:INFO:call.cc(1097)] Transport video is disconnected

[1:20:1208/133146.318106:INFO:transportcontroller.cc(648)] Set remote transport description on video

[1:20:1208/133146.318132:INFO:p2ptransportchannel.cc(362)] Received remote ICE parameters: ufrag=280w, renomination disabled

[1:20:1208/133146.318161:VERBOSE1:p2ptransportchannel.cc(1352)] Sorting 0 available connections:

[1:19:1208/133146.318167:INFO:peerconnection.cc(2300)] Session: 8449001940639177385 Old state: kStable New state: kHaveRemoteOffer

[1:20:1208/133146.318218:INFO:channel.cc(1930)] Setting remote video description

[1:20:1208/133146.318260:INFO:webrtcvideoengine.cc(726)] SetSendParameters: {codecs: [VideoCodec[96:H264]], extensions: [], max_bandwidth_bps: -1, }

[1:20:1208/133146.318325:INFO:webrtcvideoengine.cc(735)] Using codec: VideoCodec[96:H264]

[1:20:1208/133146.318353:INFO:call.cc(1000)] WebRTC.Call.UpdateCurrentBitrateConfig: calling SetBweBitrates with args (0, -1, -1)

[1:20:1208/133146.318378:INFO:webrtcvideoengine.cc(783)] SetFeedbackOptions on all the receive streams because the send codec or RTCP mode has changed.

[1:20:1208/133146.318405:INFO:webrtcvideoengine.cc(1179)] AddRecvStream: {id:janusv0;ssrcs:[3491159228];ssrc_groups:;cname:janusvideo;sync_label:janus}

[1:20:1208/133146.318450:INFO:h264.cc(95)] Creating H264DecoderImpl.

[1:20:1208/133146.318483:INFO:h264.cc(95)] Creating H264DecoderImpl.

[24926:24926:1208/133146.318905:INFO:CONSOLE(1331)] "[object RTCPeerConnection]", source: https://x.com/j/janus.js (1331)

[24926:24926:1208/133146.318944:INFO:CONSOLE(1336)] "Preparing local SDP and gathering candidates (trickle=true)", source: https://x.com/j/janus.js (1336)

[24926:24926:1208/133146.318961:INFO:CONSOLE(2494)] "isDataEnabled:", source: https://x.com/j/janus.js (2494)

[1:20:1208/133146.324267:INFO:webrtcvideoengine.cc(962)] SetRecvParameters: {codecs: [VideoCodec[96:H264]], extensions: []}

[1:20:1208/133146.324374:INFO:webrtcvideoengine.cc(977)] Changing recv codecs from {VideoCodec[96:VP8], VideoCodec[98:VP9], VideoCodec[100:H264], VideoCodec[102:H264]} to {VideoCodec[96:H264]}

[1:20:1208/133146.324433:INFO:webrtcvideoengine.cc(2287)] RecreateWebRtcVideoStream (recv) because of SetRecvParameters

[1:20:1208/133146.324490:INFO:call.cc(1156)] UpdateAggregateNetworkState: aggregate_state=down

[1:20:1208/133146.324540:INFO:send_side_congestion_controller.cc(247)] SignalNetworkState Down

[1:20:1208/133146.324625:INFO:video_receive_stream.cc(149)] ~VideoReceiveStream: {decoders: [{decoder: (VideoDecoder), payload_type: 96, payload_name: VP8, codec_params: {}}, {decoder: (VideoDecoder), payload_type: 98, payload_name: VP9, codec_params: {}}, {decoder: (VideoDecoder), payload_type: 100, payload_name: H264, codec_params: {level-asymmetry-allowed: 1packetization-mode: 1profile-level-id: 42001f}}, {decoder: (VideoDecoder), payload_type: 102, payload_name: H264, codec_params: {level-asymmetry-allowed: 1packetization-mode: 1profile-level-id: 42e01f}}], rtp: {remote_ssrc: 3491159228, local_ssrc: 1, rtcp_mode: RtcpMode::kCompound, rtcp_xr: {receiver_reference_time_report: off}, remb: on, transport_cc: off, nack: {rtp_history_ms: 1000}, ulpfec_payload_type: 106, red_type: 104, rtx_ssrc: 0, rtx_payload_types: {97 (pt) -> 96 (apt), 99 (pt) -> 98 (apt), 101 (pt) -> 100 (apt), 103 (pt) -> 102 (apt), 105 (pt) -> 104 (apt), }, extensions: []}, renderer: (renderer), render_delay_ms: 10, sync_group: janus, pre_decode_callback: nullptr, target_delay_ms: 0}

[1:20:1208/133146.325217:INFO:video_receive_stream.cc(109)] VideoReceiveStream: {decoders: [{decoder: (VideoDecoder), payload_type: 96, payload_name: H264, codec_params: {level-asymmetry-allowed: 1packetization-mode: 1profile-level-id: 42e01f}}], rtp: {remote_ssrc: 3491159228, local_ssrc: 1, rtcp_mode: RtcpMode::kCompound, rtcp_xr: {receiver_reference_time_report: off}, remb: on, transport_cc: off, nack: {rtp_history_ms: 1000}, ulpfec_payload_type: -1, red_type: -1, rtx_ssrc: 0, rtx_payload_types: {-1 (pt) -> 96 (apt), }, extensions: []}, renderer: (renderer), render_delay_ms: 10, sync_group: janus, pre_decode_callback: nullptr, target_delay_ms: 0}

[1:30:1208/133202.340316:WARNING:video_receive_stream.cc(454)] No decodable frame in 200 ms, requesting keyframe.

[1:30:1208/133202.540629:WARNING:video_receive_stream.cc(454)] No decodable frame in 200 ms, requesting keyframe.

[Anand Setlur]

With chrome63 on desktop chrome, I notice there is support for 3 profile levels. One of them is profile-level-id=64001f . Still unable to play the youtube video encoded using profile-level-id=4D401F . Any expectations that it may work with any updates from OpenH264 in future versions of chrome ?

a=rtpmap:100 H264/90000
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=fmtp:100 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=64001f
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:102 H264/90000
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack
a=rtcp-fb:102 nack pli
a=rtcp-fb:102 goog-remb
a=rtcp-fb:102 transport-cc
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:124 rtx/90000
a=fmtp:124 apt=102
a=rtpmap:127 H264/90000
a=rtcp-fb:127 ccm fir
a=rtcp-fb:127 nack
a=rtcp-fb:127 nack pli
a=rtcp-fb:127 goog-remb
a=rtcp-fb:127 transport-cc
a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f

[Harald Alvestrand]

According to my profile-ID decoder ring, 4D401F is "main" profile, which uses a different subset of the H.264 functions than High, although both are a superset of Baseline. (the bit set in the 2nd byte, 0x40, indicates that the constraint for "high" is in effect).

I haven't seen anyone say they're working on Main profile support for OpenH264, I don't know if it is common in realtime-suitable hardware decoders.

Note: My decoder ring is strictly back-of-the-envelope quality. Others may know different.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrtc+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/26645d01-f685-41f4-aa49-80ae5118aea6%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.