Firefox not accepting SDPs for H264 that contain profile-level-id 420029

[an...@huge.geek.nz]

Hi,

I've been struggling with using WebRTC together with Janus.

I have an h264 camera that I can't alter the configuration for.

The profile-level-id seems to be 420029, which means:
profile is baseline (42 hex = 66 dec = baseline)
constraints 0
level-idc is 4.1

ICE never completes because I suspect FF just barfs on this for some reason, and I couldn't, no matter how much debug I enabled, find where or why specifically.

Chrome works perfectly, but not before offering back profile-id 42e01f (which the main difference is level-idc is 3.1 instead of 4.1).

When I re-encoded the content using FFMPEG using an intermediate device, setting exactly 42e01f then FF finished ICE and played the video (although this is not a valid solution for my purposes)

Ideally I want to understand why FF (openh264) can't play 420029, since it should be baseline, and if there is a fix.

One more important note: if I took a small clip of the stream to a .mp4 file (no transcoding), and embed that in HTML5 video tag, FF plays it perfectly. So the codec SHOULD work as is, but not it doesn't when it's coming over webrtc.

Very annoying!

[an...@huge.geek.nz]

Hi all, I managed to "fix" this by rewriting the profile-level-id from 42009 to 42e01f before handing the SDP to FF.

I agree it is a hack, but as FF clearly works with this, then why don't you allow 42009?

Much more detail on the Janus google group:

https://groups.google.com/forum/#!topic/meetecho-janus/jKg5u9421kM

[Randell Jesup]

On 1/27/2017 2:27 PM, an...@huge.geek.nz wrote:
> On Friday, January 27, 2017 at 3:27:15 PM UTC+1, an...@huge.geek.nz wrote:
>

> Hi all, I managed to "fix" this by rewriting the profile-level-id from 42009 to 42e01f before handing the SDP to FF.
>
> I agree it is a hack, but as FF clearly works with this, then why don't you allow 42009?

The WG requirement is for Constrained Baseline, not Baseline, which is
what we offer/accept. It's the 'e0' portion that causes the problem;
see RFC 6184 for details on what's compatible for offer/answer. Yes,
the rules are annoying if you want to support talking to a client that
doesn't support the same profile you prefer. Officially, to support
ConstrainedBaseline and Baseline you need to offer both as separate
payload types (with independent FMTPs).

This is this same thing
(https://groups.google.com/forum/#!searchin/mozilla.dev.media/4200xx%7Csort:relevance/mozilla.dev.media/9_zkibr_Bus/8LioRaUODAAJ)
that was noted in the Janus list you referenced. And as you mention,
you can 'cheat' and claim it's constrained baseline even though it's
not, and in this case it will work. (Not every such edit will --- if
you tell it the profile is baseline when it's high, it will likely
fail.) And how well these edits work depend on details of
implementation of the decoders.

--
Randell Jesup, mozilla

[an...@huge.geek.nz]

Thank you Randell,

If I understand you correctly, then FF checks the profile-level-id and checks the constraint bits. If it doesn't find constrained baseline then it simply rejects it (according to the RFC), even though the underlying decoder would be able to play it.

In my scenario, it's an RTSP camera with no options for altering the encoding options. And then Janus doesn't want to "know" about media or codecs, it just maps it from RTSP to RTP.

There's only two ways to be "compliant" then:
1) To transcode the stream to constrained baseline (a heavy option ..)
2) During SDP signalling to offer both CBP and BP (not sure how that is done, do you offer two SDPs?)

For 2), how would Firefox behave? Would it decide it could in fact accept the BP content, and because standards were followed (two offers), it would work rather than reject it?

I guess that, the "proper" solution is to encode in the right format in the first place if you intend to use webrtc/rtp transport.

Regards,
Anton

[Randell Jesup]

On 1/31/2017 12:56 PM, an...@huge.geek.nz wrote:
> On Monday, January 30, 2017 at 2:35:54 AM UTC+1, Randell Jesup wrote:

>> The WG requirement is for Constrained Baseline, not Baseline, which is
>> what we offer/accept. It's the 'e0' portion that causes the problem;
>> see RFC 6184 for details on what's compatible for offer/answer. Yes,
>> the rules are annoying if you want to support talking to a client that
>> doesn't support the same profile you prefer. Officially, to support
>> ConstrainedBaseline and Baseline you need to offer both as separate
>> payload types (with independent FMTPs).
>>
>> This is this same thing
>> (https://groups.google.com/forum/#!searchin/mozilla.dev.media/4200xx%7Csort:relevance/mozilla.dev.media/9_zkibr_Bus/8LioRaUODAAJ)
>> that was noted in the Janus list you referenced. And as you mention,
>> you can 'cheat' and claim it's constrained baseline even though it's
>> not, and in this case it will work. (Not every such edit will --- if
>> you tell it the profile is baseline when it's high, it will likely
>> fail.) And how well these edits work depend on details of
>> implementation of the decoders.
>>
>> --
>> Randell Jesup, mozilla

> Thank you Randell,
>
> If I understand you correctly, then FF checks the profile-level-id and checks the constraint bits. If it doesn't find constrained baseline then it simply rejects it (according to the RFC), even though the underlying decoder would be able to play it.
>
> In my scenario, it's an RTSP camera with no options for altering the encoding options. And then Janus doesn't want to "know" about media or codecs, it just maps it from RTSP to RTP.
>
> There's only two ways to be "compliant" then:
> 1) To transcode the stream to constrained baseline (a heavy option ..)
> 2) During SDP signalling to offer both CBP and BP (not sure how that is done, do you offer two SDPs?)
>
> For 2), how would Firefox behave? Would it decide it could in fact accept the BP content, and because standards were followed (two offers), it would work rather than reject it?

You offer two payload types: one with profile-level-id=42e0XX, one with
profile-level-id=4200xx. Firefox will only accept the 42e0xx. However,
in this case, I really doubt there's any bitstream difference at the
sender - it's just going to send a stream, and in reality it likely
conforms to Constrained baseline:

"Baseline Profile: This profile includes all features that are
supported in the Constrained Baseline Profile, plus three additional
features that can be used for loss robustness (or for other purposes
such as low-delay multi-point video stream compositing). The
importance of this profile has faded somewhat since the definition
of the Constrained Baseline Profile in 2009. All Constrained
Baseline Profile bitstreams are also considered to be Baseline
Profile bitstreams, as these two profiles share the same profile
identifier code value."

So if it doesn't use those features, it's compatible with Constrained
Baseline. And the OpenH264 decoder may well accept those features even
if configured for Constrained Baseline.

Note that IDRs will likely be preceded by SPS/PPS NAL units, and those
will specify 4200xx instead of 40e0xx, but likely that doesn't matter
either.

So, if you *always* convert from 4200xx to 42e0xx it will likely work
with both chrome and firefox (and edge).

You can also offer 4200xx, and if it's rejected renegotiate, offering
42e0xx. That will slightly slow connect time.

> I guess that, the "proper" solution is to encode in the right format in the first place if you intend to use webrtc/rtp transport.

Sure!

--
Randell Jesup, Mozilla

[daniel....@gmail.com]

Hello,

I've came across to the same problem.
My RTSP camera is sending it's H264 stream as 420029.
Also using Janus as RTSP->WebRTC.

However if I rewrite the profile-level-id to 42e01f, FF is receiving the stream, however it can not decode it.
I've tried everything so far.
However, for example: https://youtube.com/html5 shows me that this browser is capable of H264.

I would need a deeper debug level to this problem.
How can I make the stream visible?

I'm using FF v59.0.2 on Ubuntu 16.04LTS.

Any help is greatly appreciated.
Thanks Folks!