H265 RTP payload from Apple devices

[Alex Pokotilo]

Hi all!

If I enable h265 support for IPhone, IPad or Mac like described here https://webrtcbydralex.com/index.php/2020/04/28/how-to-enable-hevc-h265-and-av1-in-webrtc-in-your-browser/ I get strange RTP payload from Apple devices.

According to https://datatracker.ietf.org/doc/html/rfc7798  the first two byte in payload should be payload header but this is not what I see in payload comes from Apple Devices.

Here is a list of type, layerid and tid I get from rtp packets(first two bytes parsed):

t=1,l=32,ti=0
t=0,l=34,ti=3
t=0,l=32,ti=2
t=0,l=54,ti=7
t=0,l=36,ti=2
t=0,l=54,ti=1
t=0,l=33,ti=3
t=1,l=0,ti=0
t=1,l=0,ti=0
t=1,l=0,ti=0
t=0,l=15,ti=5
t=1,l=0,ti=0
t=0,l=27,ti=3
t=1,l=0,ti=0
t=1,l=0,ti=0
t=0,l=20,ti=1
t=1,l=0,ti=0
t=1,l=0,ti=0
t=1,l=0,ti=0
t=0,l=23,ti=4
t=1,l=0,ti=0
t=0,l=14,ti=4
t=1,l=0,ti=0

According to the standard type 0 or 1 signals that payload is single nal unit. But I see annex_b 0 0 0 1 sequences inside rtp payload so rtp payload  is not single nal unit. it is not aggregated/fu as well.

Attached are headers and slice of payloads of the first and second rtp h265 packets from beginning or webrtc session.

If I cut strange first byte and combine all rtp payloads with the same timestamps I get valid h265 frame I can play in players. if I just combine all rtp payloads with the same timestamp I get garbage in playback.

Resulting  h265 frame has multiple nal-units separated by annex_b_delimiter 0 0 0 1

 As for me, RTP payload for h265 produced by IOS/MacOS is malformed. packetization-mode is not presented in Apple devices' offers and hence should be considered as packetization-mode=0 where the only single nal unit per rtp packet allowed. but what I see is not single not unit but something else.

Could anybody share some thoughts about this ? may be somebody had experience with h265 support in webrtc on Apple or another devices

[Alex Pokotilo]

The same problem found, described and resolved here https://github.com/AlexxIT/Blog/issues/5

In short Apple's h265 whoever implemented it is not according to rfc7798

вторник, 5 июля 2022 г. в 15:16:43 UTC+6, Alex Pokotilo:

[Philipp Hancke]

from what I recall they put the 00 00 00 01 start code into the payload. Which is not how H26* packetization works at all.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/53a5aab4-10ba-447e-a750-30fbc479dc4bn%40googlegroups.com.

[Sergio Garcia Murillo]

There is an PR already available for fixing the h265 packetization at WebKit:

https://github.com/WebKit/WebKit/pull/15494

As a I side note, we are working on contributing h265 support to libdatachannel/OBS WHIP: 

https://github.com/paullouisageneau/libdatachannel/pull/932

Any feedback/review is welcome.

Best regards

Sergio

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CADxkKiKB0BPFDrkj-9PcpXBM--921%3D1vz3drk0YoErOGdfaRxg%40mail.gmail.com.

[Alex Pokotilo]

Sergio,

thanks a lot for WebKit PR reference. How long from your POV would it take to be released on IOS to test it ?

пятница, 21 июля 2023 г. в 13:14:04 UTC+6, Sergio Garcia Murillo:

[Sergio Garcia Murillo]

I don't have much visibility about that sorry, but verifying the patches and providing reviews might make it easier for Apple to merge it.

Best regards

Sergio

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/80513632-9405-4281-b2ce-4666979477a6n%40googlegroups.com.

[Alex Pokotilo]

Sure, reviews are very important. I wish I can help being dummy with WebKit codebase

пт, 21 июл. 2023 г. в 13:49, Sergio Garcia Murillo <sergio.gar...@gmail.com>:

You received this message because you are subscribed to a topic in the Google Groups "discuss-webrtc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/discuss-webrtc/Zc3-3hddEn0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CA%2Bag07ZhYXSDhs0cMjTDdjzM42c4cYfyTRCOBjMEuTTRVOamYQ%40mail.gmail.com.

[Sergio Garcia Murillo]

While testing on iOS is hard, you can compile and test webkit on macos quite easily:

https://webkit.org/getting-started/

On Fri, Jul 21, 2023 at 9:53 AM Alex Pokotilo <gja...@gmail.com> wrote:

Sure, reviews are very important. I wish I can help being dummy with WebKit codebase

пятница, 21 июля 2023 г. в 13:49:08 UTC+6, Sergio Garcia Murillo:

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/18061f6f-a344-4ce6-a7f4-df6648c9c736n%40googlegroups.com.