H264 Codec Integration for iOS in WebRTC

[Alexandre Ohayon]

Hi everyone,

I'm currently trying to implement the support of H264 in webRTC for iOS. For my use, I'm focusing on decoding a H264 stream. I know it's pretty hard, but I might help many people.

From what I read, I see now 3 possible solutions to the problem:

- implementing directly the H264 in the webRTC framework for iOS (using what as been done for VP8 and do an analog implementation), and then recompile the WebRTC libs. This might use FFMPEG libs compiled for iOS. It still looks very complicated.

- using VIEExternalCodec (http://www.webrtc.org/reference/webrtc-internals/vieexternalcodec) and trying to integrate the H264 codec this way. The problem is that VIEExternalCodec doesn't look implemented in WebRTC for iOS

- transmitting binaries through the DataChannel, and then decode the binaries outside the WebRTC framework, using either FFMPEG or AVFoundation for the use of hardware decoding on the device. The problem is that DataChannel doesn't look implemented in WebRTC for iOS.

Did someone already succeed in doing such an implementation? If yes, what's the easiest way to do it? (maybe there's another way..)

Thank in advance for your help.

[Kaiduan Xie]

You can apply the patch from

https://groups.google.com/forum/#!topic/discuss-webrtc/U-y3or-dBOU

and build OpenH264 on iOS.

/Kaiduan

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[alexandr...@withings.com]

Hi,

Thanks a lot for your help. I carefully read your post about integration of openH264 in webRTC and made a lot of tests to implement it correctly. Currently, I'm getting the following error in the compilation of my app, which is directly linked to the lib webrtc_video_coding, modified by the patch :

"Undefined symbols for architecture armv7:
"webrtc::H264Decoder::Create()", referenced from:
webrtc::VCMCodecDataBase::CreateDecoder(webrtc::VideoCodecType) const in libwebrtc_video_coding.a(webrtc_video_coding.codec_database.o)"

the only things I did for now to get this error is getting the webRTC trunk from webrtc svn, and applied the patch you posted on this page: https://webrtc-codereview.appspot.com/11239004/

[Kaiduan Xie]

Alexandre,

You forgot to link webrtc_h264 library.

/Kaiduan

--

[alexandr...@withings.com]

Thanks for your help.

Now I have sucessfully patched my webRTC trunk and linked everything with openH264.

But now, when I get the sdp, the H264 codec is now displaying? I get several audio codecs like opus, ISAC and video codecs like VP8.

Here is the video part of my sdp:

a=mid:video
a=extmap:2 urn:ietf:params:rtp-hdrext:toffset
a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=sendrecv
a=rtcp-mux
a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:TKcN6NGHYUvPEMu...
a=rtpmap:100 VP8/90000
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=rtcp-fb:100 goog-remb
a=rtpmap:116 red/90000
a=rtpmap:117 ulpfec/90000

Do you have any idea what I need to add to have to choice to use h264?

[Kaiduan Xie]

Please apply the libjingle patch in the following post,

https://groups.google.com/forum/#!msg/discuss-webrtc/U-y3or-dBOU/DbvgQOMcx30J

/Kaiduan

[alexandr...@withings.com]

Thanks a lot, it's working perfectly ;)

[alexandr...@withings.com]

@Kaiduan,

Hi! I'm currently trying to improve your patch (especially for iOS). I get a crash on Forward Error Correction when I send frames with a big size. I tried increasing the maximum bitrate allowed in h264_impl.cc , setting the param.iTargetBitrate to a high value (like 10 000 000), but it didn't fix anything. I also tried to use this quick fix: https://groups.google.com/forum/#!searchin/discuss-webrtc/fec_packet_list_/discuss-webrtc/lk5YzEKllKs/xGgs7Ysxi6EJ .

I get a malloc_error, and it happens on several platforms: iOS and Linux, so it's a problem directly in the webrtc libs. Do you have an idea of what to modify to solve this issue? It simply avoids us to send good quality packets.

Thanks a lot for your help.

[Bridger Maxwell]

I got that patch applied successfully (thanks!) but when I run AppRTCDemo on two iOS devices (an iPad Air and an iPhone 5s), I only get one frame every 5 seconds or so, even with the camera running at 192x144. The local video renders fine, but the remote video is coming in extremely slowly. When I use vp8, it runs at a decent speed.

I'm trying to profile the app to see what is slow, but nothing obvious is showing up. The CPU is only at about 20% utilization. Any ideas on what could be happening? Is openh264 giving good results for others on iOS?

Once I get this working, I'd like to try using the hardware encoder / decoder exposed in iOS 8.

- Bridger Maxwell

[tech74]

Hi 

Did you solve problem with slow send H264 bitrate, I'm getting same results as yourself 7 fps and send bitrate of 80Kbs

THanks

[I Ainx]

Hi Bridger,

Did you ever get the hardware encoder/decoder working?

thanks,

iain

On Friday, June 27, 2014 5:37:50 AM UTC+1, Bridger Maxwell wrote:

[Divyam Goel]

still seeing lower outgoing frame rates with h264 video encoding, any advice ?

[Iñaki Baz Castillo]

2014-04-17 15:47 GMT+02:00 Alexandre Ohayon <ohayo...@gmail.com>:
> The problem is that DataChannel doesn't look implemented in WebRTC for iOS.

It is.


--
Iñaki Baz Castillo
<i...@aliax.net>