H.264 patch for Google Chrome webrtc video call

[Kaiduan Xie]

Hi all,

I created a patch to support H.264 baseline webrtc video call to Google Chrome using OpenH264 from CISCO.

The patch is based on Google Chrome Release 31.0.1650.63, and tested on Mac.

Single NALU/non-interleaved packetization mode is supported, fragmentation unit A is used for fragmentation.

Welcome comments and collaborations.

Cheers,

/Kaiduan

[Alexandre GOUAILLARD]

Hi KX,

Comments and collaboration would be happily provided ….. but where is the code? :D

Alex.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Alex. Gouaillard, PhD, PhD, MBA

------------------------------------------------------------------------------------

CTO - Temasys Communications, S'pore / Mountain View

President - CoSMo Software, Cambridge, MA

------------------------------------------------------------------------------------

sg.linkedin.com/agouaillard

• 
  

[Kaiduan Xie]

Alex,

The patch has two parts, this first part is to add switch cases to handle H264 on the webrtc media stack pipe (from video engine to rtp sender/receiver/packetization, finally to video coding module) The second part is OpenH264 encoder/decoder.

OpenH264's build system is not based on GYP, so I have to build OpenH264 separately.

The best way is to merge this patch to Google Chrome webrtc code, I am happy to contribute it back. Justin, what do you think?

Thanks,

/Kaiduan

[Kaiduan Xie]

The patch is not in Chrome code base, it is in webrtc media stack (webrtc.org), so it is helpful to anyone who uses webrtc native and wants to add H.264 support.

/Kaiduan

[M. Kirschbaum]

Hi Kaidian,

thats great news! But, where can we find the code (URL...)?

Bests
M.

[Kaiduan Xie]

Mark,

I am working with the legal team on this now.

 

/Kaiduan

[Justin Uberti]

Thanks for the heads-up. Since OpenH264 is a separate project, how do you plan to ensure that this patch doesn't cause any problems for regular webrtc.org users?

[Kaiduan Xie]

Justin,

What problems you are aware this patch may cause for regular webrtc.org users? Are you talking about the IPR issue? If yes, like OpenH264 license and webrtc.org, the patch itself is BSD licensed, and the user of this patch is responsible for paying all applicable license fees for H.264.

Thanks,

/Kaiduan

[Justin Uberti]

We cannot take any patch that introduces an IPR issue (i.e. creates a compile-time dependency on OpenH264).

[Justin Uberti]

... which is why I was asking about how you were planning to control this functionality so that by default, WebRTC doesn't depend on OpenH264.

[Kaiduan Xie]

Justin,

You raised a very good and valid point.

The patch has two parts, the first part is to add switch cases to support H.264 in the video media pipe (from video engine, rtp sender/receiver, and video coding module), this part does not depend on OpenH264.

The second part is a wrapper for OpenH264 like the vp8_impl.cc/vp8_imp.h, this part depends on OpenH264.

Can we merge the first part to webrtc.org? Of course I will submit this part for your team to review first.

And I will find way to host the second part.

Thanks,

/Kaiduan

[Justin Uberti]

That plan sounds good.

[Emmanuel Venisse]

Do you have some news about it?

Thanks

[Kaiduan Xie]

Please see the attached files for the patch.

1. webrtc/modules/video_coding/codecs/h264 (this is the OpenH264 wrapper)

      include/h264.h

      h264.gyp

      h264_imp.h

      h264_impl.cc

2. webrtc/modules/rtp_rtcp/rtp_format_h264.h/cc

3. apply the webrtc-h264.patch to webrtc directory

Justin, I will contact you offline about the review process.

Thanks,

/Kaiduan

[BobK]

Nice work Kaiduan.  Will this eventually become an integral part of Chrome?  If so, is there a rough timeframe?  

Bob Kirnum

Dialogic

[Justin Uberti]

Please upload this to https://webrtc-codereview.appspot.com/

--

[Arik Yaacob]

Kaiduan,
I was trying to do something similar, incorporating OpenH264 into WebRTC. I get an error while compiling due to the fact there is no virtual destructor defined for ISVCDecoder/ISVCEncoder. Did you faced this issue?
I also facing segmentation fault when the destructor is called. Not sure if it is related.
Thanks,
Arik

[Kaiduan Xie]

For those of you who are interested, the patch is uploaded to,

https://webrtc-codereview.appspot.com/11239004/

Many thanks to Justin and Ami for the help,

/Kaiduan

[andy424]

@kaiduan,

I applied the patchset listed here. It includes dummy h264 implementation. 

https://webrtc-codereview.appspot.com/11239004/

Can you briefly enumerate the steps on what I would need to do to undo the dummy h264 and replace with the openh264. A README file in the patchset would be most useful

Thanks

[Kaiduan Xie]

Please follow the steps listed below,

1) Replace dummy_h264.gyp with openh264.gyp in webrtc.gyp and video_coding.gypi files.

2) Build your own OpenH264 on the platform you are working on, for example Mac.

3) Replace the directory in openh264.gyp with the directory where OpenH264 header files and lib files are saved.

--

[andy424]

@kaiduan

Thanks for those tips -- I was able to follow them and successfully compile(Have not tested yet). I had to make the following minor edits to your h264_impl.cc file and also the makefile refers to libh264common.a whereas OpenH264 refers to it as libcommon.a

--- h264_impl.cc 2014-03-31 18:18:50.000000000 -0500

+++ h264_impl.cc 2014-04-08 20:29:42.000000000 -0500

@@ -53,7 +53,7 @@ int H264EncoderImpl::Release() {

     encoded_image_._buffer = NULL;

   }

   if (encoder_ != NULL) {

-    DestroySVCEncoder(encoder_);

+    WelsDestroySVCEncoder(encoder_);

     encoder_ = NULL;

   }

   inited_ = false;

@@ -103,7 +103,7 @@ int H264EncoderImpl::InitEncode(const Vi

     return ret_val;

   }

   if (encoder_ == NULL) {

- ret_val = CreateSVCEncoder(&encoder_);

+ ret_val = WelsCreateSVCEncoder(&encoder_);

     if (ret_val != 0) {

      WEBRTC_TRACE(webrtc::kTraceError, webrtc::kTraceVideoCoding, -1,

                 "H264EncoderImpl::InitEncode() fails to create encoder ret_val %d",

@@ -125,7 +125,7 @@ int H264EncoderImpl::InitEncode(const Vi

     WEBRTC_TRACE(webrtc::kTraceError, webrtc::kTraceVideoCoding, -1,

                  "H264EncoderImpl::InitEncode() fails to initialize encoder ret_val %d",

                  ret_val);

-    DestroySVCEncoder(encoder_);

+    WelsDestroySVCEncoder(encoder_);

     encoder_ = NULL;

     return WEBRTC_VIDEO_CODEC_ERROR;

   }

@@ -309,7 +309,7 @@ int H264DecoderImpl::InitDecode(const Vi

     return ret_val;

   }

   if (decoder_ == NULL) {

-    ret_val = CreateDecoder(&decoder_);

+    ret_val = WelsCreateDecoder(&decoder_);

     if (ret_val != 0) {

       decoder_ = NULL;

       return WEBRTC_VIDEO_CODEC_ERROR;

@@ -324,7 +324,7 @@ int H264DecoderImpl::InitDecode(const Vi

   ret_val = decoder_->Initialize(&dec_param);

   if (ret_val != 0) {

     decoder_->Uninitialize();

-    DestroyDecoder(decoder_);

+    WelsDestroyDecoder(decoder_);

     decoder_ = NULL;

     return WEBRTC_VIDEO_CODEC_ERROR;

   }

@@ -461,7 +461,7 @@ int H264DecoderImpl::Release() {

   }

   if (decoder_ != NULL) {

     decoder_->Uninitialize();

-    DestroyDecoder(decoder_);

+    WelsDestroyDecoder(decoder_);

     decoder_ = NULL;

   }

   inited_ = false;

[andy424]

@kaiduan

No success when I tried to run your patched version on iOS. AppRTC Application crashes immediately upon launch 

On Android I faced some of the same issues pointed by @Arik and was not able to get a compiled binary

Have you or anyone else successfully verified you patches(non-dummy) relative to a recent revision of webrtc (r5844??)

Thanks

[Arik Yaacob]

Andy,
The problem I faced was specific to building on Android. I fixed it by commenting out the lines in "build/common.gypi" and "webrtc/build/common.gypi" that enable warnings for non declared virtual destructor. Just comment out any occurrence of '-Wno-non-virtual-dtor' in the gypi files.

[Gokul D]

@Kaiduan,

   Is there any test application available to verify this(H264 webrtc) implementation ?.I am able to build the libjingle_peerconnection.jar & libjingle_peerconnection_so.so with openH264 support usign ther patch provided and not sure how to test the same. 

Thanks

Gokul 

[Kaiduan Xie]

The first message said,

"The patch is based on Google Chrome Release 31.0.1650.63, and tested on Mac."

/Kaiduan

--

[Phil Yu]

Hi Kaiduan

Could you porting this (H264 on webrtc) on newest chrome release

thanks 

kaiduan於 2014年3月20日星期四UTC+8上午3時46分33秒寫道：

Hi all,

I created a patch to support H.264 baseline webrtc video call to Google Chrome using OpenH264 from CISCO.

The patch is based on Google Chrome Release 31.0.1650.63, and tested on Mac.

[Enda Mannion]

I got this patch to work with the latest Chromium.   I attached the files I needed to change.  One note,  to improve video quality I had to increase the bitrate.  I did this in h264_impl.cc.

param.iTargetBitrate = 128000;

Works really well for me now, I test using apprtc.

https://apprtc.appspot.com/?r=87227771&debug=loopback


-Enda

[Enda Mannion]

I have got these patches to work on Windows with not too many changes.

Provided you update your h264.gyp file to point to OpenH264 you just need one other change.

I had to make this small change to content.gyp see attached patch.

Then before building
set GYP_DEFINES=component=shared_library

then build.  Works then to another windows build or to the Mac build of Chromium.

Enda


On Monday, April 28, 2014 9:45:52 AM UTC+1, Phil Yu wrote:

[Phil Yu]

Hi 

I have already porting patch on webrtc which in newest chromium , and use apprtc test , but how can I use h264 not vp8 codec

thank

[Kaiduan Xie]

Phil,

Codec negotiation is not supported yet. Assume the payload id for H.264 is 101 as defined in modules/video_coding/main/interface/video_coding_defines.h

Please apply the attached patch to libjingle library.

Thanks,

/Kaiduan

[Phil Yu]

Hi all

I use MinGW build openh264 on Windows 7 , so I make libwels.a ..... 

And chromiun => h264.gyp just use wels.lib

Any one know how to make wels.lib or how to modify h264.gyp use wels.a

thanks 

[andy424]

@Kaiduan or anyone else who has has success integrating openh264 on iOS,

I successfully compiled AppRTCDemo.app (webrtc/libjingle svn revision 6040) with the patches mentioned on this thread for iOS

When I launch AppRTCDemo, the binary immediately crashes and when I look at the console crash log using Xcode Organizer, I see that it is trying to load the dynamic library  libopenh264.dylib instead of the static library  libopenh264.a

Any idea how I can ask it to use the static library (since I don't know how to compile a dynamic library for libopenh264 ?

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Not saving crash log because we have reached the limit for logs to store on disk.  Sync or otherwise clear logs from /var/mobile/Library/Logs/CrashReporter to save new logs.

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Could not save crash report to disk!

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Incident Identifier: 3F3C2536-390C-4AF8-A373-97C7B4B34B71

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: CrashReporter Key:   063bb1cfe11e2c650092bd31be974a310750f61f

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Hardware Model:      iPhone4,1

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Process:             AppRTCDemo [859]

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Path:                /var/mobile/Applications/BBE4FEEC-5F7B-4CB5-9C6D-053850815643/AppRTCDemo.app/AppRTCDemo

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Identifier:          AppRTCDemo

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Version:             ???

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Code Type:           ARM (Native)

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Parent Process:      launchd [1]

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Date/Time:           2014-05-20 16:06:50.394 -0500

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: OS Version:          iOS 7.1 (11D167)

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Report Version:      104

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Exception Type:  EXC_BREAKPOINT (SIGTRAP)

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Exception Codes: 0x0000000000000001, 0x00000000e7ffdefe

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Triggered by Thread:  0

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Dyld Error Message:

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>:   Library not loaded: libopenh264.dylib

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>:   Referenced from: /var/mobile/Applications/BBE4FEEC-5F7B-4CB5-9C6D-053850815643/AppRTCDemo.app/AppRTCDemo

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>:   Reason: image not found

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>:   Dyld Version: 324.1

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: Binary Images:

May 20 16:06:50 iPhone4S ReportCrash[860] <Error>: 0x2be0b000 - 0x2be2bfff dyld armv7  <651a31c39f71311f965f8ac44de02c88> /usr/lib/dyld

[alexandr...@withings.com]

@andy424

Which libs did you link in your project? In my working project, i linked libwebrtc_h264.a compiled with ninja, and from openH264 libcommon.a, libdecoder.a, libencoder.a,libprocessing.a .

If it doesn't work, have a look in the h264.gyp to see if it's pointing to the correct openH264directory. (if you have an old version of h264.gyp, I think it's looking for -lwels instead of -lopenh264.

[andy424]

@alexandr

Thanks for your suggestions. It turns out, I had an extraneous libopenh264.dylib in my openh264-ios directory. Not sure if the openh264 build script generated it. Once I removed the extraneous libopenh264.dylib file from there, the AppRTCDemo build does not crash anymore as soon as I launch it. Next, I need to build a mac version of chromium with the h264 patch

Anands-MacBook-Pro:apprtc-ios-r6040-openh264 anand$ ls /Users/anand/apprtc-ios-r6040-openh264/openh264-ios/

CODING_STYLE Makefile build libcommon.a libopenh264.a test

CONTRIBUTORS README code-coverage.sh libdecoder.a libopenh264.dylib openh264.def testbin

LICENSE README.md codec libencoder.a libprocessing.a res

[Phil Yu]

Hi Kaiduan

in packet.cc

if (videoHeader.codecHeader.H264.single_nalu) {

       isFirstPacket = true;

       markerBit     = true;

}

If input stream is multiple slice in one frame , every slice is single nalu , but just last slice packet market bit will set "1"

so if detect single nalu to set markerBit = true 

in some h264 encode stream will error 

[Kaiduan Xie]

The reason why isFirstPacket/markerBit is set in packet.cc is to process the single NALU unit as a completed frame, and then jitter buffer will deliver the completed frame to decoder.

This patch is only tested with the OpenH264 encoder/decoder attached in this email chain.

/Kaiduan

[alexandr...@withings.com]

@ Kaiduan,

I noticed that the H264 Stream is always sent with a baseline Profile. Did you try with a main profile? Do you know where we can specify the profile in the patch?

Thanks a lot

[Sridhar B]

Hi,

I applied the successfully but where do I specify to use H264 codec ?

Is it by changing the SDP lines before setRemoteDescription and setLocalDescription ?

Thanks,

[Kaiduan Xie]

Sridhar,

Please apply libjingle.patch in this thread.

/Kaiduan

[Sridhar]

Thanks for the reply @kaiduan.

Now I am able to see H264 in the SDP, but I am getting an error SetLocalDescription Failed to set local offer sep, Session error code ERROR_CONTENT when setting up local description.

I am using the WebRTC revision number 6488 as with the patched revision number 5844 is unable to compile for iOS.

any clue on what is happening ?

below is the SDP info.

o=- 2929974430173482030 2 IN IP4 127.0.0.1

s=-

t=0 0

a=group:BUNDLE audio video

a=msid-semantic: WMS 4nP5yeIDzbUm6IJho5jkTps1lnfabsFvTXjH

m=audio 1 RTP/SAVPF 111 103 9 102 0 8 106 105 13 127 126

c=IN IP4 0.0.0.0

a=rtcp:1 IN IP4 0.0.0.0

a=ice-ufrag:csvcrnNYZOjdCT0g

a=ice-pwd:632AfmwA9mxyTRd+Fa1hPnTK

a=ice-options:google-ice

a=fingerprint:sha-1 98:9B:36:32:F0:E6:A8:6B:94:15:29:53:D9:D5:14:9E:DA:65:FC:28

a=setup:actpass

a=mid:audio

a=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level

a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time

a=sendrecv

a=rtcp-mux

a=rtpmap:111 opus/48000/2

a=fmtp:111 minptime=10

a=rtpmap:103 ISAC/16000

a=rtpmap:9 G722/16000

a=rtpmap:102 ILBC/8000

a=rtpmap:0 PCMU/8000

a=rtpmap:8 PCMA/8000

a=rtpmap:106 CN/32000

a=rtpmap:105 CN/16000

a=rtpmap:13 CN/8000

a=rtpmap:127 red/8000

a=rtpmap:126 telephone-event/8000

a=maxptime:60

a=ssrc:1123795929 cname:CPLUvZsbx5YoG4lJ

a=ssrc:1123795929 msid:4nP5yeIDzbUm6IJho5jkTps1lnfabsFvTXjH 4nP5yeIDzbUm6IJho5jkTps1lnfabsFvTXjH00

a=ssrc:1123795929 mslabel:4nP5yeIDzbUm6IJho5jkTps1lnfabsFvTXjH

a=ssrc:1123795929 label:4nP5yeIDzbUm6IJho5jkTps1lnfabsFvTXjH00

m=video 1 RTP/SAVPF 101 116 117 96

c=IN IP4 0.0.0.0

a=rtcp:1 IN IP4 0.0.0.0

a=ice-ufrag:csvcrnNYZOjdCT0g

a=ice-pwd:632AfmwA9mxyTRd+Fa1hPnTK

a=ice-options:google-ice

a=fingerprint:sha-1 98:9B:36:32:F0:E6:A8:6B:94:15:29:53:D9:D5:14:9E:DA:65:FC:28

a=setup:actpass

a=mid:video

a=extmap:2 urn:ietf:params:rtp-hdrext:toffset

a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time

a=sendrecv

a=rtcp-mux

a=rtpmap:101 H264/90000

a=rtcp-fb:101 ccm fir

a=rtcp-fb:101 nack

a=rtcp-fb:101 nack pli

a=rtcp-fb:101 goog-remb

a=rtpmap:116 red/90000

a=rtpmap:117 ulpfec/90000

a=rtpmap:96 rtx/90000

a=fmtp:96 apt=100

a=ssrc-group:FID 950831764 587119607

a=ssrc:950831764 cname:CPLUvZsbx5YoG4lJ

a=ssrc:950831764 msid:4nP5yeIDzbUm6IJho5jkTps1lnfabsFvTXjH 4nP5yeIDzbUm6IJho5jkTps1lnfabsFvTXjH01

a=ssrc:950831764 mslabel:4nP5yeIDzbUm6IJho5jkTps1lnfabsFvTXjH

a=ssrc:950831764 label:4nP5yeIDzbUm6IJho5jkTps1lnfabsFvTXjH01

a=ssrc:587119607 cname:CPLUvZsbx5YoG4lJ

a=ssrc:587119607 msid:4nP5yeIDzbUm6IJho5jkTps1lnfabsFvTXjH 4nP5yeIDzbUm6IJho5jkTps1lnfabsFvTXjH01

a=ssrc:587119607 mslabel:4nP5yeIDzbUm6IJho5jkTps1lnfabsFvTXjH

a=ssrc:587119607 label:4nP5yeIDzbUm6IJho5jkTps1lnfabsFvTXjH01

Thanks again.

You received this message because you are subscribed to a topic in the Google Groups "discuss-webrtc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/discuss-webrtc/U-y3or-dBOU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to discuss-webrt...@googlegroups.com.

[alexandr...@withings.com]

It's because of this line: a=fmtp:96 apt=100

it's trying to access the VP8, the problem is that the patch replace the VP8 codec by H264 codec.

A quick fix is to enable again VP8 in your SDP. To do this, uncomment the following line in the file talk/media/webrtc/webrtcvideoengine.cc:

{kVp8PayloadName, 100, -1, 0}

Le mercredi 19 mars 2014 20:46:33 UTC+1, kaiduan a écrit :

[Sridhar]

Thanks for the reply. I actually need to test the H264 patch so I have to avoid vp8 and use H264. Do you have any idea that how can I change that line in SDP so that it will point to kH264PayloadName

[Sridhar]

After deep debugging, the problem seems to be here. It is in method

bool WebRtcVideoMediaChannel::SetReceiveCodecs(

    WebRtcVideoChannelRecvInfo* info) {

      if (apt_it != associated_payload_types_.end()) {

        std::map<int, webrtc::VideoCodec*>::iterator codec_it =

            pt_to_codec.find(apt_it->second);

        // We currently only support RTX associated with VP8 due to limitations

        // in webrtc where only one RTX payload type can be registered.

        valid_apt = codec_it != pt_to_codec.end() &&

            _stricmp(codec_it->second->plName, kH264PayloadName) == 0;

      }

      if (!valid_apt) {

        LOG(LS_ERROR) << "The RTX codec isn't associated with a known and "

                         "supported payload type";

        return false;

      }

[Glenn Vlekke]

I want it also to work on libjingle.

What i did was:

1. Checkout the latest version of libjingle r6589
2. Applied the patch set 3 on https://webrtc-codereview.appspot.com/11239004/
3. Added the files that kaiduan gived to us to replace it with the dummy files
4. applied the libjingle.patch 

When im building i see this:

ninja: Entering directory `out/Debug/'

ninja: warning: multiple rules generate icudtl.dat. builds involving this target will not be correct; continuing anyway

[1932/2297] CXX obj/webrtc/modules/video_coding/codecs/h264/webrtc_h264.h264_impl.o

FAILED: /home/adminuser/webrtc/trunk/third_party/android_tools/ndk//toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86_64/bin/arm-linux-androideabi-g++ -MMD -MF obj/webrtc/modules/video_coding/codecs/h264/webrtc_h264.h264_impl.o.d -DV8_DEPRECATION_WARNINGS -DBLINK_SCALE_FILTERS_AT_RECORD_TIME -D_FILE_OFFSET_BITS=64 -DNO_TCMALLOC -DDISABLE_NACL -DCHROMIUM_BUILD -DUSE_LIBJPEG_TURBO=1 -DENABLE_WEBRTC=1 -DUSE_PROPRIETARY_CODECS -DENABLE_BROWSER_CDMS -DENABLE_CONFIGURATION_POLICY -DDISCARDABLE_MEMORY_ALWAYS_SUPPORTED_NATIVELY -DSYSTEM_NATIVELY_SIGNALS_MEMORY_PRESSURE -DENABLE_EGLIMAGE=1 -DENABLE_AUTOFILL_DIALOG=1 -DCLD_VERSION=1 -DCLD_DATA_FROM_STATIC -DENABLE_PRINTING=1 -DENABLE_MANAGED_USERS=1 -DVIDEO_HOLE=1 -DWEBRTC_RESTRICT_LOGGING -DWEBRTC_MODULE_UTILITY_VIDEO -DWEBRTC_ARCH_ARM -DWEBRTC_ARCH_ARM_V7 -DWEBRTC_ARCH_ARM_NEON -DWEBRTC_POSIX -DWEBRTC_LINUX -DWEBRTC_ANDROID -DWEBRTC_ANDROID_OPENSLES -DUSE_OPENSSL=1 -DUSE_OPENSSL_CERTS=1 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -DANDROID -D__GNU_SOURCE=1 -DUSE_STLPORT=1 -D_STLP_USE_PTR_SPECIALIZATIONS=1 '-DCHROME_BUILD_ID=""' -DHAVE_SYS_UIO_H -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DWTF_USE_DYNAMIC_ANNOTATIONS=1 -D_DEBUG -Igen -I../.. -I/home/adminuser/webrtc/openh264/codec/api/svc -I/home/adminuser/webrtc/openh264/codec/common -I/home/adminuser/webrtc/openh264/codec/encoder/core/inc -I/home/adminuser/webrtc/openh264/codec/encoder/plus/inc -I/home/adminuser/webrtc/openh264/codec/processing/interface -I../../webrtc/modules/video_coding/codecs/h264/include -I../../webrtc/common_video/interface -I../../webrtc/modules/video_coding/codecs/interface -I../../webrtc/modules/interface -I../../webrtc/common_video/interface -I../../webrtc/common_video/libyuv/include -I../../webrtc/system_wrappers/interface -fstack-protector --param=ssp-buffer-size=4 -Werror -fno-exceptions -fno-strict-aliasing -Wall -Wno-unused-parameter -Wno-missing-field-initializers -fvisibility=hidden -pipe -fPIC -Wno-unused-local-typedefs -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wno-strict-overflow -fno-builtin-cos -fno-builtin-sin -fno-builtin-cosf -fno-builtin-sinf -march=armv7-a -mtune=generic-armv7-a -mfpu=neon -mfloat-abi=softfp -mthumb -fno-tree-sra -fno-partial-inlining -fno-early-inlining -fno-tree-copy-prop -fno-tree-loop-optimize -fno-move-loop-invariants -fno-caller-saves -Wno-psabi -mthumb-interwork -ffunction-sections -funwind-tables -g -fstack-protector -fno-short-enums -finline-limit=64 -Wa,--noexecstack --sysroot=/home/adminuser/webrtc/trunk/third_party/android_tools/ndk//platforms/android-14/arch-arm -I/home/adminuser/webrtc/trunk/third_party/android_tools/ndk//sources/cxx-stl/stlport/stlport -Os -g -fomit-frame-pointer -fdata-sections -ffunction-sections -funwind-tables -fno-rtti -fno-threadsafe-statics -fvisibility-inlines-hidden -Wsign-compare -Wnon-virtual-dtor -Woverloaded-virtual -Wno-abi -std=gnu++11 -Wno-narrowing -Wno-literal-suffix  -c ../../webrtc/modules/video_coding/codecs/h264/h264_impl.cc -o obj/webrtc/modules/video_coding/codecs/h264/webrtc_h264.h264_impl.o

../../webrtc/modules/video_coding/codecs/h264/h264_impl.cc: In member function 'virtual int webrtc::H264EncoderImpl::Encode(const webrtc::I420VideoFrame&, const webrtc::CodecSpecificInfo*, const std::vector<webrtc::VideoFrameType>*)':

../../webrtc/modules/video_coding/codecs/h264/h264_impl.cc:230:38: error: 'const struct SLayerBSInfo' has no member named 'iNalLengthInByte'

         layer_size += layer_bs_info->iNalLengthInByte[nal_index];

                                      ^

../../webrtc/modules/video_coding/codecs/h264/h264_impl.cc:235:58: error: 'const struct SLayerBSInfo' has no member named 'iNalLengthInByte'

         encoded_image_._length          = layer_bs_info->iNalLengthInByte[nal_index] - 4;

                                                          ^

../../webrtc/modules/video_coding/codecs/h264/h264_impl.cc: In member function 'virtual int webrtc::H264DecoderImpl::Decode(const webrtc::EncodedImage&, bool, const webrtc::RTPFragmentationHeader*, const webrtc::CodecSpecificInfo*, int64_t)':

../../webrtc/modules/video_coding/codecs/h264/h264_impl.cc:427:109: error: invalid conversion from 'void**' to 'unsigned char**' [-fpermissive]

   DECODING_STATE rv = decoder_->DecodeFrame2(buffer_with_start_code_, encoded_image_size, data, &buffer_info);

                                                                                                             ^

In file included from ../../webrtc/modules/video_coding/codecs/h264/h264_impl.h:17:0,

                 from ../../webrtc/modules/video_coding/codecs/h264/h264_impl.cc:14:

/home/adminuser/webrtc/openh264/codec/api/svc/codec_api.h:110:33: error:   initializing argument 3 of 'virtual DECODING_STATE ISVCDecoder::DecodeFrame2(const unsigned char*, int, unsigned char**, SBufferInfo*)' [-fpermissive]

   virtual DECODING_STATE EXTAPI DecodeFrame2 (const unsigned char* pSrc,

                                 ^

[1932/2297] CC obj/webrtc/modules/audio_coding/codecs/isac/fix/source/iSACFix.decode.o

ninja: build stopped: subcommand failed.

Op woensdag 30 april 2014 02:18:02 UTC+2 schreef kaiduan:

[kapil kumar]

Hi

I also facing this error while building on Linux, i checkout and build master branch for OpenH264.

When i checked , following member variables are not there in structure.

Did u get through this ? may be need to checkout some other branch ???

[Glenn Vlekke]

@kaiduan

Can you give us a approximately release date when the new patch is ready with the RegisterExternalEncoder?

Thanks for your work!

Op woensdag 19 maart 2014 20:46:33 UTC+1 schreef kaiduan:

[kapil kumar]

And what about SDP negotiation support for other codecs.

In case H264 is not supported by other browser let say, then ideally it should switch back to VP8 ?

If i try to use H264 as default codec, and connect with browser which supprt only VP.

In that case i will get connection error, so how to switch to VP8 in that case ?

Thanks,

[Phil Yu]

Hi Kapil kumar

I modify Kaiduan libjingle.patch

const WebRtcVideoEngine::VideoCodecPref

    WebRtcVideoEngine::kVideoCodecPrefs[] = {

    //{kVp8PayloadName, 100, -1, 0},

    {kH264PayloadName, 101, -1, 0},

    {kVp8PayloadName, 100, -1, 1},

    {kRedPayloadName, 116, -1, 2},

    {kFecPayloadName, 117, -1, 3},

    {kRtxCodecName, 96, 100, 4},

};

kapil kumar於 2014年7月24日星期四UTC+8下午3時50分21秒寫道：

[KaPiL.rIcKy]

Hi Phil,

Even i tried this yesterday and it worked for me :)

It will use VP8 in case other client doesnt support H264, else use H264 if both ends supports H264.

But i wonder why Kaidun commented this :o

Any specific reason for this ?

Br,

Kapil

Sorry i forgot to update.

--

---
You received this message because you are subscribed to a topic in the Google Groups "discuss-webrtc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/discuss-webrtc/U-y3or-dBOU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to discuss-webrt...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

~~~~~~~~~~~~~~~~~~~~~~~~~~
Thanks and regards
Kapil Kumar
~~~~~~~~~~~~~~~~~~~~~~~~~~

[xfengtes...@gmail.com]

Dear Kaiduan,

Well done. The patch is clear.

I have a answer about webrtc with H.264 as below, if you have time, please help provide your suggestion or patch, thanks.

I have a USB camera which can provide H.264 codec frame one by one, in this case, I need provide each H.264 frame to webrtc for streaming out. How to do this?

Thanks.

在 2014年3月20日星期四UTC+8上午3时46分33秒，kaiduan写道：

[Harout Bulbulian]

what other encode does webrtc use then?

[xfengtes...@gmail.com]

The other encode also will be H.264. I have supported H.264 encode/decode in WebRTC native API according your provided patch.

just one side of webrtc will use USB H.264 Camera which will provide H.264 frame directly for streaming.

Thanks.

在 2014年8月18日星期一UTC+8上午2时56分25秒，Harout Bulbulian写道：

[xfengtes...@gmail.com]

The other encode also will be H.264. I have supported H.264 encode/decode in WebRTC native API according your provided patch, and streaming H.264 between two sides is okey.

So now, I want to update or extend video engine/capture of webrtc to capture and streaming the H.264 frame directly from USB H.264 Camera.

Thanks.

在 2014年8月18日星期一UTC+8上午2时56分25秒，Harout Bulbulian写道：

what other encode does webrtc use then?

[Роман Саврулин]

That's the great job you are working on! Can you, please, provide the patch you've done to be able to encode/decode h264?

вторник, 19 августа 2014 г., 5:56:36 UTC+4 пользователь xfengtes...@gmail.com написал:

[xfengtes...@gmail.com]

You can read detail about this topic and get the patchs, copy as below for your reference.

   ***https://webrtc-codereview.appspot.com/11239004/ (Adding H264 processing to webrtc video engine)

在 2014年8月19日星期二UTC+8下午5时24分37秒，Роман Саврулин写道：

[David C]

Just a quick heads up that the patch in 11239004 is a bit outdated and requires some modification to be used in current code base. 

[Hsu France]

Yes, patch 11239004 is 4 months ago, and the trunk of webrtc is very different now.

A little difference is the modification of openh264 API and classes.

Currently, I'm working on the issue:
peerconnection_client: webrtc/modules/rtp_rtcp/source/rtp_format_h264.cc:109: virtual void webrtc::RtpPacketizerH264::SetPayloadData(const uint8_t *, size_t, const webrtc::RTPFragmentationHeader *): Assertion `fragmentation' failed.

Because H264EncoderImpl::Encode sets the RTPFragmentationHeader pointer to be NULL.


David C於 2014年8月22日星期五UTC+8上午5時27分21秒寫道：

[David C]

Are there any plans of updating the patches for the current code base? 

(Or is there a new one just I haven't found out yet...)

[Hsu France]

I've tried provide RTPFragmentationHeader for encoded_complete_callback_->Encoded:
frag_info.fragmentationOffset[0] = 0;
frag_info.fragmentationLength[0] = encoded_image_._length;
frag_info.fragmentationPlType[0] = nal_type; //0;
frag_info.fragmentationTimeDiff[0] = 0;

The H264 video can be send to the other peer.
But there is still a decoding problem.
DecodeFrame2 will failed among these error code: 0x02, 0x04, 0x20, 0x24, 0x26
Also see:
[OpenH264] Error:PPS id is invalid!

The encoded video is correct, we checked this by file saving.
We guess it is because RTP fragmentation/de-fragmentation.

rtp_format_h264.{cc, h} in the h264 patch and in the current trunk have very large difference, and we don't know how to patch the part.
So we use the trunk's version in out code.
Or should we use the version of the h264 patch?

Would you provide some suggestions?
Thanks very much.

France

Hsu France於 2014年8月22日星期五UTC+8上午11時11分28秒寫道：

[xfengtes...@gmail.com]

I didn't see your issue case, but another issue exist in my test case, and add the description into the patch issue tracing.  

    https://webrtc-codereview.appspot.com/11239004/

在 2014年8月23日星期六UTC+8上午1时09分26秒，Hsu France写道：

[tech74]

Hi

I have come across this problem as well, did you manage to overcome this? Encode is working

Decode failing on  DecodeFrame2

Regards

[Gary YU]

I also see this build error:

../../webrtc/modules/video_coding/codecs/h264/h264_impl.cc: In member function 'virtual int webrtc::H264EncoderImpl::Encode(const webrtc::I420VideoFrame&, const webrtc::CodecSpecificInfo*, const std::vector<webrtc::VideoFrameType>*)':

../../webrtc/modules/video_coding/codecs/h264/h264_impl.cc:224:38: error: 'const struct SLayerBSInfo' has no member named 'iNalLengthInByte'

I find the root cause is the file "h264_impl.cc" was written for an old openh264 version but Kaiduan didn't tell us which exact version he was using at that time.

What we can do is to modify this "h264_impl.cc" to make it feasible for the latest openh264. Hope somebody will do this.

[KaPiL.rIcKy]

FYI: (from openh264 support)

The patch on Google codereview  needs some updates to build against the latest OpenH264 which is a very active project.

SLayerBSInfo.iNalLengthInByte

has changed to

SLayerBSInfo->pNalLengthInByte

--

---
You received this message because you are subscribed to a topic in the Google Groups "discuss-webrtc" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/discuss-webrtc/U-y3or-dBOU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to discuss-webrt...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Gary YU]

Sure, the openh264 is indeed a very active project! a lot of things has been changed.

Here, based on the latest OpenH264 version (I refer to Nov. 18, 2014 revision id: ...d71e in master branch), I get an updated version for h264_impl.cc file and build OK now. But still need work on the real encode/decode testing.

I upload the updated file here. And here is the diff from the original Kaiduan's version:

56c56
<     WelsDestroySVCEncoder(encoder_);
---
>     DestroySVCEncoder(encoder_);
106c106
< ret_val = WelsCreateSVCEncoder(&encoder_);
---
> ret_val = CreateSVCEncoder(&encoder_);
121,123c121
<   //param.iInputCsp = videoFormatI420;
<   param.iUsageType = CAMERA_VIDEO_REAL_TIME;
<   param.iRCMode = RC_QUALITY_MODE;
---
>   param.iInputCsp = videoFormatI420;
130c128
<     WelsDestroySVCEncoder(encoder_);
---
>     DestroySVCEncoder(encoder_);
226c224
<         layer_size += layer_bs_info->pNalLengthInByte[nal_index];
---

>         layer_size += layer_bs_info->iNalLengthInByte[nal_index];

231c229
<         encoded_image_._length          = layer_bs_info->pNalLengthInByte[nal_index] - 4;
---

>         encoded_image_._length          = layer_bs_info->iNalLengthInByte[nal_index] - 4;

314c312
<     ret_val = WelsCreateDecoder(&decoder_);
---
>     ret_val = CreateDecoder(&decoder_);
322c320
<   dec_param.eOutputColorFormat = videoFormatI420;
---
>   dec_param.iOutputColorFormat = videoFormatI420;
324c322
<   dec_param.eEcActiveIdc = ERROR_CON_FRAME_COPY;
---
>   dec_param.uiEcActiveFlag = 1;
329c327
<     WelsDestroyDecoder(decoder_);
---
>     DestroyDecoder(decoder_);
423c421
<   DECODING_STATE rv = decoder_->DecodeFrame2(buffer_with_start_code_, encoded_image_size, (unsigned char**)data, &buffer_info);
---

>   DECODING_STATE rv = decoder_->DecodeFrame2(buffer_with_start_code_, encoded_image_size, data, &buffer_info);

466c464
<     WelsDestroyDecoder(decoder_);
---
>     DestroyDecoder(decoder_);

[tech74]

Hi 

But send video bitrate on iOS anyway is very low, there is issue with encoding.

Decoding works OK, anyone noticed this issue. I think someone mentioned this 

before but doesn't seem to have been resolved.

Thanks

[raulf tang]

Hi,

It seems you have overcomed the Decode failing on  DecodeFrame2.

I stiil come across Decode failing with error 4.

Could you please give any suggests?

Thanks

[Hsu France]

In our experience, you need to make sure that NALU parsing and NALU header filling are correct in encode.

Since the RTP part of WebRTC is changed, we rewrite the code about NALU parsing.

[raulf tang]

Thanks,

I'll check it.

[Joseph Ross]

Does anyone have a recent version of this patch that will apply more cleanly to a current/recent sync of the WebRTC code?  

Is support for h.264 on the roadmap in light of the recent decision to require both VP8 and h.264 support for WebRTC endpoints?  http://andreasgal.com/2014/11/16/vp8-and-h-264-to-both-become-mandatory-for-webrtc/

[KaPiL.rIcKy]

As i saw in thirdparty/webrtc module of chrome38, it has support for h264 (although bit diff than this patch)
Libjingle didnt have h264 as prefered codec though.
So even i am curious to know the same :)

Sent from Samsung Galaxy S2

[hejin...@163.com]

Dear kaiduan:
        I don't know if you foucs on OpenH264 from CISCO in webrtc?  encounter question,  can you help me?

        My envirnment: ubuntu 14.  wetrtc november 2014 download from google.
     
        When compliler openH264, and modify h264.gpy indirect to openh264 dir
  
      
        note: openH264 generate serveral static lib

        operate command: gclient runhooks
        ninja -C out/Debug

        compile error
     
        Though I entery openH264 static lib into h264.gyp, familarly, compile error.

        do recent patch??

         jin he    



在 2014年4月28日星期一UTC+8下午5时46分04秒，Enda Mannion写道：

I got this patch to work with the latest Chromium.   I attached the files I needed to change.  One note,  to improve video quality I had to increase the bitrate.  I did this in h264_impl.cc.

param.iTargetBitrate = 128000;

Works really well for me now, I test using apprtc.

https://apprtc.appspot.com/?r=87227771&debug=loopback


-Enda


On Monday, April 28, 2014 9:45:52 AM UTC+1, Phil Yu wrote:

Hi Kaiduan

Could you porting this (H264 on webrtc) on newest chrome release

thanks 

[Min-Soo Koo]

I think supporting H.264 camera is great. Can you kindly let me know about your patch on supporting H264 USB camera?

2014년 8월 19일 화요일 오전 10시 56분 36초 UTC+9, xfengtes...@gmail.com 님의 말:

[周波]

"H264_impl.cc" realization of the old,

bug is "int H264EncoderImpl :: Encode" method

"Encoded_complete_callback _-> Encoded (encoded_image_, NULL, NULL);" is not NULL, otherwise back problems, the new version of the achieved?

在 2014年11月22日星期六 UTC+8下午10:46:21，Gary YU写道：

[周波]

Whether implemented by the new H264?

在 2014年3月20日星期四 UTC+8上午3:46:33，kaiduan写道：

[周波]

New webrtc program, integrated H264 procedural errors, does anyone integration, and sharing what

在 2014年11月22日星期六 UTC+8下午10:46:21，Gary YU写道：