Using H264 bitstream from external Encoder (native WebRTC)

[Bernhard Höckner]

Hi!

I'm working on providing encoded frames (as H264 bitstream) to WebRTC running within an native (c++) application on windows.

Currently the encoded bitstream comes from NVENC.

This works so far as described in "Sending pre-encoded h264 bitstream via WebRTC (native code)":

Instead of using the OpenH264 encoder:

 - filling a webrtc::I420Buffer with h264 bitstream

 - find the NAL startcode ("0001"), set offset and length in the fragmentation header

 - find the frame type within the NALU type (the bitstream contains IDR, I, P frames)

 - signal a finished encode picture

The way this is done looks quite hacky and not very stable, so I was wondering if there was a better way I overlooked.

Whats also missing is getting feedback on the available bitrate etc., in order to adjust the encoding on the fly. 

Thus the quality depends on the preconfigured enncoder settings.

As far as i understood the WebRTC library adjusts the (unencoded) input frames resolution in adition to adjusting the encoding bitrate?

By having a way of delievering feedback to the external encoder, this values could be adjusted before encoding.

I saw the pointer to the initial "Issue 500605, Comment 101: H264 HW encode using VideoToolbox".

But going this way would mean to move the NVENC encoder into the WebRTC library?

I would like to just provide the pre-encoded H264 bitstream to the WebRTC library.

I also looked at "Issue 5652: Support for external h264 encoders" which was closed with WontFix. Any idea if this still holds true?

Would a comment there make any sense to start a discussion?

Any comments, ideas? 

Thanks

[Henrik Kjellander]

+hbos

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrtc+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/07606ef9-e3f4-4ff7-aaa5-2a05f29d85f7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Bernhard Höckner]

Hi all,

while looking around on how to provide an pre-encoded (H264) bitstream, i found "Issue 5687: Refactor Encoder - MediaTransport layer to use rtc::VideoSink/Source interface" and within comment 1 it was mentioned this issue was done also to support sources providing encoded frames.

Does this mean there is work ongoing to support external encoders? 

Or does this refer to something different?

Thanks

[Bernhard Höckner]

Hi,

so after some more poking around in order to provide an pre-encoded H264 bitstream, i stumbled upon the possibility to provide an WebRtcVideoEncoderFactory to the PeerConnectionFactory. The EncoderFactory generates encoders with internal source (EncoderTypeHasInternalSource returning true).

This seems to work quite nice without the need for the hack of using an I420Buffer with H264 bitstream within, but the comments in Issue 42349004, Issue 1263663005 seems to suggest this is not meant to stay?

Any comments?

Thanks

[Nicolas Tizon]

Hi,

From my investigations, the easiest way to use an "external" H.264 encoder with the WebRTC native C++ sources is to extend the existing cricket::VideoCapturer class to handle H.264 elementary streams.
By using FFmpeg API in addition, one can feed the capturer with different input formats (RTSP, MPEG files,...).

Here are my latest works on this topic: https://github.com/nicotyze/Webrtc-H264Capturer

[artemiuzzz]

Hi!

How did you implement encoder with internal source? I managed to create custom WebRtcVideoEncoderFactory and custom H264Encoder, but I don't get where to put my encoded frames, where actually encoder with internal_source=true takes data? There is VideoEncoder::Encode method, but it requires webrtc::VideoFrame. I'm going to send fake frames like 1x1, or, maybe, somehow put encoded frame data in webrtc::VideoFrame. Is there a more properly use of such encoder?

[Andriy Buchynskyy]

Hello, I'm looking for the same information. If you have any information I would very appreciate if you share.