How to send two video streams(with different resoltion from on camera) to peer

[Xuegong Bian]

I am writing the code for an application which should be able to send the two video streams(such as one 720P video and one 360P video) from same camera to peer.

Based on the peerconnection_client. Below is my code: I reuse VideoSource to create two VideoTracks.

  rtc::scoped_refptr<webrtc::VideoTrackSourceInterface> video_track_source;

  video_track_source = peer_connection_factory_->CreateVideoSource(OpenVideoCaptureDevice(),NULL);

  rtc::scoped_refptr<webrtc::VideoTrackInterface> video_track(

      peer_connection_factory_->CreateVideoTrack(

          kVideoLabel,  video_track_source));

  

  rtc::scoped_refptr<webrtc::VideoTrackInterface> video_track2(

      peer_connection_factory_->CreateVideoTrack(

          "video_label2", video_track_source));

  

  rtc::scoped_refptr<webrtc::MediaStreamInterface> stream =

      peer_connection_factory_->CreateLocalMediaStream(kStreamLabel);

  stream->AddTrack(audio_track);

  stream->AddTrack(video_track);

  stream->AddTrack(video_track2);

  

But how to specify video_track and video_track2 with different resoltion?

(I also try to reuse VideoCapture object to create two VideoSources, but it does not work.) 

I think another method is using simulcast. From "Javascript Session Establishment Protocol",

"Applications request support for simulcast by configuring multiple encodings on an RTPSender..."

How to do this using Native C++ code? And this method does not support H.264 and VP9 , is it right?

[Eric Davies]

Have you tried two media streams added to the same peer connection, each with one video track?

I've done this in the browser, haven't tried it in native code.

Failing that, I would use two peer connections.

Unless of course, you actually needed the two video streams to be synchronized together.

[Sergey Grigoryev]

Hi!
I think you are not able to use the same source (Capture device) with different resolution.

So the workaround may be in having your own implementation of VideoCapturer. It should use the only instance of OpenVideoCaptureDevice() and scale the frame according to resolution.

Something like that:

YourSpecialCapturer : public VideoCapturer

{

public:

  YourSpecialCapturer (VideoCapturer *capturer);
  .... OnFrame()
  {
   ...
  CloneFrame();
  CropAccordingToResolution();
   ...
  }

}

Best Regards,
Sergey

понедельник, 7 ноября 2016 г., 10:52:13 UTC+3 пользователь Xuegong Bian написал:

[Sergey Grigoryev]

Hi Eric,

I've used adding two videoStreams to the same PC and it was working fine.

        stream->AddTrack(audio_track);

        stream->AddTrack(video_track);

        stream->AddTrack(screen_track);

That is how it looks on my side.

Br,
Sergey

понедельник, 7 ноября 2016 г., 18:12:47 UTC+3 пользователь Eric Davies написал:

[Xuegong Bian]

  This is my method to limit the max_bitrate_bps after   call SetLocalDescription(). I think this can limit the video resolution to send. 

  peer_connection_->SetLocalDescription(

      DummySetSessionDescriptionObserver::Create(), desc);

  auto senders = peer_connection_->GetSenders();

  for (const auto& sender : senders) {

      if (sender->media_type() != cricket::MEDIA_TYPE_VIDEO)

          continue;

      webrtc::RtpParameters params = sender->GetParameters();

      if(sender->id()== kVideoLabel)

          params.encodings[0].max_bitrate_bps = 200000;

      else if(sender->id()== "video_label2")

          params.encodings[0].max_bitrate_bps = 100000;

      sender->SetParameters(params);