Half video resolution in Nacl for Chrome 52

[Samson Timoner]

Hi All,

 If I use webrtc to broadcast 1280x720 video from chrome 52 to another computer using chrome 52:  the receiving computer shows half-resolution in pnacl, 640x360. 

 Might anyone know why?

Details:

• I'm using webrtc from chrome to chrome, displaying the results in pnacl/OpenGL. I've been compiling against pepper 44 (because it has been working for me up until now.)
• If I check chrome://webrtc-internals, it says I am sending and receiving a full 1280x720 video.
• If I create a video element, and send the mediatrack into the video element rather than pnacl, the video starts as half-res 640x360 but goes up to full res pretty fast -- maybe as little as one frame.
  
• If I broadcast from chrome 52 to older versions of chrome, say chrome 50 or 49, I get the same problem.
• If I broadcast from older versions of chrome, 50 or 49, to chrome 52, everything works. Everything is full-res and works great

  It seems that something has changed.  Anyone have ideas?  Might VP9 be on by default when broadcasting from chrome 52?  I'm just guessing -- I don't know how to check that.

Thanks!

-- Samson

[Bill Budge]

Can you say more about what your PNaCl plugin does? Is it using the Pepper VideoDecoder API?

[Samson Timoner]

Thanks Bill,

 I based my code on the media_stream_video demo. It takes in a video stream, and puts it in openGL for processing.

 

 // I get the media track as a resource from the dictionary

  pp::Var var_track = dictionary.Get("track");

  if (!var_track.is_resource()){

    return;

  }

  pp::Resource resource_track = var_track.AsResource();

  video_stream_track_ = pp::MediaStreamVideoTrack(resource_track);  

// Configure the track

  const int32_t attrib_list[] = {

    PP_MEDIASTREAMVIDEOTRACK_ATTRIB_FORMAT, PP_VIDEOFRAME_FORMAT_BGRA,

    PP_MEDIASTREAMVIDEOTRACK_ATTRIB_WIDTH, 0,

    PP_MEDIASTREAMVIDEOTRACK_ATTRIB_HEIGHT, 0,

    PP_MEDIASTREAMVIDEOTRACK_ATTRIB_NONE

  };

  this->first_frame_ = true;

  video_stream_track_.Configure(attrib_list, callback_factory_.NewCallback(

      &VideoController::TrackConfigureCallback));

-- Samson

[Samson Timoner]

We managed to limit it a bit more. We have a computer with chrome 51. Using chrome 51 as the broadcaster, everything works.

Thus, we think something changed from 51 to 52.

What's really weird is that if you get the bug depending on the broadcaster, not the receiver. That is, the receiver stays the same, but the broadcaster changes and it makes all the difference.

 The only thing I can think of is that the broadcaster is somehow sending a few low-res frames, and the receiver is getting them, but then not responding when the resolution goes full. 

 Any thoughts on this?

   Thanks!

-- Samson

P.S. I checked the "SetLocalDescription" in chrome://webrtc-internals. m=video 9 UDP/TLS/RTP/SAVPF 100 101 107 116 117 96 97 99 98 in cases that do work and cases that don't work.

[Samson Timoner]

I think we have wrapped our head around the problem, and a short-term work-around.

• Webrtc for chrome 52 with 1280x720 video starts with a half-resolution, and then goes to full resolution. It looks like someone wrote it that way to try to get a first  frame on the screen faster.
  
• It we set the VideoStreamVideoTrack attribute width/height to (0,0), it appears to be the case that the attribute of the frame gets set the first time a full-resolution frame comes in, and then never changes.
• For the new webrtc half-resolution, that's a disaster. That means that we start at half-res, and remain at half-res.
• The work around is to set the resolution manually. That's not very good because I don't really know the resolution.

QUESTION: For the VideoStreamVideoTrack, given that I don't know the resolution of the frame, what is the right thing to do? Is it supposed to adapt over time?

Thanks!

-- Samson

[Bill Budge]

It sounds like webrtc-jingle might be a better place to ask this:

https://groups.google.com/forum/#!forum/discuss-webrtc