how to downscale to S0 layer frame when using VP9 SVC

Vincent Bian

unread,

Feb 26, 2018, 5:04:19 AM2/26/18

to discuss-webrtc

Hi

I want to make a test with VP9 SVC. I use peerconnection client with VP9 SVC (webrtc branch 65) and add test code lines below to drop S1 layer frame, but it can not work.(The peer has no video received.)

(It can work when drop T1 or T2 layer only).

// test

if (codec_specific.codecSpecific.VP9.temporal_idx >= 1)

return WEBRTC_VIDEO_CODEC_OK;

if (codec_specific.codecSpecific.VP9.spatial_idx >= 1)

return WEBRTC_VIDEO_CODEC_OK;

int VP9EncoderImpl::GetEncodedLayerFrame(const vpx_codec_cx_pkt* pkt) {

RTC_DCHECK_EQ(pkt->kind, VPX_CODEC_CX_FRAME_PKT);

if (pkt->data.frame.sz > encoded_image_._size) {

delete[] encoded_image_._buffer;

encoded_image_._size = pkt->data.frame.sz;

encoded_image_._buffer = new uint8_t[encoded_image_._size];

}

memcpy(encoded_image_._buffer, pkt->data.frame.buf, pkt->data.frame.sz);

encoded_image_._length = pkt->data.frame.sz;

// No data partitioning in VP9, so 1 partition only.

int part_idx = 0;

RTPFragmentationHeader frag_info;

frag_info.VerifyAndAllocateFragmentationHeader(1);

frag_info.fragmentationOffset[part_idx] = 0;

frag_info.fragmentationLength[part_idx] = pkt->data.frame.sz;

frag_info.fragmentationPlType[part_idx] = 0;

frag_info.fragmentationTimeDiff[part_idx] = 0;

vpx_svc_layer_id_t layer_id = { 0 };

vpx_codec_control(encoder_, VP9E_GET_SVC_LAYER_ID, &layer_id);

if (is_flexible_mode_ && codec_.mode == kScreensharing)

spatial_layer_->LayerFrameEncoded(

static_cast<unsigned int>(encoded_image_._length),

layer_id.spatial_layer_id);

// End of frame.

// Check if encoded frame is a key frame.

encoded_image_._frameType = kVideoFrameDelta;

if (pkt->data.frame.flags & VPX_FRAME_IS_KEY) {

encoded_image_._frameType = kVideoFrameKey;

}

RTC_DCHECK_LE(encoded_image_._length, encoded_image_._size);

CodecSpecificInfo codec_specific;

PopulateCodecSpecific(&codec_specific, *pkt, input_image_->timestamp());

// test

if (codec_specific.codecSpecific.VP9.temporal_idx >= 1)

return WEBRTC_VIDEO_CODEC_OK;

if (codec_specific.codecSpecific.VP9.spatial_idx >= 1)

return WEBRTC_VIDEO_CODEC_OK;

if (encoded_image_._length > 0) {

TRACE_COUNTER1("webrtc", "EncodedFrameSize", encoded_image_._length);

encoded_image_._timeStamp = input_image_->timestamp();

encoded_image_.capture_time_ms_ = input_image_->render_time_ms();

encoded_image_.rotation_ = input_image_->rotation();

encoded_image_.content_type_ = (codec_.mode == kScreensharing)

? VideoContentType::SCREENSHARE

: VideoContentType::UNSPECIFIED;

encoded_image_._encodedHeight = raw_->d_h;

encoded_image_._encodedWidth = raw_->d_w;

encoded_image_.timing_.flags = TimingFrameFlags::kInvalid;

int qp = -1;

vpx_codec_control(encoder_, VP8E_GET_LAST_QUANTIZER, &qp);

encoded_image_.qp_ = qp;

encoded_complete_callback_->OnEncodedImage(encoded_image_, &codec_specific,

&frag_info);

}

return WEBRTC_VIDEO_CODEC_OK;

}

Sergio Garcia Murillo

unread,

Feb 26, 2018, 5:34:51 AM2/26/18

to discuss...@googlegroups.com

Have you checked this already?

https://webrtchacks.com/chrome-vp9-svc/

Best regards
Sergio

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/b63b12b6-68d9-455f-bf8e-7abbbac140c3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Vincent Bian

unread,

Feb 26, 2018, 9:16:30 PM2/26/18

to discuss-webrtc

Thanks. I have checked this. I want to find a quick method to simulate SFU behaviour dropping some layers using peerconnection client.

Alexandre GOUAILLARD

unread,

Feb 27, 2018, 12:30:59 AM2/27/18

to discuss...@googlegroups.com

Why not using chrome against Janus instead?

blog post:

www.meetecho.com/blog/availability-of-chrome-interoperable-vp9-svc-in-janus-a-meetecho-cosmo-collaboration/

demo:

https://janus.conf.meetecho.com/vp9svctest.html

To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrtc+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/0acecc01-2971-46d5-b683-cbfa470a1ac3%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Alex. Gouaillard, PhD, PhD, MBA

------------------------------------------------------------------------------------

President - CoSMo Software Consulting, Singapore

------------------------------------------------------------------------------------

sg.linkedin.com/agouaillard

Sergio Garcia Murillo

unread,

Feb 27, 2018, 1:25:52 AM2/27/18

to discuss...@googlegroups.com

Even better, you don't need to emulate it.

Use my peerconnection server echo test and use a real Sfu instead

https://github.com/medooze/peerconnection-server

Change the code from vp8 to vp9 and do the layer selection on the transponder object

https://medooze.github.io/media-server-node/#transponder

Best regards

Sergio

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/CAHgZEq4ahYHXymgUXRjNTnu_uGDZc50ES6E9zkxtgJBRnMYbvg%40mail.gmail.com.

Vincent Bian

unread,

Feb 27, 2018, 2:41:08 AM2/27/18

to discuss-webrtc

Thanks. I have downloaded medooze media server source code to try to find the anwser. I do think medooze media server can do the test.

I have a hypothesis to make a SFU based on WebRTC native code.  I find it can work when add the line below. I think this will affect some vp9 svc rtp payload paramater, but not exactly.

if (codec_specific.codecSpecific.VP9.temporal_idx >= 2)

      return WEBRTC_VIDEO_CODEC_OK;

if (codec_specific.codecSpecific.VP9.spatial_idx >= 1)

      return WEBRTC_VIDEO_CODEC_OK;

 codec_specific.codecSpecific.VP9.num_spatial_layers = 1;  // It can work when add this line

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/0acecc01-2971-46d5-b683-cbfa470a1ac3%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
Alex. Gouaillard, PhD, PhD, MBA
------------------------------------------------------------------------------------
President - CoSMo Software Consulting, Singapore
------------------------------------------------------------------------------------
sg.linkedin.com/agouaillard

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

Sergio Garcia Murillo

unread,

Feb 27, 2018, 4:18:56 AM2/27/18

to discuss...@googlegroups.com

On 27/02/2018 8:41, Vincent Bian wrote:

Thanks. I have downloaded medooze media server source code to try to find the anwser. I do think medooze media server can do the test.

I have a hypothesis to make a SFU based on WebRTC native code. I find it can work when add the line below. I think this will affect some vp9 svc rtp payload paramater, but not exactly.

if (codec_specific.codecSpecific.VP9.temporal_idx >= 2)

return WEBRTC_VIDEO_CODEC_OK;

if (codec_specific.codecSpecific.VP9.spatial_idx >= 1)

return WEBRTC_VIDEO_CODEC_OK;

codec_specific.codecSpecific.VP9.num_spatial_layers = 1; // It can work when add this line

IMHO it is not a good idea to create an SFU on top of libwerbtc, anyway, your call ;)

The code you point seems to me that it is related to the encoding/decoding process, so not sure what you are trying to accomplish with it.

Best regards
Sergio

Vincent Bian

unread,

Feb 27, 2018, 4:32:39 AM2/27/18

to discuss-webrtc

What are the disadvantages when create SFU on top of libwebrtc? (I think all features except forward packtes the peerconnection object has done.) Thanks.

Sergio Garcia Murillo

unread,

Feb 27, 2018, 4:38:21 AM2/27/18

to discuss...@googlegroups.com

Complex build system, ton of external dependencies, 80% of the code meant to be used in client side, 100s of commits per week, no internal API stability..

You will spend most of the time catching up with libwertc instead of implementing your SFU.

Regards
Sergio

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/da7164b0-e436-4a48-93ed-84f6532de73d%40googlegroups.com.

Reply all

Reply to author

Forward