Problem with VideoFrame::StretchToPlanes

[yihungbakj hung]

Dear All

The VideoFrame::StretchToPlanes was deleted by administrator. Could I have a function instead of VideoFrame::StretchToPlanes? 

Thank you.

yh

[Christoffer Jansson]

Hi,

Could you clarify a little what you mean?

/Chris

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/5f1259ef-2068-43b7-8e67-b45707573d3d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

/Chris

[yihungbakj hung]

I use the StretchToPlanes to scale down video size.

The StretchToPlanes function was removed in version M54.

Do you have any ideas? Thanks.

For example:

cricket::WebRtcVideoFrame* frame;
frame.InitToBlack(w, h, 0);
int halfW = (w+1)/2, halfH = (h+1)/2;
uint8_t *y = (uint8_t *)b.get_data();
uint8_t *u = y + w*h;
uint8_t *v = u + halfW*halfH;
frame.StretchToPlanes(y, u, v, w, halfW, halfW, w, h, true, true);


yihungbakj hung於 2016年10月17日星期一 UTC+8上午12時48分05秒寫道：

[Niels Moller]

Intended replacement function is I420Buffer::ScaleFrom.

First create a new I420Buffer of the desired target size, then call ScaleFrome with the VideoFrameBuffer you want to scale. There's also CropAndScaleFrom, useful if cropping is needed to preserve correct aspect ratio.

If you have raw yuv data rather than a VideoFrameBuffer object, you can use WrappedI420Buffer as the input frame buffer, or call the the lower-level function libyuv::I420Scale directly.

Regards,

/Niels

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrtc+unsubscribe@googlegroups.com.

[yihungbakj hung]

HI Niels

Could you give me a example on how to use a CropAndScaleFrom instead of StretchToPlanes.

Thank you.

Niels Moller於 2016年10月18日星期二 UTC+8下午5時49分41秒寫道：

Intended replacement function is I420Buffer::ScaleFrom.

First create a new I420Buffer of the desired target size, then call ScaleFrome with the VideoFrameBuffer you want to scale. There's also CropAndScaleFrom, useful if cropping is needed to preserve correct aspect ratio.

If you have raw yuv data rather than a VideoFrameBuffer object, you can use WrappedI420Buffer as the input frame buffer, or call the the lower-level function libyuv::I420Scale directly.

Regards,

/Niels

On Sun, Oct 16, 2016 at 5:12 PM, yihungbakj hung <yihun...@gmail.com> wrote:

Dear All

The VideoFrame::StretchToPlanes was deleted by administrator. Could I have a function instead of VideoFrame::StretchToPlanes? 

Thank you.

yh

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.

To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

[Niels Moller]

On Fri, Oct 21, 2016 at 5:06 AM, yihungbakj hung <yihun...@gmail.com> wrote:

Could you give me a example on how to use a CropAndScaleFrom instead of StretchToPlanes.

Here's an example from the current webrtc code:

  rtc::scoped_refptr<I420Buffer> scaled_buffer(

      buffer_pool_.CreateBuffer(target_width_, target_height_));

  scaled_buffer->CropAndScaleFrom(inFrame.video_frame_buffer());

(https://cs.chromium.org/chromium/src/third_party/webrtc/modules/video_processing/spatial_resampler.cc?q=CropAndScale&sq=package:chromium&l=59&dr=C)

And here is a cl which converted WebRtcVideoFrameFactory (a class that has since been deleted) from StretchToFrame to CropAndScaleFrom:

https://codereview.webrtc.org/2020593002/diff/280001/webrtc/media/base/videoframefactory.cc

All these use an I420Buffer as the destination of the operation. If you really need raw plane pointers for the *destination* (like the StretchToPlanes function used to do), I think you need to call libyuv::I420Scale directly. See the implementation of CropAndScaleFrom for an example,

https://cs.chromium.org/chromium/src/third_party/webrtc/common_video/video_frame_buffer.cc?sq=package:chromium&dr=C&rcl=1477006656&l=153

Regards,

/Niels

[yihungbakj hung]

I have another question I would like to ask. How to input parameters in WrappedI420Buffer()?

I want to use the WrappedI420Buffer function but I don't know how to use it.

Here is my code.

rtc::scoped_refptr<VideoFrameBuffer> buf = video_frame->video_frame_buffer();

rtc::scoped_refptr<webrtc::VideoFrameBuffer> vfbuf(new rtc::RefCountedObject<webrtc::WrappedI420Buffer>(

                                                               w, h,

                                                               y, buf->StrideY(),

                                                               u, buf-->StrideU(),

                                                               v, buf-->SrideV(),

                                                               ? )));

Niels Moller於 2016年10月21日星期五 UTC+8下午2時53分27秒寫道：

[Niels Moller]

On Fri, Oct 21, 2016 at 10:54 AM, yihungbakj hung <yihun...@gmail.com> wrote:

I have another question I would like to ask. How to input parameters in WrappedI420Buffer()?

I want to use the WrappedI420Buffer function but I don't know how to use it.

Here is my code.

rtc::scoped_refptr<VideoFrameBuffer> buf = video_frame->video_frame_buffer();

rtc::scoped_refptr<webrtc::VideoFrameBuffer> vfbuf(new rtc::RefCountedObject<webrtc::WrappedI420Buffer>(

                                                               w, h,

                                                               y, buf->StrideY(),

                                                               u, buf-->StrideU(),

                                                               v, buf-->SrideV(),

                                                               ? )));

The final argument is a callback which is invoked when the

(refcounted) WrappedI420Buffer is deleted, typically used to

deallocate underlying storage.

You don't say what the y, u and v arguments in your code come from.

If you were using buf->DataY() and friends, then you would use

KeepRefUntilDone(buf) for the last argument (declared in

base/keep_ref_until_done.h). Otherwise, I think one typically

constructs the callback using rtc::Bind(method_pointer, object, method

arguments).

[yihungbakj hung]

Hi Niels

In my opinion, It was not good enough to solve this problem with YUV.

Are there any benefits to removing the StretchToPlanes function in a webrtc project?

Thank you for your help. 

Niels Moller於 2016年10月21日星期五 UTC+8下午5時27分18秒寫道：

[Niels Moller]

On Fri, Oct 21, 2016 at 4:45 PM, yihungbakj hung <yihun...@gmail.com> wrote:

In my opinion, It was not good enough to solve this problem with YUV.

If you can describe the problem you are trying to solve in a little more detail, maybe I can give some guidance. As far as I am aware, libyuv::I420Scale should be powerful enough, if the convenience methods in webrtc aren't suitable for  your use case.

 

Are there any benefits to removing the StretchToPlanes function in a webrtc project?

In webrtc, StretchToPlanes was a helper function used mainly by the VideoFrameFactory abstraction, via one or two additional layers of method calls. VideoFrameFactory was deleted some time ago, in an effort to clean up the interfaces for custom capturers, and the old methods were deleted.

If you look at webrtc/video_frame.h, you see that quite a lot of the methods are deprecated. There's unfortunately no crystal clear line between internal interfaces and publicly advertised interfaces in webrtc.

Regards,

/Niels

[yihungbakj hung]

Hi Niels

On your suggestion I refactored this bit of code as follows. I used CropAndScaleFrom instead of StretchToPlanes.

(I don't understand the correct syntax to call this function). 

Can you check it for me, please?

Thank you.

yh

cricket::VideoFrame* originalFrame;

const cricket::VideoFrame *frame = originalFrame->GetCopyWithRotationApplied();

int64_t rtc_t = frame->GetTimeStamp()/1000;

time64_t t = us_timestamp::now();

lastFrameRtcTime = rtc::TimeMicros();

size_t frameW = frame->width(), frameH = frame->height();

shared_ptr<CaptureFrame> cf, lastFrame;

size_t lastW = 0, lastH = 0;

size_t w = 640, h = 480;

if(w>frameW || h>frameH) { // Never enlarge the frame

w = frameW, h = frameH;

} else { // Adjust ratio

size_t a = frameW*h, b = w*frameH;

if(a > b) {

h = b / frameW;

} else if(b > a) {

w = a / frameH;

}

}

if(w==lastW && h==lastH) {

cf = lastFrame;

} else {

size_t frameSize = sizeOfI420(w,h);

buffer b(frameSize);

int halfW = (w+1)/2, halfH = (h+1)/2;

uint8_t *y = (uint8_t *)b.get_data();

uint8_t *u = y + w*h;

uint8_t *v = u + halfW*halfH;

#if 0

frame->StretchToPlanes(y, u, v, w, halfW, halfW, w, h, true, true);

    #else

    rtc::scoped_refptr<webrtc::VideoFrameBuffer> inFrame(new rtc::RefCountedObject<webrtc::WrappedI420Buffer>(

                                                               frameW, frameH,

                                                               y, 0 ,

                                                               u, 0 ,

                                                               v, 0 ,

                                                               rtc::KeepRefUntilDone(inFrame)));

                   

    webrtc::I420BufferPool buf_pool;

rtc::scoped_refptr<webrtc::I420Buffer> scaled_buffer(buf_pool.CreateBuffer(w, h));

scaled_buffer->CropAndScaleFrom(inFrame);

#endif 

    cf.reset(new CaptureFrame(w,h,TAVARUA_PIX_FMT_YUV420P,b,t,rtc_t));

lastFrame = cf;

lastW = w, lastH = h;

}

Niels Moller於 2016年10月24日星期一 UTC+8下午4時22分14秒寫道：

[Niels Moller]

On Tue, Oct 25, 2016 at 10:36 AM, yihungbakj hung <yihun...@gmail.com> wrote:
> Hi Niels
>
> On your suggestion I refactored this bit of code as follows. I used
> CropAndScaleFrom instead of StretchToPlanes.

> buffer b(frameSize);
> int halfW = (w+1)/2, halfH = (h+1)/2;
> uint8_t *y = (uint8_t *)b.get_data();
> uint8_t *u = y + w*h;
> uint8_t *v = u + halfW*halfH;
>
> #if 0
> frame->StretchToPlanes(y, u, v, w, halfW, halfW, w, h, true, true);
> #else

Since it seems you want to the scaled data into a contiguous buffer
where you have full control over the layout, and you already have code
to avoid aspect-ratio related distortion, I'd suggest using libyuv
directly. I.e.,

#include "libyuv/scale.h"

rtc::scoped_refptr<webrtc::VideoFrameBuffer> srcBuf(
frame->video_frame_buffer());

libyuv::I420Scale(srcBuf->DataY(), srcBuf->StrideY(),
srcBuf->DataU(), srcBuf->StrideU(),
srcBuf->DataV(), srcBuf->StrideV(),
frameW, frameH,
y, w, u, halfW, v, halfW,
w, h, libyuv::kFilterBox);

This should write the scaled image into your buffer b, in the same way
as StretchToPlanes did.

Then I'm afraid you have some additional obstacles to get your code
working with latest webrtc. GetCopyWithRotationApplied was deleted
some month ago, see
https://codereview.chromium.org/2293253003/patch/1/10001 for an
example on how to handle it using webrtc::I420Buffer::Rotate instead,
followed by deprecation of cricket::VideoFrame (now an alias for
webrtc::VideoFrame).


Regards,
/Niels