How to input data (YUV420 Planar) into the libyuv::NV12ToI420Rotate(...) function?

[yihungbakj hung]

Hi All

I'm trying to use libyuv::NV12ToI420Rotate to do it, but it's not working.

1. How to input data (YUV420 Planar) into the libyuv::NV12ToI420Rotate().

2. What is crop_x /crop_y and how to use it?

crop_x=?

crop_y=?

crop_width=?

crop_heigh=?

Thank you.

Here is my code.

bool QueueVideoCapturer::CaptureCustomFrame(const shared_ptr<CaptureFrame>& cf) {

if (!running_) {

return false;

}

int width = GetCaptureFormat()->width;

int height = GetCaptureFormat()->height;

int fourcc = GetCaptureFormat()->fourcc;

if(cf) {

if(start_rtc_timestamp==-1) {

start_rtc_timestamp = cf->rtc_timestamp;

}

// Currently, |fourcc| is always I420 or ARGB.

// TODO(fbarchard): Extend SizeOf to take fourcc.

uint32_t size = 0u;

if (fourcc == cricket::FOURCC_ARGB) {

size = width * 4 * height;

} else if (fourcc == cricket::FOURCC_I420) {

size = FrameSizeOf(width, height);

} else {

return false; // Unsupported FOURCC.

}

if (size == 0u) {

return false; // Width and/or Height were zero.

}

rtc::scoped_refptr<webrtc::I420Buffer> buffer(webrtc::I420Buffer::Create(cf->width, cf->height));

int crop_x;

int crop_y;

int crop_width;

int crop_height;

const uint8_t* y_plane = (const uint8_t*)(cf->frame.get_data());//(YUV420 Planar) 

const uint8_t* uv_plane = y_plane + width * height;

int uv_width = (width + 1) / 2;

crop_x &= ~1;

crop_y &= ~1;

libyuv::NV12ToI420Rotate(

      y_plane + (width * crop_y + crop_x), width,

      uv_plane + uv_width * crop_y + crop_x, width, buffer->MutableDataY(),

      buffer->StrideY(),

      // Swap U and V, since we have NV21, not NV12.

      buffer->MutableDataV(), buffer->StrideV(), buffer->MutableDataU(),

      buffer->StrideU(), crop_width, crop_height,

      static_cast<libyuv::RotationMode>(0));

this->OnFrame(webrtc::VideoFrame(buffer, webrtc::kVideoRotation_0, cf->rtc_timestamp * 1000), cf->width, cf->height);

}

return true;

}

[Niels Moller]

On Tue, Apr 17, 2018 at 11:40 AM, yihungbakj hung <yihun...@gmail.com> wrote:
> Hi All
>
> I'm trying to use libyuv::NV12ToI420Rotate to do it, but it's not working.
>
> 1. How to input data (YUV420 Planar) into the libyuv::NV12ToI420Rotate().

If your input data is in YUV420 format already, you shouldn't use the
libyuv::NV12ToI420Rotate conversion function. Use libyuv::I420Copy
instead. (Or you could avoid the copying by defining a class
inheriting I420BufferInterface, and returning pointers to the original
data. But I'd recommend starting with copying into an I420FrameBuffer,
and optimizing later).

> 2. What is crop_x /crop_y and how to use it?
>
> crop_x=?
> crop_y=?
> crop_width=?
> crop_heigh=?

These define a sub-rectangle of the input to be converted. Used, e.g.,
if you need to crop the input frames to get a different aspect ratio.

Regards,
/Niels

[yihungbakj hung]

My data is I420P.  How to pass data into a webrtc::I420Buffer using the libyuv::I420Copy?

Thank you.

Niels Moller於 2018年4月17日星期二 UTC+8下午6時01分48秒寫道：

[Niels Moller]

On Wed, Apr 18, 2018 at 8:15 AM, yihungbakj hung <yihun...@gmail.com> wrote:
> My data is I420P. How to pass data into a webrtc::I420Buffer using the
> libyuv::I420Copy?

By i420P, I guess you mean separate planes for Y, U and V? With the
memory layout described at
https://en.wikipedia.org/wiki/YUV#Y%E2%80%B2UV420p_(and_Y%E2%80%B2V12_or_YV12)_to_RGB888_conversion.

That's the same format as is used by I420Buffer (except that it also
supports larger strides, e.g., to improve alignment). And it seems you
don't even have to call libyuv explicitly. You need the plane pointers
and strides for you input data, and then call the static method

static rtc::scoped_refptr<I420Buffer> Copy(
int width, int height,
const uint8_t* data_y, int stride_y,
const uint8_t* data_u, int stride_u,
const uint8_t* data_v, int stride_v);

(api/video/vi420_buffer.h). If you look at the implementation of this
method, you can see that it just creates an I420Buffer of the given
size, calls MutableData* and Stride* to get access to the planes, and
then calls libyuv::I420Copy to do the actual copying of the pixel
data,

Regards,
/Niels

[yihungbakj hung]

Thank you.

I cant understand algorithms and data structures. How to calculate the stride_y u v?

For example:

uint8_t* yuvData = reinterpret_cast<uint8_t*>(cf->frame.get_data());

int iSizeY = width * height;

int iSizeUV = (width * height)/4;

uint8_t* pY = yuvData;

uint8_t* pU = yuvData + iSizeY

uint8_t* pV = pU + iSizeUV;

rtc::scoped_refptr<webrtc::I420Buffer> buffer = webrtc::I420Buffer::Copy(

width, height,

        pY, ?,

        pU, ?,

        pV, ?);

Thank you.

yh

Niels Moller於 2018年4月18日星期三 UTC+8下午2時41分44秒寫道：

[Niels Moller]

On Wed, Apr 18, 2018 at 9:52 AM, yihungbakj hung <yihun...@gmail.com> wrote:
> Thank you.
>
> I cant understand algorithms and data structures. How to calculate the
> stride_y u v?

The stride is the distance in memory from the start of one line of
pixels to the start of the next line. I would guess you don't have any
padding, in this case, stride_y should be the same as width, and
stride_u and stride_v are both (width + 1)/2, note they should be
rounded upwards if you want to support odd width.

> For example:
>
> uint8_t* yuvData = reinterpret_cast<uint8_t*>(cf->frame.get_data());
> int iSizeY = width * height;
> int iSizeUV = (width * height)/4;

Note that this expression for the uv size is correct only in the
common case that both width and height are even. In the general case,
it's ((width+1)/2) * ((height+1)/2).

> uint8_t* pY = yuvData;
> uint8_t* pU = yuvData + iSizeY
> uint8_t* pV = pU + iSizeUV;
>
> rtc::scoped_refptr<webrtc::I420Buffer> buffer = webrtc::I420Buffer::Copy(
> width, height,
> pY, ?,
> pU, ?,
> pV, ?);

Assuming even width, it should be

... = webrtc::I420Buffer::Copy(
width, height,
pY, width,
pU, width/2,
pV, width/2);

Regards,
/Niels

[yihungbakj hung]

Update. Is it correct?

int half_width = (width + 1) / 2;

int half_height = (height + 1) / 2;

int fullProduct = width*height;

int halfProduct = half_width*half_height;

uint8_t* yuvData = reinterpret_cast<uint8_t*>(cf->frame.get_data());

uint8_t* pY = yuvData;

uint8_t* pU = yuvData + fullProduct;

uint8_t* pV = pU + halfProduct;

rtc::scoped_refptr<webrtc::I420Buffer> buffer = webrtc::I420Buffer::Copy(

width, height,

pY, width,

            pU, half_width,

            pV, half_width);

yihungbakj hung於 2018年4月18日星期三 UTC+8下午3時52分49秒寫道：

[yihungbakj hung]

Hi Niels

I get the sound but a green screen and no video. Why?

Is this code correct?

int width = cf->width;

int height = cf->height;

uint8_t* yuvData = reinterpret_cast<uint8_t*>(cf->frame.get_data());

int iSizeY = width * height;

int iSizeUV = ((width+1)/2) * ((height+1)/2);

uint8_t* pY = yuvData;

uint8_t* pU = yuvData + iSizeY;

uint8_t* pV = pU + iSizeUV;

rtc::scoped_refptr<webrtc::I420Buffer> buffer = webrtc::I420Buffer::Copy(

        width, height,

        pY, width,

        pU, width/2,

        pV, width/2);

Niels Moller於 2018年4月18日星期三 UTC+8下午4時03分32秒寫道：

[Niels Moller]

On Thu, Apr 19, 2018 at 4:56 AM, yihungbakj hung <yihun...@gmail.com> wrote:
> Hi Niels
>
> I get the sound but a green screen and no video. Why?
> Is this code correct?

This code fragment looks ok to me. You should still have (width+1)/2
for the stride arguments, and I'd prefer to use const uint8_t * for
your pointers, but that shouldn't break video.

I'm afraid you have to do a little debugging. By "green screen", does
that mean that you get frames through, and of the right size, just
contents all-green?

Regards,
/Niels Möller

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "discuss-webrtc" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to discuss-webrt...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/discuss-webrtc/8fe048ce-e5a3-4931-9538-9b88ccab3b65%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

[yihungbakj hung]

I forgot to input the yuv data before sending video frame.  The bug is fixed.  It seems to be working fine.  Thanks a lot.

I have another question I would like to ask. The frame rate is 10fps (vp9). How to change the frame rate of video?

Thank you.

yh

Niels Moller於 2018年4月19日星期四 UTC+8下午2時52分44秒寫道：

[yihungbakj hung]

Hi Niels

I have some questions I would like to ask.

1. How to increase FPS (4k VP9) in Webrtc?

2.How to Enable Hardware Encoding/Decoding (VP9) in webrtc?

Thank you.

yh

yihungbakj hung於 2018年4月20日星期五 UTC+8下午4時57分03秒寫道：