Issue 341452 in chromium: Change video capture transport colorspace from YUV420 to YV12

[chro...@googlecode.com]

Status: Untriaged
Owner: ----
CC: pe...@chromium.org, mca...@chromium.org
Labels: Type-Bug Pri-2 Cr-Blink-WebRTC OS-All

New issue 341452 by mca...@chromium.org: Change video capture transport
colorspace from YUV420 to YV12
http://code.google.com/p/chromium/issues/detail?id=341452

Version: 248706
OS: All

The current video capture colorspace pipeline features an yuv420 (a.k.a
yu12)
[0] transport colorspace, hardcoded in video_capture_controller.cc (VCC).
The
destination of the video frames (the "sink") is either libjingle+libwebrtc
and
ultimately vp8 encoder, or a local <video> compositing . Vp8 encoder is
happy with yu12 whereas compositing uses yv12 [1]. It so happens that
libjingle needs to copy incoming frames and produce some processing on
them.

The proposal would be to use yv12 as default transport colorspace, and
convert yv12->yuv420 in libjingle for the vp8 case. This would also shift
the colorspace conversion CPU cycles to renderer side, relieving the
browser.

Typical android video capture produces yv12 frames, hence one conversion
less in this platform.

Note that the difference between yuv420 and yv12 is just the ordering of
the chroma planes.

Some current colorspace scenarios in video capture follow:

* Typical video camera capturing yuv422, local compositing
camera[yuv422]->browser, convert to yuv420->renderer-><video>, converts to
yv12

* Normal video camera capturing yuv422, vp8 encoding
camera[yuv422]->browser, convert to yuv420->renderer->libjingle copy frame
-> vp8

* Typical android video camera yv12, local compositing.
camera[yv12]->browser, convert to yuv420->renderer-><video> tag, converts
to yv12 again!

With the proposed change:

* Typical video camera capturing yuv422, local compositing
camera[yuv422]->browser, convert to yv12->renderer-><video>, no conversion
needed

* Normal video camera capturing yuv422, vp8 encoding
camera[yuv422]->browser, convert to yv12->renderer->libjingle convert frame
to yuv420 -> vp8

* Typical android video camera yv12, local compositing.
camera[yv12]->browser, no conversion->renderer-><video> tag, converts to
yv12 !


[0] http://www.fourcc.org/yuv.php#IYUV
[1] http://www.fourcc.org/yuv.php#YV12


--
You received this message because this project is configured to send all
issue notifications to this address.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

[chro...@googlecode.com]

Updates:
Labels: -Type-Bug Type-Feature

Comment #1 on issue 341452 by kjell...@chromium.org: Change video capture

transport colorspace from YUV420 to YV12
http://code.google.com/p/chromium/issues/detail?id=341452

(No comment was entered for this change.)

[chro...@googlecode.com]

Comment #2 on issue 341452 by w...@chromium.org: Change video capture

transport colorspace from YUV420 to YV12
http://code.google.com/p/chromium/issues/detail?id=341452

Is "local compositing" a main use case?

Regarding "color conversion", since YUV420 and YV12 have U and V swapped,
and a video frame structure typically has 3 pointers which point to Y, U
and V, respectively. The "color conversion" between YUV420 and YV12 should
be swapping pointers of U and V, not real pixel data copying.

[chro...@googlecode.com]

Comment #7 on issue 341452 by fisc...@chromium.org: Change video capture

transport colorspace from YUV420 to YV12
http://code.google.com/p/chromium/issues/detail?id=341452

Chatted w/ Per. Sounds like the proposal here isn't about saving cycles or
otherwise improving perf, it's only about removing apparent cruft (b/c why
should a libjingle-agnostic chrome use I420 for a transport when the
rendering pipeline is gonna need YV12 anyway?).

As discussed, this change will need to be tested against getUserMedia
(local render & webrtc encode), ppapi/nacl/pnacl, and chromecast. I'm not
sure the benefit of the decrufting is worth the effort, but if someone
wants to own it then fine...

[chro...@googlecode.com]

Updates:
Owner: pe...@chromium.org

Comment #8 on issue 341452 by mca...@chromium.org: Change video capture

transport colorspace from YUV420 to YV12
http://code.google.com/p/chromium/issues/detail?id=341452

(No comment was entered for this change.)

[chro...@googlecode.com]

Comment #10 on issue 341452 by pe...@chromium.org: Change video capture

transport colorspace from YUV420 to YV12
http://code.google.com/p/chromium/issues/detail?id=341452

I think this solves the issue I was trying to bring up. But from a
different angle, Chrome now support rendering I420 so we can strive for
always using textures or I420 in MediaStreams. Thanks.

Note though, that if the actual cause for submitting r261497 was to be able
to provide a memory layout of tightly packed I420 video frames to a codec,
you might still run into problem in the future if you plant to be able to
receive video frames from other sources than a video capturer.
media::VideoFrame::CreateFrame(I420) align stride, and height to 16byte 32
byte blocks which lead to funny effects in a codec if you use for example
640*360.
In WebRtc we hope to solve this by not requiring tightly packed memory.

[chro...@googlecode.com]

Updates:
Status: Fixed

Comment #11 on issue 341452 by pe...@chromium.org: Change video capture

transport colorspace from YUV420 to YV12
http://code.google.com/p/chromium/issues/detail?id=341452

(No comment was entered for this change.)