Non-composited 3D transform using Skia perspective.

[Tien-Ren Chen]

Hello paint-dev,

I came up with this idea when I tried to write a simplistic renderer
to replay 3D transform display items in layerization mode (for testing
& sanity check). Why can't we paint everything in device coordinate?
Is it something we have thought about in the past but didn't do?
Anyway I think it is a good timing to bring discussion up again
because many of the engineering constraints have changed:

1. With S.P. we no longer have the GraphicsLayer/cc::Layer abstraction
that mandates the compositor to create a rectangular backing in local
coordinate (Not completely true. We still have raster scale. Why not
raster matrix?).

2. With Ganesh, Skia may actually be fast for non-trivial transforms.

There are a few advantages painting in device coordinate:

1. 3D contents don't need to be re-sampled (texture mapping), result
in better render quality.
2. Less memory usage because 3D contents won't need layers, whether
permanent or temporary.
3. Easier to handle unbounded surfaces.

I made a few example to demonstrate the limitation of rendering in
local coordinates: http://jsbin.com/nokeju/1/
All three examples are similar, with an inner layer that has a X-W
skew, flattened, then do an opposite X-W skew on the outer layer. With
CSS properties applied on the outer layer to enforce render surfaces.

On top: Chrome gave up because the outer layer is forced to create a
render surface by opacity:0.99, but the projection of the inner layer
on the outer layer is unbounded on the left side.

Middle: Use overflow:hidden to enforce render surface, thus the
surface is no longer unbounded. Rendered result is correct, but with
terrible quality because the inner layer is re-sampled twice.

On bottom: Quality is good because no intermediate render surface is
created, but near plane clipping is incorrect. The layer should be
clipped at x>=50 because of the intermediate flattening.

Here are a few reasons why we can't do it today that I can think of:

1. Perspective support in Skia is experimental. Here is one bug I
reported today which would be a deal breaker:
https://code.google.com/p/skia/issues/detail?id=3681
I had a quick chat with some Skia folks. If I understand correctly, it
will be a low-priority bug for Skia because none of the major client
is using perspective, but if Chrome decides to use Skia for 3D
rendering, the priority can change. (And I will be happy to contribute
too.)

2. It is unclear how anti-alias should work. FSAA probably not an
option for Chrome.

3. We probably still need layers in local coordinate for animation.

I'm not suggesting to tie this to any phase of S.P. launch, but it
could be a stepping stone for layerization algorithm. Again I came up
with this idea because I want to be able to test the 3D transform
display items generated by Blink, which can't be done without a
layerization algorithm in place. However layerization algorithm too
will be difficult to test without see real data generated by Blink. I
think having a simplistic 3D display list renderer without
layerization would be a good approach to break the engineering chicken
& egg problem. Also it provides a benchmark reference for us to try
and fine tune different layerization triggers. WDYT?

[Dana Jansens]

Software compositing uses 3d software raster when compositing various layers in a 3d scene.

 

2. It is unclear how anti-alias should work. FSAA probably not an
option for Chrome.

There are some TODOs in software_renderer.cc about AA as well, there's no support for AA on a single edge which makes for uglies.

 

3. We probably still need layers in local coordinate for animation.

I'm not suggesting to tie this to any phase of S.P. launch, but it
could be a stepping stone for layerization algorithm. Again I came up
with this idea because I want to be able to test the 3D transform
display items generated by Blink, which can't be done without a
layerization algorithm in place. However layerization algorithm too
will be difficult to test without see real data generated by Blink. I
think having a simplistic 3D display list renderer without
layerization would be a good approach to break the engineering chicken
& egg problem. Also it provides a benchmark reference for us to try
and fine tune different layerization triggers. WDYT?

--
You received this message because you are subscribed to the Google Groups "paint-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to paint-dev+...@chromium.org.
To post to this group, send email to pain...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/paint-dev/CAKxkHO-X5avyzBHfsEWS9SBSj8rzbgr9vkGAmkAdfMjzPiG3gA%40mail.gmail.com.

[Jeremy Roman]

This seems like a direction worth exploring, but since we will definitely need to support the compositor path still, maybe this is something we can investigate after shipping SP phase 2 (to give the compositor more freedom to not layerize cases where Skia can handle it)?

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/paint-dev/CAHtyhaSqLZcNa4rU4S%2BUiR4Oib7ndxHkiDXyQzHS_rgWzQ23uQ%40mail.gmail.com.

[Chris Harrelson]

This is an interesting discussion, but I feel obligated to play the Boring Cop and repeat what Jeremy said. :)

That being said, a clarifying question: do you mean rastering into a single surface, or literally outputting display items with global coordinates?

Chris

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/paint-dev/CACuR13eHTu4e8UyL9pXOtphx1QpmW%3DN2L%2BkfA7Q1JD0zeTufXA%40mail.gmail.com.

[Ian Vollick]

On Thu, Apr 9, 2015 at 11:59 AM Chris Harrelson <chri...@chromium.org> wrote:

This is an interesting discussion, but I feel obligated to play the Boring Cop and repeat what Jeremy said. :)

That being said, a clarifying question: do you mean rastering into a single surface, or literally outputting display items with global coordinates?

(The latter would be a bummer for all compositor-driven effects, which may need to apply hierarchically.) 

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/paint-dev/CAOMQ%2Bw_A%3DFeqv5M65YEETjzKaE9J8nGg0hfR7TAJDeogM33KNA%40mail.gmail.com.

[Adrienne Walker]

2015-04-08 17:14 GMT-07:00 'Tien-Ren Chen' via paint-dev
<pain...@chromium.org>:

> I came up with this idea when I tried to write a simplistic renderer
> to replay 3D transform display items in layerization mode (for testing
> & sanity check). Why can't we paint everything in device coordinate?
> Is it something we have thought about in the past but didn't do?
> Anyway I think it is a good timing to bring discussion up again
> because many of the engineering constraints have changed:

By paint, I assume you mean raster?

I agree that the current layerization code is too aggressive in
creating layers for perspective transforms. I think this is something
that would be easy to take into account when writing the new
layerization algorithm. Playing back a set of partitions of a display
list that happen to include perspective should "just work" today
(modulo bugs that you mention below).

> 1. Perspective support in Skia is experimental. Here is one bug I
> reported today which would be a deal breaker:
> https://code.google.com/p/skia/issues/detail?id=3681
> I had a quick chat with some Skia folks. If I understand correctly, it
> will be a low-priority bug for Skia because none of the major client
> is using perspective, but if Chrome decides to use Skia for 3D
> rendering, the priority can change. (And I will be happy to contribute
> too.)

As danakj says, this bug would appear in Chrome today when using
software rasterization. See the setMatrix code here:
https://code.google.com/p/chromium/codesearch#chromium/src/cc/output/software_renderer.cc&l=263

[Tien-Ren Chen]

On Thu, Apr 9, 2015 at 10:50 AM, Adrienne Walker <en...@chromium.org> wrote:
> 2015-04-08 17:14 GMT-07:00 'Tien-Ren Chen' via paint-dev
> <pain...@chromium.org>:
>> I came up with this idea when I tried to write a simplistic renderer
>> to replay 3D transform display items in layerization mode (for testing
>> & sanity check). Why can't we paint everything in device coordinate?
>> Is it something we have thought about in the past but didn't do?
>> Anyway I think it is a good timing to bring discussion up again
>> because many of the engineering constraints have changed:
>
> By paint, I assume you mean raster?

Yes. I didn't realize paint != raster since the introduction of
impl-side painting. :)

> I agree that the current layerization code is too aggressive in
> creating layers for perspective transforms. I think this is something
> that would be easy to take into account when writing the new
> layerization algorithm. Playing back a set of partitions of a display
> list that happen to include perspective should "just work" today
> (modulo bugs that you mention below).

Yes, except in addition to what we do today (polygon splitting), we
also need to take care of near plane clipping (w>0) for flattening,
which was taken care of by the intermediate render surfaces.

>> 1. Perspective support in Skia is experimental. Here is one bug I
>> reported today which would be a deal breaker:
>> https://code.google.com/p/skia/issues/detail?id=3681
>> I had a quick chat with some Skia folks. If I understand correctly, it
>> will be a low-priority bug for Skia because none of the major client
>> is using perspective, but if Chrome decides to use Skia for 3D
>> rendering, the priority can change. (And I will be happy to contribute
>> too.)
>
> As danakj says, this bug would appear in Chrome today when using
> software rasterization. See the setMatrix code here:
> https://code.google.com/p/chromium/codesearch#chromium/src/cc/output/software_renderer.cc&l=263

Yep, I was surprised this doesn't reproduce trivially, but I can
confirm this bug does reproduce in Chrome. It takes some trick.

So the story is that even with software compositing Chrome still
creates tiles, unless we use the resourceless rendering mode (which
won't simply work with browser compositor). When we raster on the
tiles, we raster in local coordinate (albeit with raster scale
applied), so we won't see the bug on the tiles. However the bug still
affects drawing tile quads onto the render surface.

When we generate tile quads, we do calculate the visible content rect
for quads to reduce pixel count. Obviously the actual visible "region"
of the quad will be always in front of the observer, thus immune to
the w zero-crossing bug in Skia. To reproduce the bug, we want to make
at least one corner of the visible rect (= bounding box of the visible
region) to get behind the observer. Here is one such example:
http://jsbin.com/jekiwi/2/

When rendered on a sufficiently big viewport, some pixel on the right
edge and some pixel on the bottom edge of the layer are both visible,
thus with a full visible content rect. However the bottom right corner
of the layer is actually behind the observer. With HW compositing this
w zero-crossing is handled by GL correctly, but with SW compositing
Skia flips the behind corner to opposite location in the front.

Bonus: Try resize the window, when the visible content rect gets small
enough so the bottom right corner is no longer behind, suddenly Skia
renders correctly.

Bonus 2: Enable the debug border. Those are not clipped. In the above
example you can always see a chevron quad. If you tune the matrix a
bit to make two corners behind, you get a butterfly quad. :)

[Tien-Ren Chen]

On Thu, Apr 9, 2015 at 8:59 AM, Chris Harrelson <chri...@chromium.org> wrote:
> This is an interesting discussion, but I feel obligated to play the Boring
> Cop and repeat what Jeremy said. :)

Yep I totally agree exploring more freedom in layerization should be
after a successful phase 2 launch.

But implementing a simplistic dumb renderer that rasters 3D
transformed display items without layerization will help to verify
correctness of 3D transform recorders in Blink. I imagine it would be
dead simple if we don't implement depth sorting.

> That being said, a clarifying question: do you mean rastering into a single
> surface, or literally outputting display items with global coordinates?

Yes, rastering into a single surface. More specific, allowing generic
raster matrix other than only simple scale.

[Chris Harrelson]

On Thu, Apr 9, 2015 at 5:29 PM, Tien-Ren Chen <trc...@google.com> wrote:

On Thu, Apr 9, 2015 at 8:59 AM, Chris Harrelson <chri...@chromium.org> wrote:
> This is an interesting discussion, but I feel obligated to play the Boring
> Cop and repeat what Jeremy said. :)

Yep I totally agree exploring more freedom in layerization should be
after a successful phase 2 launch.

But implementing a simplistic dumb renderer that rasters 3D
transformed display items without layerization will help to verify
correctness of 3D transform recorders in Blink. I imagine it would be
dead simple if we don't implement depth sorting.

Cool idea. 

And software compositing would do it for you, modulo the class of bugs Enne referred to.