How to use FilterOperations on layers

[beha...@samsung.com]

Hi,
I tried applying FiltersOperations(invert,zoom ... etc ) to the layers in layer tree host.
But it is not working.
Can someone help me. I want to use Filters

https://code.google.com/p/chromium/codesearch#chromium/src/cc/output/filter_operation.h&type=cs&sq=package:chromium&l=24&rcl=1400145541

[Tom Wiltzius]

+stephen

[Dana Jansens]

Here's some example:

https://code.google.com/p/chromium/codesearch#chromium/src/ui/compositor/layer.cc&l=342

[Shyam Patro]

@Dana Jansens thanks for your reply the example is for setting filters to UI compositor layers.

How to use the same filter operations for src/cc/layers/layer.cc.

I tried setting the filters using Layer::SetFilters(const FilterOperations& filters) and Layer::SetBackgroundFilters(const FilterOperations& filters).

But there was no output am i missing something?

[Dana Jansens]

On Mon, May 19, 2014 at 12:36 AM, Shyam Patro <patro...@gmail.com> wrote:

@Dana Jansens thanks for your reply the example is for setting filters to UI compositor layers.

How to use the same filter operations for src/cc/layers/layer.cc.

I tried setting the filters using Layer::SetFilters(const FilterOperations& filters) and Layer::SetBackgroundFilters(const FilterOperations& filters).

But there was no output am i missing something?

I'm not sure what you mean by no output. You'd set it on a layer that otherwise already shows some content, and the filters will modify it.

[loy...@gmail.com]

Рш!

On Monday, May 19, 2014 8:36:48 AM UTC+4, Shyam wrote:

> How to use the same filter operations for src/cc/layers/layer.cc.
> I tried setting the filters using Layer::SetFilters(const FilterOperations& filters) and Layer::SetBackgroundFilters(const FilterOperations& filters).
> But there was no output am i missing something?

What helped me:
You should set animation->set_needs_synchronized_start_time(true) flag to run your animations on the Impl side (even if using SingleThreadProxy LayerTreeHost/compositor).

Sincerely yours,
Loyso.

[beha...@samsung.com]

@Dana Following the example code you pointed to I was able to apply filters to cc:layer. I added filter operations to current layer using SetBackgroundFilters(const FilterOperations& filters). The filter was applied to all the layers below that. And I made the current layer transparent to see the effect.

Thanks
Shyam

[siva.g...@samsung.com]

@Dana,

Background filters gets applied to layers below the current one.
Say i have applied a zoom filter at location 0,40 ,100,100.
It magnifies the underlying background content rect, can i transform (the origin) this magnified layer contents to get displayed at some other location ..like 40,40,100,100 (display here magnified contents of 0,40,100,100 rect), with out using CopyOutputRequest?

Thanks,
Siva

[Dana Jansens]

Did you try the foreground filters (aka |filters|)? They are applied to the layer and its subtree, not to the things below it. If you're trying to display content twice, you could just place that content in the tree twice.

 

Thanks,
Siva

[Alexey Baskakov]

Hi, Dana!

> If you're trying to display content twice, you could just place that content in the tree twice.

Do you mean Layer::SetReplicaLayer?
Layer::AddChild removes a newly added child from any previous parent layer.

Loyso.
Yandex Browser Team.

[Dana Jansens]

On Thu, Jul 3, 2014 at 10:01 AM, Alexey Baskakov <loy...@gmail.com> wrote:

Hi, Dana!

> If you're trying to display content twice, you could just place that content in the tree twice.

Do you mean Layer::SetReplicaLayer?

That's one way.

 

Layer::AddChild removes a newly added child from any previous parent layer.

You could also just make two layers with the same content in them.

[Alexey Baskakov]

Hi again, Dana!

On Thursday, July 3, 2014 7:30:37 PM UTC+4, Dana Jansens wrote:

On Thu, Jul 3, 2014 at 10:01 AM, Alexey Baskakov <loy...@gmail.com> wrote:

Hi, Dana!

> If you're trying to display content twice, you could just place that content in the tree twice.

Do you mean Layer::SetReplicaLayer?

That's one way.

 

Layer::AddChild removes a newly added child from any previous parent layer.

You could also just make two layers with the same content in them.

So, what would be a good way to have _live_ content layer twice (two instances overlapped, transformed, scaled, with individual filters)?

It looks like Replica layer is shown always behind it's parent. (It's is the opposite for regular children layers - they are always in front of parent. In case of simple 2d translations, of course).

Thank you for the answers!

Loyso.

[Dana Jansens]

On Wed, Jul 9, 2014 at 4:20 AM, Alexey Baskakov <loy...@gmail.com> wrote:

Hi again, Dana!



On Thursday, July 3, 2014 7:30:37 PM UTC+4, Dana Jansens wrote:

On Thu, Jul 3, 2014 at 10:01 AM, Alexey Baskakov <loy...@gmail.com> wrote:

Hi, Dana!

> If you're trying to display content twice, you could just place that content in the tree twice.

Do you mean Layer::SetReplicaLayer?

That's one way.

 

Layer::AddChild removes a newly added child from any previous parent layer.

You could also just make two layers with the same content in them.

So, what would be a good way to have _live_ content layer twice (two instances overlapped, transformed, scaled, with individual filters)?

It looks like Replica layer is shown always behind it's parent. (It's is the opposite for regular children layers - they are always in front of parent. In case of simple 2d translations, of course).

Is this a problem? Can you swap the position of the layer and the replica? I think they can have different filters no?

[Alexey Baskakov]

On Wednesday, July 9, 2014 8:04:01 PM UTC+4, Dana Jansens wrote:

On Wed, Jul 9, 2014 at 4:20 AM, Alexey Baskakov <loy...@gmail.com> wrote:

Hi again, Dana!



On Thursday, July 3, 2014 7:30:37 PM UTC+4, Dana Jansens wrote:

On Thu, Jul 3, 2014 at 10:01 AM, Alexey Baskakov <loy...@gmail.com> wrote:

Hi, Dana!

> If you're trying to display content twice, you could just place that content in the tree twice.

Do you mean Layer::SetReplicaLayer?

That's one way.

 

Layer::AddChild removes a newly added child from any previous parent layer.

You could also just make two layers with the same content in them.

So, what would be a good way to have _live_ content layer twice (two instances overlapped, transformed, scaled, with individual filters)?

It looks like Replica layer is shown always behind it's parent. (It's is the opposite for regular children layers - they are always in front of parent. In case of simple 2d translations, of course).

Is this a problem? Can you swap the position of the layer and the replica? I think they can have different filters no?

Yes, that's a bit problematic. What do you mean by swapping the position? The only machinery I know is "children stacking order" (and z-order, probbaly. But I don't want to dive into 3d for now).

Layer::SetReplicaLayer is unidirectional relationship (as I understand), I can't swap a parent and a child in this case.

So, I don't understand something.
 

[Dana Jansens]

On Thu, Jul 10, 2014 at 3:21 AM, Alexey Baskakov <loy...@gmail.com> wrote:

On Wednesday, July 9, 2014 8:04:01 PM UTC+4, Dana Jansens wrote:

On Wed, Jul 9, 2014 at 4:20 AM, Alexey Baskakov <loy...@gmail.com> wrote:

Hi again, Dana!



On Thursday, July 3, 2014 7:30:37 PM UTC+4, Dana Jansens wrote:

On Thu, Jul 3, 2014 at 10:01 AM, Alexey Baskakov <loy...@gmail.com> wrote:

Hi, Dana!

> If you're trying to display content twice, you could just place that content in the tree twice.

Do you mean Layer::SetReplicaLayer?

That's one way.

 

Layer::AddChild removes a newly added child from any previous parent layer.

You could also just make two layers with the same content in them.

So, what would be a good way to have _live_ content layer twice (two instances overlapped, transformed, scaled, with individual filters)?

It looks like Replica layer is shown always behind it's parent. (It's is the opposite for regular children layers - they are always in front of parent. In case of simple 2d translations, of course).

Is this a problem? Can you swap the position of the layer and the replica? I think they can have different filters no?

Yes, that's a bit problematic. What do you mean by swapping the position? The only machinery I know is "children stacking order" (and z-order, probbaly. But I don't want to dive into 3d for now).

Layer::SetReplicaLayer is unidirectional relationship (as I understand), I can't swap a parent and a child in this case.

So, I don't understand something.

It sounded like the Replica being behind its parent was a problem. If the parent is at 10,10 and the replica at 20,20, but you want the 20,20 to appear above, you could put the parent at 20,20 and the replica at 10,10.

It seems simple so probably I don't understand really what problem you're facing.

[Alexey Baskakov]

On Thursday, July 10, 2014 9:22:16 PM UTC+4, Dana Jansens wrote:

On Thu, Jul 10, 2014 at 3:21 AM, Alexey Baskakov <loy...@gmail.com> wrote:

On Wednesday, July 9, 2014 8:04:01 PM UTC+4, Dana Jansens wrote:

On Wed, Jul 9, 2014 at 4:20 AM, Alexey Baskakov <loy...@gmail.com> wrote:

Hi again, Dana!



On Thursday, July 3, 2014 7:30:37 PM UTC+4, Dana Jansens wrote:

On Thu, Jul 3, 2014 at 10:01 AM, Alexey Baskakov <loy...@gmail.com> wrote:

Hi, Dana!

> If you're trying to display content twice, you could just place that content in the tree twice.

Do you mean Layer::SetReplicaLayer?

That's one way.

 

Layer::AddChild removes a newly added child from any previous parent layer.

You could also just make two layers with the same content in them.

So, what would be a good way to have _live_ content layer twice (two instances overlapped, transformed, scaled, with individual filters)?

It looks like Replica layer is shown always behind it's parent. (It's is the opposite for regular children layers - they are always in front of parent. In case of simple 2d translations, of course).

Is this a problem? Can you swap the position of the layer and the replica? I think they can have different filters no?

Yes, that's a bit problematic. What do you mean by swapping the position? The only machinery I know is "children stacking order" (and z-order, probbaly. But I don't want to dive into 3d for now).

Layer::SetReplicaLayer is unidirectional relationship (as I understand), I can't swap a parent and a child in this case.

So, I don't understand something.

It sounded like the Replica being behind its parent was a problem. If the parent is at 10,10 and the replica at 20,20, but you want the 20,20 to appear above, you could put the parent at 20,20 and the replica at 10,10.

It seems simple so probably I don't understand really what problem you're facing.

1) The parent layer propagates it's transform and filters to the replica layer.
2) But a trasform is inversible. Filters are not.
3) Replica layer ignores filters set on it.

So, I can't have individual filters for them, as I mentioned earlier.

In my example parent layer is smaller then it's replica. And the parent's layer bounding rectangle is a subset of the replica bounding rect (located inside of it).

The only option I see is extra layers with background filters. The tricky part is a rectangle 'hole' where necessary.  I need 4+1 extra layers on top for that. Not good at all :(

Loyso.
 

[Alexey Baskakov]

p.s. I've found FilterOperation::set_region(). Not that good approach anyway... Would be nice to have parent/replica individual filters working...
 

[Dana Jansens]

That's only for one specific filter type.

What kind of layer/content are you trying to duplicate?

[Alexey Baskakov]

That's DelegatedRendererLayer (from ContentViewCore on Android), in fact.

It could be a TextureLayer (as a result of readback from DelegatedRendererLayer), in theory.

[Dana Jansens]

For DRL you can just make another DRL and give it the same FrameProvider. See DelegatedFrameHost::OnLayerRecreated. https://code.google.com/p/chromium/codesearch#chromium/src/content/browser/compositor/delegated_frame_host.cc&sq=package:chromium&rcl=1404978341&l=840

[Alexey Baskakov]

Thanks! I saw the idea in the CL for Android readback (second hidden DRL is used there)

 

Technically, it works. However, there is no ui::Layers on Android.

Moreover, RenderWidgetHostViewAndroid::frame_provider_ is private.

Can we introduce an accessor OR implement something like RenderWidgetHostViewAndroid::CreateDelegatedRendererLayerForFrameProvider() ?

The next question: How to freeze it at just one frame or convert it into TextureLayer? Say, I don't need sequence of frames.

 

Loyso.

[Dana Jansens]

That's a question for the android people. +aelias

The next question: How to freeze it at just one frame or convert it into TextureLayer? Say, I don't need sequence of frames.

You'd have to insert a CopyOutputRequest, which gives you back a texture you can put into a TextureLayer.

[Alexey Baskakov]

That's what I definitely want to avoid! The readback process into CPU SkBitmap is very slow. So, that was the whole point to use a DRL.

[Dana Jansens]

In GL compositing mode CopyOutputRequest returns a texture not a bitmap.

[Alexandre Elias]

On Mon, Jul 14, 2014 at 11:25 AM, Dana Jansens <dan...@chromium.org> wrote:

For DRL you can just make another DRL and give it the same FrameProvider. See DelegatedFrameHost::OnLayerRecreated. https://code.google.com/p/chromium/codesearch#chromium/src/content/browser/compositor/delegated_frame_host.cc&sq=package:chromium&rcl=1404978341&l=840

Thanks! I saw the idea in the CL for Android readback (second hidden DRL is used there)

 

Technically, it works. However, there is no ui::Layers on Android.

Moreover, RenderWidgetHostViewAndroid::frame_provider_ is private.

Can we introduce an accessor OR implement something like RenderWidgetHostViewAndroid::CreateDelegatedRendererLayerForFrameProvider() ?

That's a question for the android people. +aelias

That sounds reasonable to me.

[Alexey Baskakov]

Hi again! Sorry for late answer. I did an attempt to put a CopyOutputRequest into TextureLayer.

Result: it takes 300 ms on Nexus 4.
So it's probably useless to deliver a seamless UI flow on the DRL to screenshot switches.
 

[Dana Jansens]

300ms for a texture copy? Are you doing a readback to a bitmap, upload to a texture, instead of just getting a texture back and inserting it into TextureLayer?

[Alexey Baskakov]

No, I'm not doing a readback. That's for Layer::RequestCopyOfOutput which returns CopyOutputResult with HasTexture() == true.

UPDATE: I've disabled ContentViewCore::GetScaledContentBitmap request (which was running in parallel) and now the texture copy consumes 100 ms. That's better, but still no opportunity for seamless user interaction flow...
 

[Dana Jansens]

Can it be your driver is particularly poor at doing glCopyTexImage2D? Or I wonder if we should change that CopyTexImage2D to a CopyTextureCHROMIUM in gl_renderer.cc. +piman

[Alexey Baskakov]

That's Google/LG Nexus 4. Not sure if it has a poor driver.

So, the texture copy time is 66-100 ms (+/- a vsync delay at 30 FPS maybe?)

Extra info: that is a texture copy from the secondary DelegatedRendererLayer, created around RenderWidgetHostViewAndroid::frame_provider_ (a DelegatedFrameProvider).
 

[Antoine Labour]

On most archs I would expect CopyTexImage2D to be faster than CopyTextureCHROMIUM. If anything, I would treat this as a driver bug and add a fallback to CopyTextureCHROMIUM on the service side for affected drivers.

100ms for a CopyTexImage2D is unreasonable.

Antoine

[Eric Penner]

Sorry for the lateness. I'm just curious, what was the size of the texture being copied? 100ms does seem way too high unless the texture is really huge. However, I'm skeptical if this 100ms is purely in the driver (N4 driver is typically decent in this area). Grabbing a systrace of the GL calls would be helpful:

adb_profile_chrome --categories="gpu,cc" --systrace='gfx'  (with GL tracing enabled in android developer tools)

On performance, ideally CopyTexImage2D should be the best for any driver, but it's typically a memcpy. At that point glBlitFrameBuffer is the best on GLES3, followed by a shader copy (but needs to be a fairly big copy on some drivers).

I like the idea of workarounds in glCopyTexImage2D, and reserving CopyTextureCHROMIUM only for the additional functionality, rather than for performance.

[Alexey Baskakov]

Thanks for the answer anyway! :)
 

See a pseudo-code here: https://gist.github.com/loyso/9b3a0d6a4760f8bd0674

What if that's just an Impl-syncing delay? Layer::RequestCopyOfOutput just marks itsels as SetNeedsCommit.

 

[Alexey Baskakov]

p.s. That was an LG Nexus 4 screenshot. I.e. 768x1134 (the size of a SurfaceView mSurfaceView)
 

[Dana Jansens]

If you take a trace you should see where the time is being spent.