is there any plan to enable cursor plane for sun4i-drm?

[张宁]

Hi, Maxime, Icenowy and other developers

In https://linux-sunxi.org/Xorg, it says legacy display engine driver supports HW cursor, but there is no code actually creates a cursor plane in mainline linux, this is also said in: https://groups.google.com/forum/#!searchin/linux-sunxi/subject$3A$20cursor|sort:date/linux-sunxi/6dZVBtNgh5Q/dpBDfvEjBgAJ

currently,  lima usespace driver in mesa already has the basic function for desktop, if lima is enabled, cursor rendering will possibly go GPU, this will be less efficient than HW cursor.

do you have plan to enable cursor plane?

[Vasily Khoruzhick]

That's up to X11 developers. sun4i-drm exposes UI and VI planes on
SoCs with DE2, and there's no dedicated cursor plane in hardware.
Basically sun4i-drm exposes what's available in hardware. But
xf86-video-modesetting doesn't use VI plane for anything while it
could use it for cursor.

> --
> You received this message because you are subscribed to the Google Groups "linux-sunxi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to linux-sunxi...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/linux-sunxi/8d091584-8e01-431d-b9b0-93dd7e0f0cec%40googlegroups.com.

[张宁]

Hi, Vasily

from source, it looks like VI layer is a video plane, right?

let xf86-vidoe-modesetting driver support sun4i-drm's VI layer is a way to support HW cursor,

I want to ask whether it's possible to change VI layer type to cursor plane, then no changes in modesetting driver?

BR.

Ning.

On Tuesday, September 3, 2019 at 11:21:48 PM UTC+8, Vasily Khoruzhick wrote:

On Tue, Sep 3, 2019 at 1:49 AM 张宁 <zhang...@gmail.com> wrote:
>
> Hi, Maxime, Icenowy and other developers
>
> In https://linux-sunxi.org/Xorg, it says legacy display engine driver supports HW cursor, but there is no code actually creates a cursor plane in mainline linux, this is also said in: https://groups.google.com/forum/#!searchin/linux-sunxi/subject$3A$20cursor|sort:date/linux-sunxi/6dZVBtNgh5Q/dpBDfvEjBgAJ
>
> currently,  lima usespace driver in mesa already has the basic function for desktop, if lima is enabled, cursor rendering will possibly go GPU, this will be less efficient than HW cursor.
>
> do you have plan to enable cursor plane?

That's up to X11 developers. sun4i-drm exposes UI and VI planes on
SoCs with DE2, and there's no dedicated cursor plane in hardware.
Basically sun4i-drm exposes what's available in hardware. But
xf86-video-modesetting doesn't use VI plane for anything while it
could use it for cursor.

> --
> You received this message because you are subscribed to the Google Groups "linux-sunxi" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to linux...@googlegroups.com.

[张宁]

just check drm_mode_cursor_universal, cursor plane needs to support DRM_FORMAT_ARGB8888

but VI layer doesn't support alpha, directly change VI layer to cursor plane is not possible.

could we use UI layers?

[Jernej Škrabec]

Dne sreda, 04. september 2019 ob 05:08:14 CEST je 张宁 napisal(a):

> just check drm_mode_cursor_universal, cursor plane needs to support
> DRM_FORMAT_ARGB8888
>
> but VI layer doesn't support alpha, directly change VI layer to cursor
> plane is not possible.
>
> could we use UI layers?

Sure, but note that second mixer usually supports only one VI and one UI
plane. Note that primary plane is always on first UI, so you can run out of UI
planes.

Most notable example of that combination is A64 HDMI, while LCD output on A64
has one VI and three UI planes. There is a switch in hardware to connect 1 VI/
3 UI mixer to A64 HDMI, but it's not implemented and user space would have to
be aware of that.

Best regards,
Jernej

[张宁]

thanks for the explain, I want to have a try on H3 (orangepipc, armbian)

before testing, there are 2 questions in my mind.

1, what's the Z-order of UI/VI layers?

2, can Xserver use overlays?

[张宁]

I have the answer for 2 from a friend.

and for question 1:

default zpos for UI layer is channel = mixer->cfg->vi_num + index;

does it means UI layers are higher layers than VI layer?

could we use VI layer as crtc's primary plane, and topest UI layer as cursor plane.

then all mixers has cursor plane.

[Ondřej Jirman]

On Wed, Sep 04, 2019 at 07:29:39AM +0200, Jernej Škrabec wrote:
> Dne sreda, 04. september 2019 ob 05:08:14 CEST je 张宁 napisal(a):
> > just check drm_mode_cursor_universal, cursor plane needs to support
> > DRM_FORMAT_ARGB8888
> >
> > but VI layer doesn't support alpha, directly change VI layer to cursor
> > plane is not possible.
> >
> > could we use UI layers?
>
> Sure, but note that second mixer usually supports only one VI and one UI
> plane. Note that primary plane is always on first UI, so you can run out of UI
> planes.
>
> Most notable example of that combination is A64 HDMI, while LCD output on A64
> has one VI and three UI planes. There is a switch in hardware to connect 1 VI/
> 3 UI mixer to A64 HDMI, but it's not implemented and user space would have to
> be aware of that.

Each mixer channel has 4 sub-layers, so it should be possible even with one UI
channel to have a a cursor plane within it (with some constraints).

> --
> You received this message because you are subscribed to the Google Groups "linux-sunxi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to linux-sunxi...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/linux-sunxi/8613808.bs98POC9MK%40jernej-laptop.

[Jernej Škrabec]

Dne sreda, 04. september 2019 ob 22:45:47 CEST je Ondřej Jirman napisal(a):

> On Wed, Sep 04, 2019 at 07:29:39AM +0200, Jernej Škrabec wrote:
> > Dne sreda, 04. september 2019 ob 05:08:14 CEST je 张宁 napisal(a):
> > > just check drm_mode_cursor_universal, cursor plane needs to support
> > > DRM_FORMAT_ARGB8888
> > >
> > > but VI layer doesn't support alpha, directly change VI layer to cursor
> > > plane is not possible.
> > >
> > > could we use UI layers?
> >
> > Sure, but note that second mixer usually supports only one VI and one UI
> > plane. Note that primary plane is always on first UI, so you can run out
> > of UI planes.
> >
> > Most notable example of that combination is A64 HDMI, while LCD output on
> > A64 has one VI and three UI planes. There is a switch in hardware to
> > connect 1 VI/ 3 UI mixer to A64 HDMI, but it's not implemented and user
> > space would have to be aware of that.
>
> Each mixer channel has 4 sub-layers, so it should be possible even with one
> UI channel to have a a cursor plane within it (with some constraints).

If it's acceptable that cursor is square image then yes. There is no blending
between sub-layers.

[Ondřej Jirman]

On Wed, Sep 04, 2019 at 11:02:33PM +0200, Jernej Škrabec wrote:
> Dne sreda, 04. september 2019 ob 22:45:47 CEST je Ondřej Jirman napisal(a):
> > On Wed, Sep 04, 2019 at 07:29:39AM +0200, Jernej Škrabec wrote:
> > > Dne sreda, 04. september 2019 ob 05:08:14 CEST je 张宁 napisal(a):
> > > > just check drm_mode_cursor_universal, cursor plane needs to support
> > > > DRM_FORMAT_ARGB8888
> > > >
> > > > but VI layer doesn't support alpha, directly change VI layer to cursor
> > > > plane is not possible.
> > > >
> > > > could we use UI layers?
> > >
> > > Sure, but note that second mixer usually supports only one VI and one UI
> > > plane. Note that primary plane is always on first UI, so you can run out
> > > of UI planes.
> > >
> > > Most notable example of that combination is A64 HDMI, while LCD output on
> > > A64 has one VI and three UI planes. There is a switch in hardware to
> > > connect 1 VI/ 3 UI mixer to A64 HDMI, but it's not implemented and user
> > > space would have to be aware of that.
> >
> > Each mixer channel has 4 sub-layers, so it should be possible even with one
> > UI channel to have a a cursor plane within it (with some constraints).
>
> If it's acceptable that cursor is square image then yes. There is no blending
> between sub-layers.

Ah, bummer.

[Jernej Škrabec]

Dne sreda, 04. september 2019 ob 23:10:15 CEST je Ondřej Jirman napisal(a):

Looking at DE2 documentation, sub-layer concept is very simple from HW point
of view. There is no adjustable Z-pos, scaling, blending or pixel format
conversion (doc states that they must be in same format) between sub-layers.
To no surprise DMA engine is not part of sub-layer according to mixer block
diagram. It seems like that sub-layers part only feeds DMA with correct order
of memory locations which DMA needs to transfer to the next unit (scaler in
this case).

I don't think these sub-layers fit in current DRM design. I'm also not sure in
which case they would be usable.

/offtopic

[张宁]

Update my test result:

1, use 1st UI layer as primary plane and 3rd UI layer as cursor plane.

display normally but slow with fbdev

display abnormally (only cursor shows) with modesetting (lima enabled)

2, use VI layer as primary plane, and toppest UI layer as cursor plane

display abnormal in both fbdev and modesetting( lima enabled)

so as said by Jernej: these sub-layers don't fit in current DRM design.

[张宁]

in this post: https://groups.google.com/forum/#!searchin/linux-sunxi/cursor|sort:date/linux-sunxi/9AaiVMgtfUE/kk-ONXZ2BwAJ

it tried to use some higher channel for cursor. but it was dropped.

[Ondřej Jirman]

On Thu, Sep 05, 2019 at 06:30:42PM -0700, 张宁 wrote:
> Update my test result:
>
> 1, use 1st UI layer as primary plane and 3rd UI layer as cursor plane.
> display normally but slow with fbdev
> display abnormally (only cursor shows) with modesetting (lima enabled)

You can "fix" this by switching to console and back. I've debugged this
somewhat and it the sun4i drm driver is probably buggy. The drm layers/
framebuffers are exactly same before and after the switch.

I have not yet dumped mixer registers before/after, but I suspect, there
will be some difference, for whatever reason.

regards,
o.

> --
> You received this message because you are subscribed to the Google Groups "linux-sunxi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to linux-sunxi...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/linux-sunxi/6aba8962-c9a5-49b3-96b8-e623562a06f1%40googlegroups.com.

[Ondřej Jirman]

On Thu, Sep 05, 2019 at 06:30:42PM -0700, 张宁 wrote:

> Update my test result:
>
> 1, use 1st UI layer as primary plane and 3rd UI layer as cursor plane.
> display normally but slow with fbdev
> display abnormally (only cursor shows) with modesetting (lima enabled)

BTW, I use this patch:

https://megous.com/dl/tmp/0001-drm-sun4i-Mark-one-of-the-UI-planes-as-a-cursor-one.patch

o.

> --
> You received this message because you are subscribed to the Google Groups "linux-sunxi" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to linux-sunxi...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/linux-sunxi/6aba8962-c9a5-49b3-96b8-e623562a06f1%40googlegroups.com.

[Ondřej Jirman]

Hi,

On Sun, Sep 08, 2019 at 12:21:45AM +0200, megous hlavni wrote:
> On Thu, Sep 05, 2019 at 06:30:42PM -0700, 张宁 wrote:
> > Update my test result:
> >
> > 1, use 1st UI layer as primary plane and 3rd UI layer as cursor plane.
> > display normally but slow with fbdev
> > display abnormally (only cursor shows) with modesetting (lima enabled)
>
> You can "fix" this by switching to console and back. I've debugged this
> somewhat and it the sun4i drm driver is probably buggy. The drm layers/
> framebuffers are exactly same before and after the switch.
>
> I have not yet dumped mixer registers before/after, but I suspect, there
> will be some difference, for whatever reason.

So I have dumped registers, and found out this:

mixer0:

first Xorg run:

0x01101000 : 00000201
0x01101080 : 00000030

BLD_FILL_COLOR_CTL: (aka SUN8I_MIXER_BLEND_PIPE_CTL)
P1_EN

BLD_CH_RTCTL: (aka SUN8I_MIXER_BLEND_ROUTE)
P0_RTCTL channel0
P1_RTCTL channel3


after switch to console and back to Xorg:

0x01101000 : 00000301
0x01101080 : 00000031

BLD_FILL_COLOR_CTL:
P1_EN and P0_EN

BLD_CH_RTCTL:
P0_RTCTL channel1
P1_RTCTL channel3


This is despite the layers config being exactly the same as indicated
in /sys/kernel/debug/dri/0/{state,framebuffer}.

sun4i-drm driver just doesn't handle layer change correctly in all cases.

For example when enabling a layer, it may disable a different layer if
old_zpos matches the zpos of the other layer.

I've made a fix for this:

https://megous.com/git/linux/commit/?h=private-5.3&id=c186b256a869bab09103075a133f3c0d5f7b48cc
https://megous.com/git/linux/commit/?h=private-5.3&id=5af208e90de5ced30350fc0fba8419e9662e9bb7

So with these two patches, Xorg server/modesetting driver will use the cursor
plane, and plane enable/switching issues are gone (in general, even if you
don't use the cursor plane patch).

Anyway, this makes Xorg server work nicely with lima/panfrost without the
need to hack Xorg's modesetting driver to use non-cursor plane for cursor.

regards,
o.

> To view this discussion on the web, visit https://groups.google.com/d/msgid/linux-sunxi/20190907222145.4yfa6qy2uwjlkrut%40core.my.home.

[Priit Laes]

Please CC these patches also to Maxime and Wens.

> To view this discussion on the web, visit https://groups.google.com/d/msgid/linux-sunxi/20190908102050.5oqrxdedtbrnxtlp%40core.my.home.

[Jernej Škrabec]

Dne nedelja, 08. september 2019 ob 12:20:50 CEST je Ondřej Jirman napisal(a):

Why would that be a hack? AFAIK overlay plane has all and additional
functionality of cursor plane. I think it's better to change modesetting
driver to use overlay planes. Some (most?) cursor planes are limited and
really useful only for cursors, so apps might just skip them.

Best regards,
Jernej

[Ondřej Jirman]

Hi,

Read that as a synonym for "program". :)

Modesetting driver is a builtin and Xorg server releases are few and far
between, and I'm not going to wait for years and have to manage, compile, and
distribute my own patched Xorg server in the meantime if I don't have to. This
solution works and is pretty much equivalent. One layer would be lost to the
cursor use either way. And kernel change works with existing and legacy code.

I didn't even look at how hard it would be to change just for this reason alone.

Anyway, while having a special cursor plane is nice and lowers the CPU load
quite a bit while moving a mouse, the layer management bugs in the current code
need fixing first.

Maybe later we can add kernel command line option for optionally marking one of
the layers as a cursor one, so that people can choose what they want, depending
on their use case.

regards,
o.

[Ondřej Jirman]

It would probably be usable if you don't need transparency and scaling inside
the square areas for each sub-layer.

Myabe some hypothetical tiling WM would be able to use that many layers. The
layers would not even need to overlap, or have differnent FB formats.

Z-ordering can be done by not having a fixed mapping between DRM planes and DE2
layers/sub-layers and doing the mapping dynamically based on the desired zpos
for each sub-layer. Anyway if the sub-layers don't overlap there's no need for
Z-ordering.

I aggree that the current DRM API is not a great match for exposing this.

regards,
o.

[张宁]

thanks for your patches.

BR.

Ning.

> > To view this discussion on the web, visit https://groups.google.com/d/msgid/linux-sunxi/6aba8962-c9a5-49b3-96b8-e623562a06f1%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups "linux-sunxi" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to linux...@googlegroups.com.

> To view this discussion on the web, visit https://groups.google.com/d/msgid/linux-sunxi/20190907222145.4yfa6qy2uwjlkrut%40core.my.home.

[security researcher]

What's the latest solution here? I see a few patches in the thread, but not sure which ones to pick and choose?

[Ondřej Jirman]

On Tue, Sep 17, 2019 at 08:55:42AM -0700, security researcher wrote:
> What's the latest solution here? I see a few patches in the thread, but not
> sure which ones to pick and choose?

Use both:

https://megous.com/git/linux/commit/?h=private-5.3&id=c186b256a869bab09103075a133f3c0d5f7b48cc
https://megous.com/git/linux/commit/?h=private-5.3&id=5af208e90de5ced30350fc0fba8419e9662e9bb7

> To unsubscribe from this group and stop receiving emails from it, send an email to linux-sunxi...@googlegroups.com.
> To view this discussion on the web, visit https://groups.google.com/d/msgid/linux-sunxi/14392370-4e1a-4a07-8350-26925b6d5192%40googlegroups.com.