A20 - Mainline 5.8.0 - Display Engine

[Milos Ladni]

Hi all,

I am trying to port my custom board from legacy kernel 3.4 to the mainline.
I am using an LVDS display with a capacitive touchscreen and it works fine. For the test, i am currently running Armbian Debian Buster without Desktop environment because i need only one application on top of all.
I additionally installed manually by apt-get X window and ratpoison window manager. Run firefox which is used for Javascript GUI interpretation, Cedar VE for encoding/decoding and DisplayEngine for showing the video on display on a particular place and particular size.
On mainline, i recompiled Cedar VE instead of Cedrus because i already wrote all needed features for both Encoding/Decoding H264 and MJPEG for old legacy kernel and because encoding is not currently supported by Cedrus.
Under legacy kernel i wrote my VideoEngine application and managed to get 4 simultaneous operation (Encode H264 and MJPEG and Decode H264 and MJPEG), all this works smoothly for a long time in legacy kernel 3.4 with any issue.

For mainline, i recompiled CedarVE from uboborov repo: https://github.com/uboborov/sunxi-cedar-mainline.
I already tested CedarVE Encoder part but for Decoder i need to show video on my display on particular place with particular resolution.
On old kernel, there was /dev/disp node which was used for showing video across firefox application in a particular window place or even in full screen.

My question is how to do this in Mainline kernel?
I set my dtb and enable Display Engine but i am stuck there.
I see in dmesg that framebuffer is switched from simple to sun4i-drm-fb and i have /dev/dri/card0 device node but i am not sure what is my next step.
Do i need to create some drm plane for that purpose and how? Is there any simple example of how to do this?

Also when i run '$startx ratpoison' in its output i can see next message: 'failed to load driver: sun4i-drm'. Does it mean that X window could not use DRM or what? I am not sure if drm is working or not. And did i create drm display engine pipeline as well?
Also, i think that firefox which i am using for running my Javascript GUI is an overkill a little bit. Does someone else can recommend some other interpreter?
Do i need to compile and run lima driver for my purpose?

Sorry for long description. I also attached my dtb files and dmesg output here.

Thanks,
Milos Ladicorbic

[Andrey Lebedev]

Hi Milos,

I'm not exactly sure what problem are you solving. Do you see anything on your LVDS screen? Lima driver is for graphics/OpenGL acceleration. It is separate from DRM display drivers, but you probably want to have it too (you should expect both /dev/dri/card0 and /dev/dri/card1 present) .

I have A20 (cubieboard) running mainline kernel with lima and DRM drivers with these device tree and kernel configs:

* https://github.com/Openvario/meta-openvario/blob/warrior/recipes-kernel/linux/linux-mainline/openvario-7-CH070.dts

* https://github.com/Openvario/meta-openvario/blob/warrior/recipes-kernel/linux/linux-mainline/openvario-common.dts

* https://github.com/Openvario/meta-openvario/blob/warrior/recipes-kernel/linux/linux-mainline/arm/defconfig

I can post dmesg output of a working system too if you are interested.

[Milos Ladni]

 Hi Andrey, thank you for supporting.

Actually, i am not sure if i need 'lima' driver or not... Because i run a simple web based (Javasctipt) GUI application in firefox in fullscreen kiosk mode. It has only two tabs (contexts) with fixed size without scrolling, a few buttons and window space for playing video on top of all without desktop environment.

GUI application is very simple. I am not a web programmer but i know that i am not using any specific openGL code in web application.

Maybe lima can help with CPU load because no matter how simple my GUI application is, it does not run smoothly when i switch between these two tabs. CPU can take more than 50% during switching and showing different contexts to the user.

Anyway i also tried to load Lima driver and i got /dev/dri/card0,  card1  and renderD128 but i am not sure how to test it.

When i try to load xwindow or kmscube i get following error message:

root@olinuxino:~/workspace# kmscube
gbm: failed to open any driver (search paths /usr/lib/arm-linux-gnueabihf/dri:\$${ORIGIN}/dri:/usr/lib/dri)
gbm: Last dlopen error: /usr/lib/dri/sun4i-drm_dri.so: cannot open shared object file: No such file or directory

failed to load driver: sun4i-drm

kmscube fail at this point but xwindow show only this error and continue running. I can run gui application after xwindow is started.

My LVDS display is working well and one of my issues is how to put decoded video to a specific location in GUI application.

After the video is decoded by cedar VE it is in YUV NV12 RAW format. I have to scale it to another resolution and show it inside of GUI application.

On legacy kernel i used Display Engine and /dev/disp device node for this purpose.

Unfortunately on mainline /dev/disp vanish and we are forced to use drm driver instead of it which is if i understand well the same Display Engine driver in the background.

Regards,
Milos Ladicorbic

[Sertac Tüllük]

Hi, I am trying to maintain DS167, which is a similar device to yours ( A20, LVDS 7" and 10.1" LCDs with capacitive touchscreens ). 

Many years ago, I was running this product on Android 4.2.2 and Linux. As you know, Android UI performance on A20 was quite acceptable for basic HMI functionalities. 

For linux, I used debian 7 with 3.4 kernel, with firefox. It was again enough for basic functionalites, but of course the UI responsiveness was not on par with Android 4.2.2.

You are right, in the legacy kernel (3.4), there was MALI driver for GPU (3d graphics acceleration) and Cedar VE for video decoding/encoding acceleration.

Not all allwinner SoCs support it, but afaik A20 had G2D, and theoritically this gave us nice 2D acceleration.  This 2D acceleration could be achieved via compiling a specific xorg driver. ( Somebody please correct me if I am wrong). 

And in the mainline (current 5.8+), afaik, for GPU we have LIMA and optionally MALI also. LIMA works out-of-the-box, because it is open source. Unfortunately:

1) There is no clean explanation on sunxi-wiki or there is no documentation about how to test it on :

1.1) Raw Framebuffer

1.2) Qt on Framebuffer

1.3) X11

1.4) Wayland

2) I don't know how to achieve 2D hardware acceleration on mainline kernel. I couldn't find any documentation or explanation anywhere. I am volunteer to write this documentation on sunxi wiki if somebody helps me to achieve this on A20.

Now, on mainline, for video decoding acceleration, there is cedrus. Unfortunately there is no clear explanation or documentation, how to use or how to test Cedrus in MPV or Gstreamer. There is a PoC work for VLC, which is not suitable for applications like yours.

Again, if somebody guide me, I am volunteer to write official documentation for it.

Finally, you asked for recommendation for your application. I suggest you to experiment native QT for that. I believe it is not extremely hard to write your hello world app on simple framebuffer. For porting QT5 to your sysroot, I think you may check tutorials for Raspberrypi. 

I believe, if we can achieve to integrate QT on eglfs-kms , it must give us the UI responsiveness of Android 4.2.2 running on A20.

Again, we need good recommendations from the experts in this mail group, about how to achieve this.

6 Ağu 2020 Per 20:19 tarihinde Milos Ladni <milosla...@gmail.com> şunu yazdı:

--
You received this message because you are subscribed to the Google Groups "linux-sunxi" group.
To unsubscribe from this group and stop receiving emails from it, send an email to linux-sunxi...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/linux-sunxi/49e7f2a7-5c34-4265-8c2c-d868eeb9af14o%40googlegroups.com.

[Milos Ladni]

I think i have two problems:

1. When i decode video i need to find a way to show it under GUI application by using Display Engine (DRM in this case think) as a separate layer that is always on top. On legacy kernel, i used Display Engine by /dev/disp but it does not exist anymore.
2. Find a way how to speed up firefox. I tried to set 'layers.acceleration.force-enabled' under about:config but i didn't get any improvement. QT unfortunately is not an option because of license and guy who will maintain GUI application in the future currently does not have time to learn QT.
I suspect that my window manager uses LIMA in the right way, actually, i suspect that it uses lima at all because of errors which i posted in the previous message.

About cedar VE:
If you are interested in video encoding/decoding and PoC i have written a good example of how to do this in legacy kernel 3.4 a few years ago.
With my PoC you can have more than one operation simultaneously. I wrote it according to a few PoC which you could find on sunxi at that time. You can do H264 encoding/decoding and MJPEG encoding/decoding at the same time and it works like a charm without any CPU load and 25fps for each task (100 fps in total) for PAL resolution.
The good thing about this PoC beside simultaneous operation is H264 decoder which you couldn't find anywhere outside of VDPAU and cedrus. Because if you use VDPAU you can't do any other cedar task in parallel (only one operation is available) i decide to rewrite it by looking to source code for vdpau cedar plugin for GStreamer i think, i am not sure anymore.
This is the only example of how to direct use cedar VE for H264 decoding in a raw way.
I have to say that for input i used composite CVBS input which is not available in the mainline. It also wasn't available for legacy 3.4 but i patched it to get it working.
This PoC should also work on the mainline but it needs slight modification. Some cedar VE IOCTLs were changed and for video input you have to use some digital camera instead of TV in.

Best regards,
Milos

[Milos Ladni]

I forgot to put link for cedar VE PoC :)
https://github.com/milosladni/sunxi-tvin2jpeg_h264

[Andrey Lebedev]

On 8/6/20 8:19 PM, Milos Ladni wrote:
> Anyway i also tried to load Lima driver and i got /dev/dri/card0, 
> card1  and renderD128 but i am not sure how to test it.
>
> When i try to load xwindow or kmscube i get following error message:
> root@olinuxino:~/workspace# kmscube
> gbm: failed to open any driver (search paths
> /usr/lib/arm-linux-gnueabihf/dri:\$${ORIGIN}/dri:/usr/lib/dri)
> gbm: Last dlopen error: /usr/lib/dri/sun4i-drm_dri.so: cannot open
> shared object file: No such file or directory
> failed to load driver: sun4i-drm

Unfortunately I can't help with firefox or video decoding, but for
running kmscube (and making sure lima actually works), you also need
"mesa" (preferably not older than 20.0.0) with "lima" and "kmsro"
gallium drivers compiled.

Also, one gotcha is that most apps expect /dev/dri/card0 to be a display
device (IIRC), but sometimes kernel assigns /dev/dri/card0 to be lima
device. If that's the case, you'll need to specify device explicitly
with "-d" option for kmscube.

--
Andrey Lebedev aka -.- . -.. -.. . .-.
Software engineer
Homepage: http://lebedev.lt/