On Qemu, some apps ignore my input (9.0-r2)

[NovHak]

Dear group readers,

Currently I have a problem with two apps : Onitama (com.asmodeedigital.onitama) and Armello (com.leagueofgeeks.armello).

I would like to know if I'm the only one encountering this problem on the current 9.0r2 image.

It's the most obvious with Armello, where I'm stuck at the language selection screen. There's a blinking "Confirm" button at the bottom right but clicking on it does nothing.

However Onitama is a much smaller app (<100MB, Armello is >800MB), hence much easier to test for anyone. The problem there is I can't move on the game board, clicking on my pawns is ignored. However, other clicks work. This can be seen very soon in the tutorial.

Onitama being available for x86 and ARM, I reinstalled, forcing the ARM ABI, but it behaves exactly in the same way.

Armello is available for x86 and ARM too, but I didn't try ARM yet.

I'm running 9.0r2 with Qemu with the following command line :

qemu-system-x86_64 -machine q35,vmport=off -cpu host -accel kvm -smp 2 -m 4G -audiodev pa,id=pasound,timer-period=5000 -device ES1370,audiodev=pasound -device virtio-vga,virgl=on,edid=off -device virtio-mouse -device virtio-keyboard -device usb-host,vendorid=0x5986,productid=0x055c -drive file=androidx86.img,if=virtio,media=disk,format=raw -display sdl,ctrl_grab=on,window_close=off,gl=on -usb -nodefaults -monitor vc -nic user

The usb-host device is my webcam, but I don't think it's relevant. I tried changing the input, using the default keyboard and mouse, using virtio-tablet/usb-tablet/usb-wacom-tablet instead of the mouse. I tried usb-mouse and usb-keyboard too, to no avail.

Any advice ?

[Chih-Wei Huang]

Some apps only accept touchscreen input and ignore others.
You may ask the apps authors to add the support.

NovHak <nov...@gmail.com> 於 2020年4月5日 週日 下午10:04寫道：

--
Chih-Wei
Android-x86 project
http://www.android-x86.org

[NovHak]

That's what I thought too, and the reason why I tried the virtio-tablet, usb-tablet and usb-wacom-tablet devices in replacement of the mouse, but that didn't change anything. Moreover, I'm pretty sure 99% of the app developers don't even think about implementing mouse support, and it's the Android system that mimics touchscreen input when there's a mouse so that apps don't even notice a mouse is being used.

That's why I don't really know where the problem is. I would be curious to know if everyone is impacted, because if it works well for some people, that means I should be able to address that issue.

I changed virtio-vga to the plain, default VGA device too since I was thinking about some problem where selection would involve some form of ray tracing and collision detection in OpenGL that could pose problems with virglrenderer's system, but it didn't change anything either.

[Grey Christoforo]

That doesn't seem like a great approach. The vast majority of Android devices are touch screen interface devices. Why would a developer spend time re-coding and testing their apps to ensure they work with mouse input? I think maybe you're underestimating the number of apps that don't work properly with pointer type input. Op only mentioned two apps, but there are many, many more (I've come across a number not mentioned here as well) , maybe most of them are games. With the very high 3d rendering performance I'm now seeing with android-x86 both on bare metal and in emulators with virgl, this input problem is a real issue. Contacting each one of those developers doesn't seem like a reasonable approach. To play devil's advocate, why does android-x86 need the native-bridge option? Why don't we just ask all developers to release x86 versions of their apps alongside the arm ones?

Android-x86 should somehow at least have the ability to deliver touchscreen-like taps to apps with standard PC input systems. I've been looking into solving this issue for myself by editing idc files[1] by setting `touch.deviceType = touchScreen` and `touch.gestureMode = spots` and that works really great, except when it doesn't. Unfortunately this commit[2] (and all others similar to it in other branches) by Jon Doe breaks editing of IDC files by locking any absolute input device with a mouse button and REL_WHEL into being interpreted as a pointer type device (you might be surprised by how many input devices this filter catches, the touchpad on my laptop for example). Pointer type input is fine for maybe 90% of apps but it breaks maybe 10% with no possible solution other than to recompile android-x86 without these patches or find a new input device (I actually ended up recompiling QEMU so that its tablet input didn't have REL_WHEEL to fix this for myself). This commit overrides what the user has set in their .idc files (very bad) so viable solutions available to a user are very few.

I'm not really sure what the best solution here is. I understand why Jon Doe submitted his patch and why it was accepted, but somehow android-x86 really needs a way to get touch input events into apps that are not coded to work with pointer device types. I'm thinking of and testing several potential solutions and I'll write back later with some ideas. One might be a settings menu item (like the enable native bridge toggle) that somehow changes input behavior, but I'm not exactly sure how that should look yet. But maybe I can find one single solution that works for everything without a toggle. I'll write back when I've done more testing and thought about this some more. But for now, this feels to me like a major problem to me.

Maybe someone else has some good ideas about how to solve this problem?

Cheers,

~grey

[1]: https://source.android.com/devices/input/input-device-configuration-files
[2]: https://osdn.net/projects/android-x86/scm/git/frameworks-native/commits/590010a2b526df769e78974c3f8fa956f4544368

--
You received this message because you are subscribed to the Google Groups "Android-x86" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-x86...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-x86/CAKc24n0iTr7VqxShROnpdw8OcfL7uQrkYsvb2UgxN4J%2BVLxesg%40mail.gmail.com.

[NovHak]

I absolutely agree with you, a better approach would be that touchscreen and mouse inputs should be transparently unified somehow. Actually I thought it was already the case...

So now if it's really a mouse problem, I would like to configure Qemu so that it provides an emulated touch interface. I thought the *-tablet devices were doing the job, but according to what you said it's not the case. I passed wheel-axis=off to my virtio-mouse device but it didn't do the trick, I'll try again with the emulated tablet input devices just in case.

And btw, last but not necessary least, one problem with Qemu is that emulated hardware always includes a PS/2 mouse and an AT keyboard, and that doesn't seem to be configurable. In the present case, I realised that blacklisting the atkbd and psmouse modules would do the trick of hiding them to Android. I passed module_blacklist=atkbd,psmouse as a kernel parameter and it did the job !

Now I'll try what I said about disabling the wheel axis, and come back...

To unsubscribe from this group and stop receiving emails from it, send an email to andro...@googlegroups.com.

[Grey Christoforo]

Hah, pretty annoying that `wheel-axis=off` doesn't do what you think
it does, right!? I tried that too, the wheel still shows up in Android
even with this flag (guess that's a QEMU bug) :-(

Re-build qemu with
sed -i 's,#define REL_WHEEL\t\t0x08,#define REL_WHEEL\t\t0x09,g'
include/standard-headers/linux/input-event-codes.h

Then you'll lose QEMU's wheel and you'll stop getting bitten by Jon Doe's patch.

Nice tip for removing the extra mouse and keyboard! I was annoyed by that!

A couple other pointers I've learned in dealing with this you (or
anyone else) might find helpful here:
- In Android, use `dumpsys input | less` to look at how the OS sees
your input devices, ones with "Mode: pointer" under the Parameters
heading won't work in many apps.
- Even after modifying QEMU to work around the bug, you'll probably
need to set up an .idc file to get your input device working the way
you might expect:
```
# cat /system/usr/idc/QEMU_Virtio_Tablet.idc
touch.deviceType = touchScreen
#touch.deviceType = pointer
touch.gestureMode = spots
#touch.gestureMode = pointer
touch.orientationAware = 1
touch.toolSize.calibration = default
touch.pressure.calibration = default
touch.size.calibration = default
touch.orientation.calibration = none
device.internal = 1
```
This is for qemu with "-device virtio-tablet-pci"
Probably those calibration lines are not needed but I'm sure that works.
- You can use the same .idc file but with a different name:
"QEMU_QEMU_USB_Tablet.idc" if you're using QEMU's usb tablet device.
- The idc files should have the following perms to ensure they can be
read!: -rw-r--r--, with root as owner and group
- Launch qemu with `-show-cursor` because when it's working properly
android won't draw one anymore, and that's annoyong (tested working in
an sdl window, might be broken in a gtk one)
- If you're playing with .idc files, you can run `start; stop` to
reload and test your changes without needing to reboot the whole vm
For posterity, here's my whole qemu launch command:

qemu-system-x86_64 \
-drive if=pflash,format=raw,readonly,file=/usr/share/ovmf/x64/OVMF_CODE.fd \
-machine type=pc,vmport=off \
-enable-kvm \
-show-cursor \
-m 4096 \
-smp 2 \
-cpu host \
-soundhw hda \
-device virtio-tablet-pci \
-device virtio-keyboard-pci \
-device virtio-vga,virgl=on \
-serial mon:stdio \
-net nic \
-net user,hostfwd=tcp::47000-:5000 \
-display sdl,gl=on \
-hda ${HOME}/scratch/android.qcow2 \
-monitor telnet:127.0.0.1:2323,server,nowait #\
#-cdrom ${HOME}/Downloads/android-x86_64-9.0-r2-k49.iso

> To unsubscribe from this group and stop receiving emails from it, send an email to android-x86...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/android-x86/0d6b232c-d0cb-410d-ba2a-a61799005825%40googlegroups.com.

[NovHak]

Thanks for that very informative answer ! I'm going to compile Qemu today, but I'll try another change such as removing REL_WHEEL from the device event bitmap instead of shifting its position by one bit.

That patch from Jon Doe is atrocious indeed, since not only can it impact users outside of Qemu, but it's even an annoyance for people using Qemu, as long as they want proper emulation. Some would argue that the tablet devices on Qemu should not provide the REL_WHEEL event, but the funny thing is this patch was done precisely with the idea of forcing mouse interpretation for tablet inputs, hence removing REL_WHEEL would render the patch useless in the very spirit of its creator.

And thanks for pointing to the start/stop commands, I didn't know how to restart the surface flinger without restarting the whole thing, which I usually don't have to do anyway, but as you said it may become useful once I have Qemu recompiled.

> To view this discussion on the web visit https://groups.google.com/d/msgid/android-x86/0d6b232c-d0cb-410d-ba2a-a61799005825%40googlegroups.com.

[NovHak]

Finally ! My change worked, here's what I did :

--- qemu-4.2.0/hw/input/virtio-input-hid.c    2020-04-06 19:55:56.113287753 +0000
+++ qemu-4.2.0-patched/hw/input/virtio-input-hid.c    2020-04-06 20:17:33.645122601 +0000
@@ -459,7 +459,7 @@
         .size      = 2,
         .u.bitmap  = {
             0,
-            (1 << (REL_WHEEL - 8))
+            0
         },
     },{
         .select    = VIRTIO_INPUT_CFG_ABS_INFO,

When I started I immediately noticed I had no cursor, which was good news, but Android was still managing it in pointer mode, I then modified the idc file that already existed for the virtio tablet, only adding "touch.deviceType = touchScreen". Actually I added "touch.gestureMode = pointer" too since I didn't want the device to be considered as multi-touch (we're still using a mouse after all and I would prefer the system to provide as many mouse-friendly features as possible), but it was ignored. Maybe I will check that later.

And now finally the two games I mentioned, Armello and Onitama, accept my input. That being said, I would prefer that the pointer remains captured as it's the case when I emulate a mouse device, so that I don't risk clicking outside, hence I will probably stick to the mouse and use only the touch screen when it's needed, but anyway that's still good progress !

[NovHak]

After spending some more time in the code, and some testing, it turns out the problem takes place during the VM initialisation.

The wheel-axis parameter is supposed to impact the emulation at two levels :

1. During the hardware setup, where the Virtio tablet device should be setup with an event bitmap that doesn't contain any relative event, hence no REL_WHEEL event
2. During runtime, when all REL_WHEEL events received from the host should be filtered out

It turns out that while everything is in the code indeed, only step 2 actually works. There's a check on the wheel-axis property during hardware setup, but at that time it's still at its compiled-in default value, wheel-axis=on. I don't know why (yet ?), I'm not comfortable enough with the qemu source code. Changing the global runtime default value with "-global virtio-tablet-device.wheel-axis=off" exhibits the same behaviour as passing the option to the device, i.e. it remains on during the hardware setup phase.

So yes, it is clearly a bug.

The cleanest I came up with at this stage, is to change the compiled-in default for wheel-axis to false (off). And it works.

Here's the patch :

--- qemu-4.2.0/hw/input/virtio-input-hid.c    2019-12-12 18:20:47.000000000 +0000
+++ qemu-4.2.0-patched/hw/input/virtio-input-hid.c    2020-04-07 23:46:22.453444127 +0000
@@ -478,7 +478,7 @@
 };
 
 static Property virtio_tablet_properties[] = {
-    DEFINE_PROP_BOOL("wheel-axis", VirtIOInputHID, wheel_axis, true),
+    DEFINE_PROP_BOOL("wheel-axis", VirtIOInputHID, wheel_axis, false),
     DEFINE_PROP_END_OF_LIST(),
 };

 And now at last, we have a tablet device that behaves like one !