Problem with running Android x86 on whpx-accelerated QEMU

[Hao Lin]

Hi all,

Have anyone tried running Android x86 with qemu -accel whpx (namely using Windows Hyper-V as the hardware acceleration solution)? I found that Android x86 (specifically, I used Android 9 x86_64 official build) is stuck in boot animation when running in this manner. My own debugging shows that the problem occurs on ZygoteServer's runSelectLoop function, where the line Os.poll failed with SIGSEGV. The detailed backtrace is as below:

backtrace:

#00 pc 00000db9  [vdso:e80ab000] (__kernel_vsyscall+9)

#01 pc 00084ea6  /system/lib/libc.so (__ppoll+38)

#02 pc 0003c35a  /system/lib/libc.so (poll+90)

#03 pc 00026a41  /system/lib/libjavacore.so (Linux_poll(_JNIEnv*, _jobject*, _jobjectArray*, int)+593)

#04 pc 00081dc4  /system/framework/x86/boot-core-libart.oat (offset 0x7a000) (libcore.io.Linux.fcntlVoid [DEDUPED]+148)

#05 pc 001c265d  /system/framework/x86/boot-core-libart.oat (offset 0x7a000) (libcore.io.BlockGuardOs.poll+205)

#06 pc 0010b9d8  /system/framework/x86/boot-core-libart.oat (offset 0x7a000) (android.system.Os.poll+88)

#07 pc 00bb0e12  /system/framework/x86/boot-framework.oat (offset 0x3ca000) (com.android.internal.os.ZygoteServer.runSelectLoop+722)

#08 pc 00bad9cd  /system/framework/x86/boot-framework.oat (offset 0x3ca000) (com.android.internal.os.ZygoteInit.main+3053)

#09 pc 0062e4d2  /system/lib/libart.so (art_quick_invoke_static_stub+418)

#10 pc 000a301f  /system/lib/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+239)

#11 pc 0050862f  /system/lib/libart.so (art::(anonymous namespace)::InvokeWithArgArray(art::ScopedObjectAccessAlreadyRunnable const&, art::ArtMethod*, art::(anonymous namespace)::ArgArray*, art::JValue*, char const*)+111)

#12 pc 0050829d  /system/lib/libart.so (art::InvokeWithVarArgs(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, _jmethodID*, char*)+429)

#13 pc 003d02dd  /system/lib/libart.so (art::JNI::CallStaticVoidMethodV(_JNIEnv*, _jclass*, _jmethodID*, char*)+1085)

#14 pc 000711be  /system/lib/libandroid_runtime.so (_JNIEnv::CallStaticVoidMethod(_jclass*, _jmethodID*, ...)+62)

#15 pc 00074428  /system/lib/libandroid_runtime.so (android::AndroidRuntime::start(char const*, android::Vector<android::String8> const&, bool)+744)

#16 pc 00001bb2  /system/bin/app_process32 (main+1506)

#17 pc 000cd014  /system/lib/libc.so (__libc_init+100)

PS. My host environment is Windows 10 Home (also tried on Windows 11 Home, same problem). Also, I attempt to use Hyper-V acceleration rather than HAXM (which works fine) because HAXM's performance is rather unsatisfying especially when it comes to multithreading...

[Mauro Rossi]

Hi,

could you provide add2line -Cfe output to pinpoint the line of code involned in the crash for /system/lib/libc.so and /system/lib/libjavacore.so ?

Are you sure you are using the x86_64 iso, because the backtrace seems the one of x86 32bit build

Mauro

[Hao Lin]

Hi Mauro,

Thanks for your suggestions! I did addr2line but didn't seem to find useful info. Still, I'll post my results.

For libc.so 000cd014, it's __libc_init

bionic/libc/bionic/libc_init_dynamic.cpp:129.

For libc.so 0003c35a, it's poll

bionic/libc/bionic/poll.cpp:47.

For libc.so 00084ea6, it's __ppoll

/proc/self/cwd/bionic/libc/arch-x86/syscalls/__ppoll.S:33.

For libjavacore.so 00026a4, it's poll(pollfd*, unsigned int pass_object_size1, int)

bionic/libc/include/bits/fortify/poll.h:49.

As for the x86 stuff, it's because the ZygoteInit process is starting a secondary Zygote process which is x86-based.

I found out about this by logging inside the problematic code (in ZygoteInit's main function before it calls the runSelectLoop) and the corresponding process ID.

My logging shows that the ZygoteInit is starting the secondary process with the following command args:

ZygoteServer --abi-list=x86 --socket-name=zygote_secondary --enable-lazy-preload.

I'm pretty sure I'm using the x64 build, here's my qemu command:

qemu-system-x86_64 -accel whpx -m 2048 -smp 4 -soundhw hda -machine usb=on -device usb-kbd -device usb-tablet -boot menu=on -serial mon:stdio -net nic -net user,hostfwd=tcp::5555-:5555 -display sdl -hda androidx86_hda.img -cdrom android-x86_64-9.0-r2.iso

[Hao Lin]

I just got the problem fixed thanks to Senjosei (QEMU + WHPX + Virgl on Windows Host running Android x86 - YouTube). It turns out that one needs to enable the xsave CPU feature (i.e., xsave=on) to run Android-x86 on QEMU with WHPX acceleration.

[Hao Lin]

The specific command params for QEMU would be "-cpu kvm64,xsave=on,monitor=off" for running with WHPX acceleration. I've tested on both Intel and AMD PCs.

[Guan Zhang]

Hello:

I was wondering if you are still working on this. I got Senjosei on my AndroidEmu Discord server and we have been testing Android-x86 and Bliss OS builds on Windows 10 using QEMU + Hyper-V. Would love to get more people involved in tuning it. If anybody else is interested please feel free to join: https://discord.gg/mRpT4Qq

Thanks and Happy Holidays!

[Hao Lin]

Sure, I'm in! Thanks for the invitation! I and my partners have also developed an Android emulator based on QEMU (see  TrinityEmulator/TrinityEmulator: Trinity is an Android emulator designed to simultaneously meet the goals of good compatibility, security and efficiency with the novel notion of graphics projection space. (github.com) ), which hopefully can provide much better GPU performance over other commercial emulators. We've also got Hyper-V to work fine by providing a customised virtual CPU called android64 that enables several features like SIMD for better performance. 

[Guan Zhang]

Hi Hao:

Thanks for the link to Trinity, it looks really interesting. I'm also
going to watch your USENIX presentation to learn more about it.
Looking forward to further discussions and sharing our results back
with the Android-x86 community :-)