Android on x86-64

634 views
Skip to first unread message

John Dallman

unread,
May 28, 2021, 5:17:30 AM5/28/21
to andro...@googlegroups.com
Preface: I am not producing an Android app. I work for a software component business, creating shared libraries, compiled from C and C++ code, for use in third-party customers' apps. I test my libraries in a command-line test harness, which I run in the ADB shell. 

At present, I am only producing software for 64-bit ARM, because none of the customers wanted 32-bit code. Now, however, a customer is asking for x86-64 code, because it will run faster in the emulator, and they apparently have automated testing for their app in the emulator, but not on-device. 

Things I'd like to check, before I do this: 
  • As far as I've been able to discover, there are very few x86-64 Android devices on the market. Correct? 
  • However, most current Chromebooks are x86-64-based, and Android apps that  use native code will run faster on them if they include x86-64 versions of the native code. Correct? 
  • It would be possible to produce an ASOP-based x86-64 OS for conventional PCs, but nobody seems to do this at present. Correct? I'm not going to try: not my field of expertise. 
  • Testing x86-64 code in the emulator is good for testing UI and logic development, but can't provide the same assurance that the ARM64 code will work as testing ARM64 code would, because there's always the possibility of different compiler behavior.  Correct?
  • Google has no announced plans to drop x86-64 support from NDK, and it does not seem likely that such plans would happen in the near future.  Correct?
Thanks in advance,

John Dallman

Andrew Esh

unread,
May 28, 2021, 10:09:30 AM5/28/21
to andro...@googlegroups.com
I work in a group that produces shared libraries for TV OEMs and other device manufacturers. We mainly target Linux and Android. Our products have to be cross-compilable for x86_64 and ARM (both ARM-32 and ARM-64). Most of our testing is done on x86_64 because that is the architecture of our build machines, and all our Linux targets. We test our Android builds directly on Android devices. We build from one product source code base, to avoid dual maintenance. We do all our building and x86_64 testing in Virtual Machines running Linux.

When building a release product, we have to test on a sample of platforms from each architecture. There is always a chance that something will compile and run perfectly on x86_64, and the ARM-64 build will have a few problems. Compiles are different.

I can't speak directly to your question about testing on ADB, or building for x86_64 in the NDK. Those are combinations we don't cover. All I can say is: It is possible to get all of those platforms built with common source code. The test location and NDK target issues are minor problems that I am sure are solvable.

I've also done AOSP builds in the past. While a great deal of control can be gained by that, I am not sure it is needed to solve the problems. I have never heard of anyone trying to use an AOSP build on a standard PC.

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAH1xqgkdBMKF0%3D_wkAioBHSiLBE0Y0haPYNcQUNvViCgcYKHXQ%40mail.gmail.com.

Alias R. Cummins

unread,
May 30, 2021, 3:21:11 AM5/30/21
to andro...@googlegroups.com
Assuming that it's a modern android emulator, running on a processor which supports hardware assisted virtualization (most modern PCs), I think
it's very likely the ARM code will run faster. You could probably benchmark it in VMWare. Create a test case in an environment that matches theirs.

Do benchmarks first, it's likely a fools errand IMHO.

Hope this helps
A

On Fri, 28 May 2021 at 13:42, Alias R. Cummins <alia...@gmail.com> wrote:


On Fri, 28 May 2021 at 11:17, John Dallman <jgdats...@gmail.com> wrote:
Preface: I am not producing an Android app. I work for a software component business, creating shared libraries, compiled from C and C++ code, for use in third-party customers' apps. I test my libraries in a command-line test harness, which I run in the ADB shell. 

At present, I am only producing software for 64-bit ARM, because none of the customers wanted 32-bit code. Now, however, a customer is asking for x86-64 code, because it will run faster in the emulator, and they apparently have automated testing for their app in the emulator, but not on-device. 


Which emulator? This is the key question. Android emulators actually run arm code faster in many ways. Android is essentially an SE Linux distro, so it depends on your deployment target.
https://en.wikipedia.org/wiki/Hardware-assisted_virtualization
 
Things I'd like to check, before I do this: 
  • As far as I've been able to discover, there are very few x86-64 Android devices on the market. Correct?

Yes they are quite rare nowadays.
 
  • However, most current Chromebooks are x86-64-based, and Android apps that  use native code will run faster on them if they include x86-64 versions of the native code. Correct?
Probably not. It will depend on the hardware virtualisation in the CPU. The notion that x86 emulation is faster isn't really true anymore, but it depends on which emulator you're using.
  • It would be possible to produce an ASOP-based x86-64 OS for conventional PCs, but nobody seems to do this at present. Correct? I'm not going to try: not my field of expertise.

Yes.
  • Testing x86-64 code in the emulator is good for testing UI and logic development, but can't provide the same assurance that the ARM64 code will work as testing ARM64 code would, because there's always the possibility of different compiler behavior.  Correct?
Depends on the emulator. Are you using a linux VM? Virtualbox? VMWare? The emulator itself will make more of a difference in speed than the native code.
  • Google has no announced plans to drop x86-64 support from NDK, and it does not seem likely that such plans would happen in the near future.  Correct?
No but they seem to be edging away from it slowly but surely.

Thanks in advance,

John Dallman

Alias R. Cummins

unread,
May 30, 2021, 3:21:11 AM5/30/21
to andro...@googlegroups.com
I've built AOSP on a standard PC, if you have the time and a good grasp of CMake it's pretty easy.

Alias R. Cummins

unread,
May 30, 2021, 3:21:11 AM5/30/21
to andro...@googlegroups.com
On Fri, 28 May 2021 at 11:17, John Dallman <jgdats...@gmail.com> wrote:
Preface: I am not producing an Android app. I work for a software component business, creating shared libraries, compiled from C and C++ code, for use in third-party customers' apps. I test my libraries in a command-line test harness, which I run in the ADB shell. 

At present, I am only producing software for 64-bit ARM, because none of the customers wanted 32-bit code. Now, however, a customer is asking for x86-64 code, because it will run faster in the emulator, and they apparently have automated testing for their app in the emulator, but not on-device. 


Which emulator? This is the key question. Android emulators actually run arm code faster in many ways. Android is essentially an SE Linux distro, so it depends on your deployment target.
https://en.wikipedia.org/wiki/Hardware-assisted_virtualization
 
Things I'd like to check, before I do this: 
  • As far as I've been able to discover, there are very few x86-64 Android devices on the market. Correct?

Yes they are quite rare nowadays.

  • However, most current Chromebooks are x86-64-based, and Android apps that  use native code will run faster on them if they include x86-64 versions of the native code. Correct?
Probably not. It will depend on the hardware virtualisation in the CPU. The notion that x86 emulation is faster isn't really true anymore, but it depends on which emulator you're using.
  • It would be possible to produce an ASOP-based x86-64 OS for conventional PCs, but nobody seems to do this at present. Correct? I'm not going to try: not my field of expertise.

Yes.
  • Testing x86-64 code in the emulator is good for testing UI and logic development, but can't provide the same assurance that the ARM64 code will work as testing ARM64 code would, because there's always the possibility of different compiler behavior.  Correct?
Depends on the emulator. Are you using a linux VM? Virtualbox? VMWare? The emulator itself will make more of a difference in speed than the native code.
  • Google has no announced plans to drop x86-64 support from NDK, and it does not seem likely that such plans would happen in the near future.  Correct?
No but they seem to be edging away from it slowly but surely.

Thanks in advance,

John Dallman

enh

unread,
Jun 1, 2021, 11:44:11 AM6/1/21
to android-ndk
(since there seems to be a lot of confusion about emulation and virtualization from others who've responded...)

On Fri, May 28, 2021 at 2:17 AM John Dallman <jgdats...@gmail.com> wrote:
Preface: I am not producing an Android app. I work for a software component business, creating shared libraries, compiled from C and C++ code, for use in third-party customers' apps. I test my libraries in a command-line test harness, which I run in the ADB shell. 

At present, I am only producing software for 64-bit ARM, because none of the customers wanted 32-bit code. Now, however, a customer is asking for x86-64 code, because it will run faster in the emulator, and they apparently have automated testing for their app in the emulator, but not on-device. 

Things I'd like to check, before I do this: 
  • As far as I've been able to discover, there are very few x86-64 Android devices on the market. Correct? 
you can actually work out the numbers for yourself in the Play Developer Console, if you play with the controls that let you allow/deny various devices :-) 
  • However, most current Chromebooks are x86-64-based,
by number of SKUs or by volume? higher-end Chromebooks are usually x86-64 while the cheapest ones are usually ARM.
  • and Android apps that  use native code will run faster on them if they include x86-64 versions of the native code. Correct? 
ChromeOS uses dynamic binary translation. so your ARM code gets converted to x86 code, and then that's run. there's some bridging between ARM/bionic native types and x86/glibc. so actual performance (and correctness) will depend on  your specific code. (just like when we first added a JIT to Dalvik a decade ago --- most apps of the time didn't show much of a difference because they were spending  all their time in the OS' C++ graphics code anyway. but if you had a chess program or mandelbrot or something, you'd notice.)

"test it and see" :-)
  • It would be possible to produce an ASOP-based x86-64 OS for conventional PCs, but nobody seems to do this at present. Correct? I'm not going to try: not my field of expertise. 
i think there are a few projects that do.

perhaps more interestingly, the main impediment to  running x86-64/bionic code on your desktop is that the PT_INTERP in the  ELF header says "/system/bin/linker64". if you set up a chroot or symlinks on your desktop so that you have /system/bin and /system/lib64 you can actually run command-line stuff just fine on the host. we sometimes do this for the bionic tests, for example.

the big saving here is that you don't have to transfer code via adb.

some assembly required (in the IKEA sense), but see these scripts that the two parts we use to run the bionic  tests  on the desktop:



btw, if you're using gtest for your unit tests, you might also want to check out our parallel test runner: https://android.googlesource.com/platform/system/testing/gtest_extras/+/refs/heads/master

that parallelism (plus the massive memory bandwidth) lets us run all the bionic tests in <2s on our desktops vs 30+s on the device (not counting the transfer time, which is larger for the device than the 2s total for the desktop).
  • Testing x86-64 code in the emulator is good for testing UI and logic development, but can't provide the same assurance that the ARM64 code will work as testing ARM64 code would, because there's always the possibility of different compiler behavior.  Correct?
correct. also things like alignment/packing and other typical arch differences, or if you have any arch-specific assembler. 
  • Google has no announced plans to drop x86-64 support from NDK, and it does not seem likely that such plans would happen in the near future.  Correct?
correct. the emulators right now are all x86-64 because that's what everyone's desktop is, so that's how you get the best: by having the same instruction set for guest and host so that you can use virtualization.

Thanks in advance,

John Dallman

John Dallman

unread,
Jun 1, 2021, 11:50:39 AM6/1/21
to andro...@googlegroups.com
On Sun, May 30, 2021 at 8:21 AM Alias R. Cummins <alia...@gmail.com> wrote:
On Fri, 28 May 2021 at 11:17, John Dallman <jgdats...@gmail.com> wrote:
At present, I am only producing software for 64-bit ARM, because none of the customers wanted 32-bit code. Now, however, a customer is asking for x86-64 code, because it will run faster in the emulator, and they apparently have automated testing for their app in the emulator, but not on-device..
Which emulator? This is the key question. Android emulators actually run arm code faster in many ways. Android is essentially an SE Linux distro, so it depends on your deployment target.
https://en.wikipedia.org/wiki/Hardware-assisted_virtualization

The emulator supplied with the Android development tools, I presume. I do all my Android work on Linux. I also do straight Linux builds of the same libraries, but they are quite separate, and not part of this discussion. I don't understand very much about modern emulators, but I think there's a misunderstanding somewhere in my question and your reply. 

I have three potential situations here:
  1. ARM64 code for Android, running on an ARM64 Android device.
  2. The same ARM64 code for Android, running in the Android emulator on an x86-64 Linux machine. 
  3. The sane source code, compiled for x86-64 Android, running in the Android emulator on the same x86-64 machine. 
Clearly, the relative performance of cases 1 and 2 depends on the relative performance of the Android device and the Linux machine. Since my testing workload is quite large and computationally intensive, I make sure to have fast devices. Currently I have Black Shark 2 'phones, and I'm trying to buy Black Shark 4 Pros. 

However, it seems implausible to me that case 2 can be faster than case 3, but that's what you seem to be saying above. Where's the misunderstanding? 

Hardware-assisted virtualization doesn't overcome a different instruction set, AFAICS. 

For the avoidance of doubt, I am not doing builds in Android Studio, or CMake: I'm using my employer's build system for UNIX-like OSes, which gives me complete control over what I'm compiling, because I'm using Clang directly, from its command line. I am only generating ARM64 code; there is nothing adding x86-64 code to my builds. 

If I end up doing Android x86-64 builds, they will be entirely separate from the ARM64 Android builds. 

John

John Dallman

unread,
Jun 1, 2021, 11:52:48 AM6/1/21
to andro...@googlegroups.com
No grasp of CMake, and no desire to mess around with device drivers trying to get AOSP to run on a PC. I wasn't talking about building ASOSP for a device, but to run on PC hardware. 

John

On Sun, May 30, 2021 at 8:21 AM Alias R. Cummins <alia...@gmail.com> wrote:

John Dallman

unread,
Jun 1, 2021, 12:09:51 PM6/1/21
to andro...@googlegroups.com
On Tue, Jun 1, 2021 at 4:44 PM 'enh' via android-ndk <andro...@googlegroups.com> wrote:
(since there seems to be a lot of confusion about emulation and virtualization from others who've responded...)

On Fri, May 28, 2021 at 2:17 AM John Dallman <jgdats...@gmail.com> wrote:
  • As far as I've been able to discover, there are very few x86-64 Android devices on the market. Correct? 
you can actually work out the numbers for yourself in the Play Developer Console, if you play with the controls that let you allow/deny various devices :-) 

I'll settle for "not many". I found a web page on intel.com that was trying to promote Intel Android, but had only two devices listed, both clearly inadequate. 

by number of SKUs or by volume? higher-end Chromebooks are usually x86-64 while the cheapest ones are usually ARM.

OK. 
 
> most apps of the time didn't show much of a difference because they were spending  all their time in the OS' C++ 
> graphics code anyway. but if you had a chess program or mandelbrot or something, you'd notice.)

Great big mathematical modeler, no user interface in the test harness. I think we'd notice. 
 
perhaps more interestingly, the main impediment to  running x86-64/bionic code on your desktop is that the PT_INTERP in the  ELF header says "/system/bin/linker64". if you set up a chroot or symlinks on your desktop so that you have /system/bin and /system/lib64 you can actually run command-line stuff just fine on the host. we sometimes do this for the bionic tests, for example.

That is interesting! 
 
the big saving here is that you don't have to transfer code via adb.

Transferring code is not a problem, there's only about 85MB of it. 50GB of test data is a different matter. With a few symlinks, I ought to be able to get the testing to read the test data off the server, rather than making copies. 
 
> btw, if you're using gtest for your unit tests ...

Thanks, but I'm not. The test harness binds the API as keywords in a LISP interpreter, letting us write the tests in LISP. 

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.

enh

unread,
Jun 1, 2021, 12:19:43 PM6/1/21
to android-ndk
On Tue, Jun 1, 2021 at 8:50 AM John Dallman <jgdats...@gmail.com> wrote:

On Sun, May 30, 2021 at 8:21 AM Alias R. Cummins <alia...@gmail.com> wrote:
On Fri, 28 May 2021 at 11:17, John Dallman <jgdats...@gmail.com> wrote:
At present, I am only producing software for 64-bit ARM, because none of the customers wanted 32-bit code. Now, however, a customer is asking for x86-64 code, because it will run faster in the emulator, and they apparently have automated testing for their app in the emulator, but not on-device..
Which emulator? This is the key question. Android emulators actually run arm code faster in many ways. Android is essentially an SE Linux distro, so it depends on your deployment target.
https://en.wikipedia.org/wiki/Hardware-assisted_virtualization

The emulator supplied with the Android development tools, I presume. I do all my Android work on Linux. I also do straight Linux builds of the same libraries, but they are quite separate, and not part of this discussion. I don't understand very much about modern emulators, but I think there's a misunderstanding somewhere in my question and your reply. 

I have three potential situations here:
  1. ARM64 code for Android, running on an ARM64 Android device.
  2. The same ARM64 code for Android, running in the Android emulator on an x86-64 Linux machine. 
  3. The sane source code, compiled for x86-64 Android, running in the Android emulator on the same x86-64 machine. 
Clearly, the relative performance of cases 1 and 2 depends on the relative performance of the Android device and the Linux machine. Since my testing workload is quite large and computationally intensive, I make sure to have fast devices. Currently I have Black Shark 2 'phones, and I'm trying to buy Black Shark 4 Pros. 

However, it seems implausible to me that case 2 can be faster than case 3, but that's what you seem to be saying above. Where's the misunderstanding? 

Hardware-assisted virtualization doesn't overcome a different instruction set, AFAICS. 

(correct. this specifically is what i meant when i said there appears to be "some confusion" on this thread.)
 
For the avoidance of doubt, I am not doing builds in Android Studio, or CMake: I'm using my employer's build system for UNIX-like OSes, which gives me complete control over what I'm compiling, because I'm using Clang directly, from its command line. I am only generating ARM64 code; there is nothing adding x86-64 code to my builds. 

If I end up doing Android x86-64 builds, they will be entirely separate from the ARM64 Android builds. 

John

--
You received this message because you are subscribed to the Google Groups "android-ndk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.

John Dallman

unread,
Jun 1, 2021, 12:42:02 PM6/1/21
to andro...@googlegroups.com

John Dallman

unread,
Dec 16, 2021, 12:53:14 PM12/16/21
to andro...@googlegroups.com
> perhaps more interestingly, the main impediment to  running x86-64/bionic code on your 
> desktop is that the PT_INTERP in the  ELF header says "/system/bin/linker64". if you set 
> up a chroot or symlinks on your desktop so that you have /system/bin and /system/lib64 
> you can actually run command-line stuff just fine on the host. we sometimes do this for 
> the bionic tests, for example.

I'm trying this, but for a different reason. Running the CPUs at full speed for 12-18 hours a day on Android devices is causing breakdowns due to bulged batteries. I have an ARM Linux machine, so I'm trying with that.

My example executable uses few shared objects:

$ readelf --dynamic test_executable
Dynamic section at offset 0x3965dc0 contains 31 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libm.so]
 0x0000000000000001 (NEEDED)             Shared library: [libc++_shared.so]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so]

So I've created the following: 

$ ls -Rl /system/
/system/:
total 0
drwxrwxr-x 2 root root 22 Dec 16 17:14 bin
drwxrwxr-x 2 root root 76 Dec 16 17:14 lib64

/system/bin:
total 1768
-rwxr-xr-x 1 root root 1809104 Dec 16 16:50 linker64

/system/lib64:
total 6416
-rwxr-xr-x 1 root root 6235072 Dec 16 16:59 libc++_shared.so
-rwxr-xr-x 1 root root  253504 Dec 16 16:58 libc.so
-rwxr-xr-x 1 root root   12464 Dec 16 16:58 libdl.so
-rwxr-xr-x 1 root root   60088 Dec 16 16:58 libm.so

The libraries came from ndk21b/sysroot/usr/lib/aarch64-linux-android/24, since I'm compiling for API24 at present, except for libc++_shared.so, which came from ndk21b/sysroot/usr/lib/aarch64-linux-android

There wasn't a linker64 in the ndk21b tree, so I grabbed it off one of my test devices. 

This seems to start loading, but hits a SIGSEGV: 

test_executable: libc: Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xffffc046dff0 in tid 26801 (test_executable), pid 26801 (test_executable) Segmentation fault

Any ideas?  

On Tue, Jun 1, 2021 at 4:44 PM 'enh' via android-ndk <andro...@googlegroups.com> wrote:
On Fri, May 28, 2021 at 2:17 AM John Dallman <jgdats...@gmail.com> wrote:
Preface: I am not producing an Android app. I work for a software component business, creating shared libraries, compiled from C and C++ code, for use in third-party customers' apps. I test my libraries in a command-line test harness, which I run in the ADB shell. 

At present, I am only producing software for 64-bit ARM, because none of the customers wanted 32-bit code. Now, however, a customer is asking for x86-64 code, because it will run faster in the emulator, and they apparently have automated testing for their app in the emulator, but not on-device. 
perhaps more interestingly, the main impediment to  running x86-64/bionic code on your desktop is that the PT_INTERP in the  ELF header says "/system/bin/linker64". if you set up a chroot or symlinks on your desktop so that you have /system/bin and /system/lib64 you can actually run command-line stuff just fine on the host. we sometimes do this for the bionic tests, for example.

enh

unread,
Dec 16, 2021, 12:57:15 PM12/16/21
to andro...@googlegroups.com
the .so files in the NDK are a lie... just enough information for ld
to do its work.

you'll want real .so files from a compatible device build.
> --
> You received this message because you are subscribed to the Google Groups "android-ndk" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to android-ndk...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/android-ndk/CAH1xqgnTz6feFXmuHUbx_h4BBUWc23iuXcKHeXF1OMpnubs2nw%40mail.gmail.com.

John Dallman

unread,
Dec 17, 2021, 4:09:02 AM12/17/21
to andro...@googlegroups.com
Well, that explains it!

Thanks,

John

John Dallman

unread,
Dec 17, 2021, 4:23:46 AM12/17/21
to andro...@googlegroups.com
It works! Bwahahaha!

John

Dan Albert

unread,
Jan 11, 2022, 7:08:47 PM1/11/22
to android-ndk
Running the CPUs at full speed for 12-18 hours a day on Android devices is causing breakdowns due to bulged batteries.

Sorry for the very late response, but it took me until now to find the information :)

For newer Pixels (P3 and newer?) there is a battery-longevity mode that kicks in automatically after the phone has been charging for a few days: https://9to5google.com/2021/07/14/pixel-80-percent-battery-health-charging/, which should help keep the batteries from bulging under constant heavy use. I haven't been testing heavily on devices that new for very long, but I've heard from others that it made a meaningful difference.

For older devices there was the sys.retaildemo.enabled system property that did the same or similar, but I think it's only usable on rootable devices. Not sure if that works for you or not. It also might be Nexus/Pixel specific.

John Dallman

unread,
Jan 13, 2022, 8:04:14 AM1/13/22
to andro...@googlegroups.com
> Sorry for the very late response, but it took me until now to find the information :)

Thanks anyway. 

> For newer Pixels (P3 and newer?) there is a battery-longevity mode that kicks in automatically after the phone has been 
> keep the batteries from bulging under constant heavy use. I haven't been testing heavily on devices that new for very long, 
> but I've heard from others that it made a meaningful difference.

I do not have any Pixels. They are rather expensive, and don't come with enough RAM for some of my test cases. Those cases may well not be realistic for 'phone software, since they're tests for a big mathematical modeler that mainly runs on high-powered PCs. Nonetheless, it is far easier to run all the tests than to re-arrange all the testing to produce tests suites suitable for smaller devices. 

However, this does tell me that it's worth leaving battery health settings turned on for third-party devices. 

> For older devices there was the sys.retaildemo.enabled system property that did the same or similar, but I think it's only 
> usable on rootable devices. Not sure if that works for you or not. It also might be Nexus/Pixel specific.

It isn't specific to Nexus and Pixel, since my Black Sharks have it. I'll investigate setting that. 

Thanks very much,

John



Reply all
Reply to author
Forward
0 new messages