Android NDK vs CodeSourcery performances

[teknoraver]

Hi people,

I did some benchmarks and I noticed that executables compiled with the
Codesourcery toolchain are noticeably faster than the ones compiled
with the Android NDK.

here are teh results of the classic FLOPS benchmark:


default:
/data/matteo # ./flops-ndk

FLOPS C Program (Double Precision), V2.0 18 Dec 1992

Module Error RunTime MFLOPS
(usec)
1 -7.6739e-13 0.1164 120.2685
2 -5.7021e-13 0.0714 98.0306
3 -2.4314e-14 0.0955 177.9959
4 6.8501e-14 0.1005 149.2421
5 -1.6320e-14 0.2795 103.7741
6 1.3961e-13 0.2213 131.0503
7 -3.6152e-11 0.2242 53.5285
8 9.0483e-15 0.1922 156.0976

Iterations = 256000000
NullTime (usec) = 0.0000
MFLOPS(1) = 114.9072
MFLOPS(2) = 88.8950
MFLOPS(3) = 118.7444
MFLOPS(4) = 149.3046
/data/matteo # ./flops-cs

FLOPS C Program (Double Precision), V2.0 18 Dec 1992

Module Error RunTime MFLOPS
(usec)
1 -7.6739e-13 0.1159 120.7954
2 -5.7021e-13 0.0746 93.8711
3 -2.4314e-14 0.0946 179.6862
4 6.8501e-14 0.0936 160.3340
5 -1.6320e-14 0.1780 162.8785
6 1.3961e-13 0.1168 248.2113
7 -3.6152e-11 0.2275 52.7382
8 9.0483e-15 0.1259 238.2134

Iterations = 256000000
NullTime (usec) = 0.0000
MFLOPS(1) = 111.2392
MFLOPS(2) = 100.7222
MFLOPS(3) = 153.2934
MFLOPS(4) = 211.1675

vfp:
/data/matteo # ./flops-ndk

FLOPS C Program (Double Precision), V2.0 18 Dec 1992

Module Error RunTime MFLOPS
(usec)
1 2.8422e-14 0.1260 111.0973
2 2.5047e-13 0.0768 91.1495
3 -7.6605e-15 0.0960 177.0545
4 2.2771e-13 0.0934 160.6695
5 3.8858e-14 0.2164 134.0072
6 7.5495e-15 0.1826 158.8361
7 -1.1369e-13 0.2286 52.4949
8 1.2612e-13 0.1819 164.9485

Iterations = 128000000
NullTime (usec) = 0.0000
MFLOPS(1) = 108.3333
MFLOPS(2) = 93.3231
MFLOPS(3) = 129.7958
MFLOPS(4) = 164.3109
/data/matteo # ./flops-cs

FLOPS C Program (Double Precision), V2.0 18 Dec 1992

Module Error RunTime MFLOPS
(usec)
1 -7.6739e-13 0.1153 121.4092
2 -5.7021e-13 0.0740 94.6146
3 -2.4314e-14 0.0950 178.8738
4 6.8501e-14 0.0932 160.9388
5 -1.6320e-14 0.1803 160.8319
6 1.3961e-13 0.1181 245.5839
7 -3.6152e-11 0.2247 53.3982
8 9.0483e-15 0.1275 235.2941

Iterations = 256000000
NullTime (usec) = 0.0000
MFLOPS(1) = 111.8374
MFLOPS(2) = 101.2753
MFLOPS(3) = 153.0110
MFLOPS(4) = 209.7605

neon:
/data/matteo # ./flops-ndk

FLOPS C Program (Double Precision), V2.0 18 Dec 1992

Module Error RunTime MFLOPS
(usec)
1 2.8422e-14 0.1250 112.0000
2 2.5047e-13 0.0752 93.0426
3 -7.6605e-15 0.0946 179.6862
4 2.2771e-13 0.0935 160.4010
5 3.8858e-14 0.2155 134.5415
6 7.5495e-15 0.1843 157.3548
7 -1.1369e-13 0.2319 51.7520
8 1.2612e-13 0.1794 167.2474

Iterations = 128000000
NullTime (usec) = 0.0000
MFLOPS(1) = 110.4547
MFLOPS(2) = 92.6564
MFLOPS(3) = 129.8680
MFLOPS(4) = 164.9158
/data/matteo # ./flops-cs

FLOPS C Program (Double Precision), V2.0 18 Dec 1992

Module Error RunTime MFLOPS
(usec)
1 -7.6739e-13 0.1155 121.1629
2 -5.7021e-13 0.0745 94.0189
3 -2.4314e-14 0.0946 179.7604
4 6.8501e-14 0.0939 159.6674
5 -1.6320e-14 0.1780 162.8785
6 1.3961e-13 0.1169 248.1283
7 -3.6152e-11 0.2259 53.1212
8 9.0483e-15 0.1267 236.7448

Iterations = 256000000
NullTime (usec) = 0.0000
MFLOPS(1) = 111.3882
MFLOPS(2) = 101.1595
MFLOPS(3) = 153.4256
MFLOPS(4) = 210.5948


Can't ANdroid merge the CodeSourcery patches in the Android NDK tree?
Sure there will be a performance benefit.
Also I noticed that the executables are a bit smaller.


Cheers,
Matteo Croce

[Doug Schaefer]

That's not surprising since CodeSourcery has worked a lot on improving
the performance on ARM processors. I'd certainly want to use them.

I'm just not sure what the right strategy is to combine these with the
NDK. We already have crystax's full C++ support, and mingw-android's
port to MinGW. Do we need yet another on that brings in CodeSourcery's
patches? I guess we'll have to. But I wish these things would become
standard in the NDK itself so we can have the best toolchain available
to build our apps.

> --
> You received this message because you are subscribed to the Google Groups "android-ndk" group.
> To post to this group, send email to andro...@googlegroups.com.
> To unsubscribe from this group, send email to android-ndk...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/android-ndk?hl=en.
>
>

[Doug Schaefer]

Here's the million dollar question. How did you get the CodeSourcery
toolchain working for Android? Is this something that can be useful to
others or were you just experimenting?

I've just been looking at the latest source for their toolchain and
the build scripts. They ship a tarball of their modified gcc. Has
anyone tried to apply the Android patches to it?

Also, they do a Canadian cross from Linux to produce the MinGW
binaries for Windows. Combine that with crystax's work and the CS
performance improvements, we could end up with one hell of a
toolchain. Thoughts?

On Sat, Oct 2, 2010 at 7:53 PM, teknoraver <techn...@gmail.com> wrote:

[Onur Cinar]

I forgot to mention. Also the TOOLCHAIN_NAME should be changed to arm-2010q1 in setup.mk. 

On Mon, Oct 4, 2010 at 4:33 PM, Onur Cinar <onur....@gmail.com> wrote:

Hi,

After reading this thread, I experimented with ARM-2010q1 build of CodeSourcery (lite version). The integration was easy. Here is what I did to get it working:

Got the TAR package from:
http://www.codesourcery.com/sgpp/lite/arm/portal/release1293

- Extracted the TAR package into $NDK/build/prebuilt/linux-x86
- cd $NDK/build/toolchains
- cp -Rp arm-eabi-4.40 arm-2010q1
- cd arm-2010q1
- Edit setup.mk, and changed TOOLCHAIN_PREFIX to ... arm-none-linux-gnueabi-

To do a quick test

- export NDK_TOOLCHAIN=arm-2010q1
- ndk-build

I didn't do extensive testing, but I didn't see much performance improvement in my application, but this is highly application related.

Does this setup looks similar or is there anything I missed to tune?


Regards,

-onur

---
www.zdo.com

[Onur Cinar]

Hi,

After reading this thread, I experimented with ARM-2010q1 build of CodeSourcery (lite version). The integration was easy. Here is what I did to get it working:

Got the TAR package from:
http://www.codesourcery.com/sgpp/lite/arm/portal/release1293

- Extracted the TAR package into $NDK/build/prebuilt/linux-x86
- cd $NDK/build/toolchains
- cp -Rp arm-eabi-4.40 arm-2010q1
- cd arm-2010q1
- Edit setup.mk, and changed TOOLCHAIN_PREFIX to ... arm-none-linux-gnueabi-

To do a quick test

- export NDK_TOOLCHAIN=arm-2010q1
- ndk-build

I didn't do extensive testing, but I didn't see much performance improvement in my application, but this is highly application related.

Does this setup looks similar or is there anything I missed to tune?


Regards,

-onur

On Mon, Oct 4, 2010 at 4:22 PM, Doug Schaefer <cdt...@gmail.com> wrote:

---
www.zdo.com

[Doug Schaefer]

Cool. That's pretty much what I did when I tried it a few weeks ago.
But I didn't really trust it at the time. I'm going to take a closer
look at the Android patch and see if it really could be that easy.

[teknoraver]

Nothing of these.
I built the CodeSourcery using OpenWrt.
It is the standard OpenWrt compiler and building any ARM (or MIPS)
target builds
the CodeSourcery toolchain.
Too bad OpenWrt uses uClibc as C library so I have to link all
executables statically

[teknoraver]

Here is a more readable graphic:
https://spreadsheets.google.com/ccc?key=0Anxj-ui-2sOYdF9OR0cwSlB1NlJtaC0tVWNtVjZuSVE

[Chris Stratton]

On Oct 5, 9:15 am, teknoraver <technobo...@gmail.com> wrote:

> It is the standard OpenWrt compiler and building any ARM (or MIPS)
> target builds
> the CodeSourcery toolchain.
> Too bad OpenWrt uses uClibc as C library so I have to link all
> executables statically

The different C library may have some contribution to your results in
addition to the different compiler.

Since you seem to have run a floating point benchmark, it would also
be interesting to look at how floating point emulation is handled in
the two cases.

[teknoraver]

Enabled in both case.
If I disable floating point I get very very bad results, let's say 50x
slower