Distributed build and Android framework

[Jérôme Barois]

Hello,

My PC is quite old and building ICS with it takes a _very_ long time.
As there is a powerful server on my network, I consider using it
instead.

I thought of 3 possibilities:

- share a NFS folder with the Android sources on my PC, mount it on
the server and build from an SSH session on the server
-> this works but it's still very slow (because of NFS...)

- put the Android sources directly on the server, and build from
there. Share this folder over NFS and mount it on my PC to do the
"file editing" part with my favourite tools.
-> this works but it uses a lot of space on the server

- find a way to deploy distcc (or any other distributed build tool) on
my system. I know I'd still have to compile the java parts locally,
but this would already greatly improve my build time.
This is where I'm currently stuck, because Android build uses
"prebuilt" compilation binaries, which make things a lot more
complicated. To make things even more complicated, I'm cross compiling
for ARM, my PC is x86 and the server is x86-64...

Did somenone ever succeed in setting up a distcc system to build the
Android platform?

[Jean-Baptiste Queru]

We've been looking into distributed compilation, but we found that the
Android source tree doesn't play nice with that. Specifically, there
are very many include paths, which cause very many remote stat()s,
which in our experience result in slower builds than when building
locally on a single of our workstations.

Your best bet is probably do to everything server-side and to use the
output over NFS. You'll indeed need about 25GB of disk space on the
server. The binaries used by AOSP are all 32-bit (except the
clearsilver JNI library, but it's only used while compiling, you don't
need it to run the emulator or flash a device).

JBQ

2011/12/5 Jérôme Barois <jerome...@gmail.com>:

> --
> You received this message because you are subscribed to the "Android Building" mailing list.
> To post to this group, send email to android-...@googlegroups.com
> To unsubscribe from this group, send email to
> android-buildi...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/android-building?hl=en
>

--
Jean-Baptiste M. "JBQ" Queru
Software Engineer, Android Open-Source Project, Google.

Questions sent directly to me that have no reason for being private
will likely get ignored or forwarded to a public forum with no further
warning.

[Jérôme Barois]

Ok, that's what I was afraid of. Anyway, thanks for your quick answer.

Btw, as I'm now trying to build on the server, I tried a simple "make -
j" and was surprised to see the build end up very quickly without
errors, but also without having compiled anything (or very little...).
An even simpler "make" works fine though.

Is there any reason why parallel build shouldn't work on Android
compilation?

On 5 déc, 17:28, Jean-Baptiste Queru <j...@android.com> wrote:
> We've been looking into distributed compilation, but we found that the
> Android source tree doesn't play nice with that. Specifically, there
> are very many include paths, which cause very many remote stat()s,
> which in our experience result in slower builds than when building
> locally on a single of our workstations.
>
> Your best bet is probably do to everything server-side and to use the
> output over NFS. You'll indeed need about 25GB of disk space on the
> server. The binaries used by AOSP are all 32-bit (except the
> clearsilver JNI library, but it's only used while compiling, you don't
> need it to run the emulator or flash a device).
>
> JBQ
>

> 2011/12/5 Jérôme Barois <jerome.bar...@gmail.com>:

[Jean-Baptiste Queru]

If you literally did a "make -j", you've pretty much fork-bombed your
server: the Android build system is extremely good at parallelizing
tasks, so you would have quickly ended up with hundreds of parallel
copies of gcc competing for all sorts of resources, and that might
have caused the kernel to fight back and kill all that.

I do my builds at -j32 (dual-quad-core HT), and typically recommend
using 2x the number of hardware threads your CPUs can run overall. In
my experience, going beyond that increases RAM usage for the working
set at the expense of the disk cache, and causes bottlenecks on I/O.
You want to be sure that you have enough resources available (RAM,
processes, file descriptors...)

JBQ

2011/12/5 Jérôme Barois <jerome...@gmail.com>:

[Dominic Binks]

For building over NFS we've found that gigabit ethernet makes it
bearable. 100baseT is hopeless though.

On 12/5/2011 8:28 AM, Jean-Baptiste Queru wrote:
> We've been looking into distributed compilation, but we found that the
> Android source tree doesn't play nice with that. Specifically, there
> are very many include paths, which cause very many remote stat()s,
> which in our experience result in slower builds than when building
> locally on a single of our workstations.
>
> Your best bet is probably do to everything server-side and to use the
> output over NFS. You'll indeed need about 25GB of disk space on the
> server. The binaries used by AOSP are all 32-bit (except the
> clearsilver JNI library, but it's only used while compiling, you don't
> need it to run the emulator or flash a device).
>
> JBQ
>

> 2011/12/5 J�r�me Barois<jerome...@gmail.com>:

--
Dominic Binks: dbi...@codeaurora.org
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum