clang distributed build system

[Hochhaus, Andrew]

Hello,

I would like to setup a clang distributed build system (similar to
what could be done with distcc and gcc). I watched Nico and Hans
excellent LLVM dev meeting talk, and they provide benchmarks for the
distributed vs. local only compile. However, the talk is light on
technical details of the configuration of the distributed compile.

http://www.youtube.com/watch?v=IvL3f8xY7Uw
http://llvm.org/devmtg/2011-11/Weber_Wennborg_UsingClangInChromium.pdf

Can anyone offer pointers (or links to documentation) about how the
chromium distributed clang compile works? Is it built upon distcc? Was
a custom system built for chromium (and if so, is it open source)?

Thanks,
-Andy

[Nico Weber]

Hi Andrew,

Chromium's distributed build system is called "goma". It's similar to
pump-mode distcc, but it's google-internal and not available as open
source. We used distcc for a short while however, and that worked
mostly ok too.

Nico

[Hochhaus, Andrew]

Thanks Nico.

On Mon, Jun 11, 2012 at 11:05 AM, Nico Weber <tha...@chromium.org> wrote:
> We used distcc for a short while however, and that worked
> mostly ok too.

Can you offer any highlights of the problem areas you hit with dictcc
and clang? What was the major motivation behind developing goma
instead of the distcc solution?

Thanks,
-Andy

[Nico Weber]

On Mon, Jun 11, 2012 at 9:11 AM, Hochhaus, Andrew
<ahoc...@samegoal.com> wrote:
> Thanks Nico.
>
> On Mon, Jun 11, 2012 at 11:05 AM, Nico Weber <tha...@chromium.org> wrote:
>> We used distcc for a short while however, and that worked
>> mostly ok too.
>
> Can you offer any highlights of the problem areas you hit with dictcc
> and clang?

One minor nuisance was that clang suppresses many warnings when they
come from macros, and distcc does expands macros before sending the
file to remote machines (at least if you don't use pump mode) -- this
means you'll get lots of warnings from the remote clang. There's a
thread about this somewhere on cfe-dev.

> What was the major motivation behind developing goma
> instead of the distcc solution?

distcc needs a cluster configuration and a cluster of machines. goma
on the other hand runs sandboxed compiler binaries in google's
datacenters, where thousands of machines are available. I believe
that's the main motivation (but I'm not on the goma team, so that's
just a guess).

Nico