RBE - Distributed compiler system to build chromium for Windows/Linux/Mac/iOS/Android

[Manjunath E]

Hi,

We are looking for a solution to build chromium in as much less time as possible. A solution which works for all the platforms(Windows/Linux/Mac/iOS/Android) would be nice. It is OK If a single solution cannot support all but at least supports majority of these platforms.

We have gone through Goma, Bazel with BuildFarm and Buck. Below are the findings on each one of them

1. Bazel: Bazel needs BUILD file with rules for each project and creating them is not a simple task with chromium as it has huge number of projects. An automated solution is required and that is what Goma seems to be doing where parser for each compiler has been written and RBE is used to distribute the compilation task similar to Bazel. It seems the parser is written to get away with BUILD file.
2. Goma: Goma client and proxy has been made open source but the backend service is not available. But google has made that available in GCP. Spoke to google folks and found that it has certain limitations. I am not sure whether I am allowed to discuss about that here as it is available only for people who have access to alpha and seems like it is confidential.
3. Buck: Buck also needs build rule file for each project like Bazel.

What is the recommended way to build chromium in less time? Want to hear how the chromium projects are built in enterprise. Any suggestions or advices are appreciated.

Can someone who is using Goma with Remote Build Execution(RBE) explain what is the build time we can expect and how many workers are being used to achieve that? 

Is there any other alternatives to achieve what we are looking for?

Is there an alternate solution available where Goma can be used without RBE in GCP?

[Mostyn Bramley-Moore]

IMO the only feasible options are those which you can inject easily via GN, ie you should only consider compiler wrappers and not bazel/buck/meson/etc.

Goma is the most efficient compiler wrapper usable with Chromium that I am aware of. You could potentially use goma client + goma server + an opensource bazel remote execution api v2 backend implementation if you don't want to use RBE, but the goma server might require some modification to make this work. I know people have experimented with this setup, but I haven't heard the results.  I don't know the status of Mac or Windows support with opensource goma.

You could also consider distcc pump mode, distcc in regular mode, or icecc, all with or without ccache (or potentially sccache).

-Mostyn.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/a7d48d3c-ca5f-4282-99c0-617946e689c1%40chromium.org.

--

Mostyn Bramley-Moore

Vewd Software

mos...@vewd.com

[Manjunath E]

Could you please name few of the opensource bazel remote execution api v2 backend implementations? It would be helpful if you can add the people who have  experimented with this setup to this thread.

distcc, cccahe doesn't support windows. I am looking for solution which supports at least windows and linux.

To unsubscribe from this group and stop receiving emails from it, send an email to chromi...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/a7d48d3c-ca5f-4282-99c0-617946e689c1%40chromium.org.

[Mostyn Bramley-Moore]

On Thu, Sep 12, 2019 at 12:52 PM Manjunath E <rce.ma...@gmail.com> wrote:

Could you please name few of the opensource bazel remote execution api v2 backend implementations? It would be helpful if you can add the people who have  experimented with this setup to this thread.

The bazel documentation lists a few self-hosted backends:

https://docs.bazel.build/versions/master/remote-execution.html

Here's the original thread I was thinking of:

https://groups.google.com/a/chromium.org/forum/#!topic/infra-dev/-6qIunOC-BI

-Mostyn.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/c8a0e568-cb1d-46c4-a9c3-4943e303c0a9%40chromium.org.