Chromium Build Support for the Remote Execution API

[Dirk Pranke]

bcc: infra-a...@chromium.org ...

Hi all, 

Chrome has always believed in the value of fast, remote builds for the Chromium project. We pioneered this by creating Goma, a remote build caching layer that has been available to Googlers, some Chromium project members, and as open source server and client libraries to all. As part of our next step in making faster builds more widely available and easier to deploy, Chrome is adding support for a new client build tool that uses the Remote Execution API. The RE API is what underpins modern Goma today. With this change, we will simplify our usage of the API, and reduce the need for a separate standalone server.

Support for the new client build tool, which will be open source, will be added to the Chromium codebase over the next several quarters. While implementing direct support for the Remote Execution API within our build system will take some time, we wanted to provide an early heads-up to the Goma community that we are planning to move away from our usage of Goma codebase in the future. If you are dependent on the Goma Client or Server, you should be aware that after Chromium transitions to using Remote Execution directly, we intend to drop support for Goma in Chrome/Chromium build scripts and CI at a later date.

We are hopeful that transitioning our remote build support to a public standard will enable many more Chromium developers to be able take advantage of faster builds via any provider of the Remote Execution API, without the need to support bespoke infrastructure.

If you have questions or concerns, please reach out to infr...@chromium.org.

Thanks!

-- Dirk

[Mostyn Bramley-Moore]

On 9/11/20 9:27 pm, 'Dirk Pranke' via infra-dev wrote:
> Support for the new client build tool, which will be open source, will be added to the Chromium codebase over the next several quarters. While implementing direct support for the Remote Execution API within our build system will take some time, we wanted to provide an early heads-up to the Goma community that we are planning to move away from our usage of Goma codebase in the future. If you are dependent on the Goma Client or Server, you should be aware that after Chromium transitions to using Remote Execution directly, we intend to drop support for Goma in Chrome/Chromium build scripts and CI at a later date.

Will the new REAPI client live in its own repository (or in the main chromium repository)?


-Mostyn.

[Moore, Justin1]

Can Chrome/Chromium OS builds benefit from this in any way?

Regards,

Justin

---

From: 'Dirk Pranke' via infra-dev <infr...@chromium.org>
Sent: Monday, November 9, 2020 12:27 PM
To: infra-dev <infr...@chromium.org>
Subject: Chromium Build Support for the Remote Execution API

 

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CAEoffTBBzbYfcLd17vOgqYrUJhBcp7-1XnywybFrPbGfrAFdZA%40mail.gmail.com.

[Dirk Pranke]

It will live in its own repository.

Answering another question asked on the #goma Slack channel, I believe the new client will speak the existing RE apis, so it should be fully possible to configure it to talk to other existing (already open source) backends like Buildbarn. I will confirm that, though, and report back.

[Dirk Pranke]

Are you asking about Chromium and ChromiumOS separately, or Chromium-on-ChromiumOS?

If the former, any place that currently uses goma will be able to use the new thing, so definitely Chromium and probably ChromiumOS in at least some places.

If the latter, that's just a special case of "Chromium", so, yes, there too :).

-- Dirk

[Moore, Justin1]

The former, is there any documentation that you can point me to that shows how Goma can be leveraged

for applicable portions of the ChromiumOS build.

Thank you.

Regards,

Justin

---

From: Dirk Pranke <dpr...@google.com>
Sent: Monday, November 9, 2020 1:48 PM
To: Moore, Justin1 <justin...@intel.com>
Cc: infra-dev <infr...@chromium.org>
Subject: Re: Chromium Build Support for the Remote Execution API

 

[Dirk Pranke]

I'm not aware of any public documentation at the moment (though there may be some I'm not aware of).

I know the teams have been poking at this internally, so I'll reach out to them and see what I can dig up for you.

-- Dirk

[Moore, Justin1]

That would be much appreciated Dirk.

Thank you.

Regards,

Justin

---

From: Dirk Pranke <dpr...@google.com>
Sent: Monday, November 9, 2020 1:53 PM