Bazel Buildbarn: a remote execution implementation written in Go

[e...@nuxi.nl]

Hello there,

Just a tiny message to announce a project I've been working on over the last two weeks, called Bazel Buildbarn. Bazel Buildbarn is an implementation of a buildfarm for Bazel, using the Remote Execution API (REAPI). Sources can be found on GitHub:

https://github.com/EdSchouten/bazel-buildbarn

Compared to Bazel Buildfarm, the implementation available through the Bazel GitHub organization, it differs in the following areas:

- It's written in Go instead of Java.

- In addition to having a scheduler process and a pool of workers, Bazel Buildbarn has a pool of frontend processes. The reasoning behind this is that I've observed that services like ByteStream can be pretty CPU/memory/network intensive, whereas the actual scheduling load is fairly low. By putting stateless frontends in front of the scheduler, it's a lot easier to scale out.

- Workers use UNIX user ID based privilege separation for basic build sandboxing. The worker process runs as root (UID 0), whereas the build is performed as user build (UID 1). This allows the worker to hardlink files into the build environment, while ensuring that the build step doesn't corrupt the on-disk cache.

- The CAS and AC may be stored in S3-compatible buckets (e.g., AWS S3, Minio).

The code is still fairly experimental and poorly documented. That said, feel free to browse through it and let me know what you think. I happen to be at PromCon @ Google Munich next month. Love to say hi!

Ed Schouten

[Jakob Buchgraber]

Hi Ed,

thanks for sharing! I'll make sure to take a look and I'd be happy to meet up during PromCon!

Let's grab launch together or so?

Best,

Jakob

Jakob Buchgraber

Software Engineer

Google Germany GmbH

Erika-Mann-Straße 33

80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Diese E-Mail ist vertraulich. Falls sie diese fälschlicherweise erhalten haben sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail an die falsche Person gesendet wurde.

    

This e-mail is confidential. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/4e706829-6893-41ea-b8bf-a45f7db128d7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.