How to gradually move to Bazel?

[Stefano Azzalini]

I'm looking for suggestions on how to migrate ~100 repositories to monorepo using Bazel as a build system.

Switching all at once to language-specific rules such as java_binary etc, would be a too big step and hard to be widely accepted from the get-go.

I would like, at first, to keep using the existing toolchains, but still keep the "build only what has been changed" idea by building only projects that have been changed.

My plan:

1. Create new repository
2. Clone ProjectX to the monorepo
3. Configure Bazel to build ProjectX with the exising toolchain (npm build, mvn install, etc)
4. Repeat for ProjectY
5. Gradually migrate to language-specific rules once everything has settled and knowledge of Bazel reached all teams.

Thanks in advantage for your ideas and suggestions,

Stefano

[Allan Clark]

I use Bazel even in small projects that may not have another contributor: it’s just easier at this point.

Did you consider simply converting some of these polyrepos to build with Bazel, and then making the determination whether to merge to monorepo?  You can gain consistency of pipeline and tool domain with mostly copy-paste BUILD files and WORKSPACEs, seed some success and approval in simplifying the build, then discuss a merge to gain the benefits in a monorepo (that I don’t need to list in this audience!)

Good luck, Stefano, whatever course you take!

Allan

[Fredrik Medley]

I've been involved in migrations where we kept the repositories as is, but used submodules and the submodule update subscription feature in Gerrit to always have a super repo pointing to the correct versions of each submodule. Note that we reduced our tree to one level of submodules (unless totally impossible). This way, we can

* keep different access rights the repositories (submodules)

* follow a global state in super.git

* remove old repositories (submodules) to reduce fetch and checkout burden

In addition, the git-toprepo tool (still beta) integrates the submodule history client side, so on my machine I actually only work with a monorepo. This works pretty well in practice.

With this tooling, you can keep your old structure and convert to a Bazel monorepo in parallell, step by step.

https://gerrit-review.googlesource.com/Documentation/user-submodules.html

https://github.com/meroton/git-toprepo/ (pipx install git-toprepo)

/Fredrik

[Matt Brown]

In a strategy like this, are you adding BUILD.bazel files into each submodule’s repo, even though they presumably don’t have a submodule-repo-level WORKSPACE or anything in the repo (or attached build pipelines) using them?

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/ee2d2357-b5f8-4aff-a5df-f292a4240feen%40googlegroups.com.

[Fredrik Medley]

I add BUILD.bazel files in the submodules. If the submodule contain code that belongs to the monorepo code base, I don\t add MODULE.bazel. This way, `bazel test //...` works as expected. If the submodule is a vendored rules_pkg for example, I add a MODULE.bazel and use a local?path?override.

[Manuel]

On top of the great advice from Allan I would advice to focus on one language at the time IMO and experience Java, Python and Golang get the best support right now. Explore bzlmod as much as possible as nowadays it's pretty stable and useful. Include aspect.dev suggested bazelrc flags. Make sure you have some form of remote cache available, even on https, as otherwise your CI will be slow. And use gazelle to manage your build files, it gets cumbersome to do by hand very quickly. As much as the plague avoid using genrule, if you need to customize starlark go the extra mile into rules from day one.

--

You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/ca7c085e-0d8c-4eed-b6fe-113d234264ben%40googlegroups.com.