Bzlmod and multi-repo setup

[mailto...@gmail.com]

Hi!

I have a use case where I want to share a set of toolchains and platform definitions across multiple repositories. I was wondering if the bzlmod dependency system could be used to define a common module that acts as an umbrella for all shared Starlark code, toolchain and platform registrations etc.

I am in the early phase of testing this out, but it seems to me as if this will not work as I hoped. One example is the toolchain registration process. Ideally I would want those registered by the common module, however, toolchains such as llvm seems to require that the extensions are only called by the root module.

What is your experience, is this a use-case that is out of scope for bzlmod?

Kind regards,

Jonas Hansson

[Xudong Yang]

Just FYI, the llvm case is a special constraint they introduced in that module, not a general Bzlmod limitation. Your use case seems fine enough.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/69155cb1-2ef6-4b50-922a-22d8407cd372n%40googlegroups.com.

[Fabian Meumertzheim]

Feel free to open an issue on the LLVM toolchain repo. I argued for this particular limitation since we weren't sure how to reconcile toolchains defined in different modules and failing is better than getting this wrong. But we may be able to come up with a better solution now that the toolchain has seen more real-world usage with Bzlmod.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CADMn-5aiYXV34PR7gQ%3DtPXceB%3DNoH7F_BGcU4UauZR0EO0N4og%40mail.gmail.com.

[mailto...@gmail.com]

Thanks for your replies!

I have not found a way forward on howto implement an "umbrella" module. I have experimented on creating an extension with tags that define attributes that depends the sub-module I want to facade.

However, I fail to get Bazel to download the submodule without explicit calls to the submodule extensions.

Something like the following (pseudo code):

umbrella repository, MODULE.bazel:

bazel_dep(name = "submodule_x", version = "0.0.1")

submodule_x = use_extension("@submodule_x//extensions.bzl", "submodule_x"),
use_repo(submodule_x, "submodule_x")


umbrella repository, extensions.bzl:

_attrs_submodule_x = {
    "__submodule_x": attr.label(
        default="@submodule_x//:anyfile.bzl",
        allow_single_file=True,
    ),
}

umbrella = module_extension(
    implementation = _umbrella_impl_,
    tag_classes = {
        "submodule_x": tag_class(
            attrs = _attrs_submodule_x,
        ),
    },
)

Root repository, MODULE.bazel:

bazel_dep(name = "umbrella", version = "0.0.1")

umbrella = use_extension("@umbrella//extensions.bzl", "umbrella"),
umbrella.submodule_x()
use_repo(umbrella, "submodule_x")

However, after calling the umbrella extension from the root repository, the submodule_x is not available in the externals and cannot be referenced. Do you think this should work?

Kind regards,

Jonas Hansson

[James Sharpe]

There's also an issue with the way in which the check is currently implemented in the LLVM repo; it doesn't allow any other modules to use the extension to access the toolchain defined in the main module. I've been meaning to put in a PR to fix this!

James Sharpe | Software Architecture & Security | Zenotech

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CALR1tuNq%3DzEMHxn_USzsF3usv_w947gp1kcXQGH7FTBYkuLXkQ%40mail.gmail.com.

[Xudong Yang]

It's not clear to me what you're trying to do with your example. Maybe a more concrete example in a test git repo with a concrete failure would help.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/82106140-beb9-4a44-9c54-90a1d8971048n%40googlegroups.com.

[mailto...@gmail.com]

Here is a tarball with my example (that does not work!). In a real scenario I would have several repositories {module_x, module_y, ..., module_n} that would be handled by the umbrella module.
Hope it is readable!