bzlmod - using data from other .bzl files in your module definition

[Jonathan Perry]

Hello!  I have what I'm sure is a silly question but...

We hve some machinery in our setup that managed versions of external dependencies - in particular it goes and resolves the set of maven jars that we want to use.

That machinery generates a .bzl file which has two top-level symbols: 'repository', which is a string (the maven repo URL), and maven_artifacts, which is a list of calls to @rules_jvm_external//:specs.bzl's maven.artifact.

In WORKSPACE land, I simply load() the symbols and then use them in 3 invocations of maven_install, generating 3 separate external repos, with 3 separate maven_install lockfiles.

What should I do to most-supportably achieve the same outcome in bzlmod land please?  I tried various things to do with defining my own module_extension, but I fried my brain trying to understand what I was doing, so thought I'd request help...

Thank you!

[Jonathan Perry]

Ooh, is it as simple as defining a module_extension that does:

maven = use_extension("@rules_jvm_external//:extensions.bzl", "maven")

maven.install(name = "first_maven_repo", ...)

maven.install(name = "second_maven_repo", ...)

maven.install(name = "third_maven_repo", ...)

and then in my top level MODULE.bazel

my_cool_extension = use_extension(...)

use_repo(my_cool_extension, "first_maven_repo", "second_maven_repo", "thrid_maven_repo")

?

[Jonathan Perry]

(The answer is 'no', since you can't invoke use_extension from inside the implementation of a module_extension)

I'm very lost now - am I supposed to move all this logic into my MODULE.bazel file?  And do I have to make my bots all generate into that single file?  I must surely be missing something obvious!

[Fabian Meumertzheim]

With Bazel 7.2.0, you can have your bot generate a `MODULE.bazel` segment that you can load with `include` (https://bazel.build/rules/lib/globals/module#include).

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/45e88aad-4848-4da4-82e1-ad8c9f9430a2n%40googlegroups.com.

[Jonathan Perry]

Right thank you, that's a bit better, as it means I can declare constants from my bot.

But fundamentally I can't use real code to drive the generation of my three repos, right?  I would need to ask rules_jvm_external to provide an API so that I can write a module_extension that just ends up driving theirs?

(Specifically, I use various starlark functions to slice, dice and remap my maven deps to satisfy some annoying downstream requirements where I export a subset of my built artifacts to a maven repo)

[Fabian Meumertzheim]

The bad experiences with WORKSPACE were at least partially being caused by the API being "code" (workspace macros). With Bzlmod, all you can do in `MODULE.bazel` is pass in "data" (extension tags), which allows for much better parallelization and decreased brittleness.

The flipside is that you can't programmatically interface with extensions. But your bot could still generate arbitrary `maven.install` tag arguments that you could stash away in a separate file. 

Since `maven.install` by itself is super cheap (no repo is fetched unless explicitly needed in a given build), you could statically add one tag for each such subset of deps needed and (hopefully) get rid of dynamic configuration alltogether.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/a3533b6c-bea1-409a-8898-d3cf2d2e49b3n%40googlegroups.com.

[Jonathan Perry]

I see, thank you.  I need to think about how to shuffle my code around in a way I'm going to be able to support.  Being unable to wrap the low level APIs is a big loss for me (though I get your point about why we are here and it makes sense).

If I declare a variable in a file

foo = 1

and then include("//:my_file.MODULE.bazel")

is it intended that 'foo' is defined?  Because I'm not seeing that but maybe I'm holding it wrong.

Thank you for your time

[Fabian Meumertzheim]

No, the segments are all evaluated individually. The registered tags and deps are combined afterwards.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/bee78b68-697f-4a0b-bb7e-ff6030758fa0n%40googlegroups.com.

[Jonathan Perry]

Oh I see, so the included files each have to do their own little use_extension/maven.install call.  Hence your statement that I could add tags for each subset of deps needed.  I understnad much more now thank you!

[Jonathan Perry]

Ugh this is super painful, not being able to use any functions at all.  I can't reuse approximately anything from my current setup...

Makes me want to generate the MODULE.bazel snippets from a pre-build step, but that would be even sadder!