A question about repository rules and migrating to modules

[Ajith Ramanathan]

Hi.

I'm trying to migrate our build to bzlmod.  One of our dependencies (bazel-latex) doesn't use modules, and makes use of a repository rule to set up its dependencies:

# Root WORKSPACE.bazel

http_archive(name="bazel-latex", ... )

load("@bazel_latex//:repositories.bzl", "latex_repositories")

latex_repositories()

I have a custom registry for our internal dependencies, and I'm using an overlay to add a MODULE.bazel to bazel-latex.

As I understand it, I have two options to migrate this to bzlmod:

1. I can manually introduce every dependency pulled in by latex_repositories() into the overlaid module file, or,

2. I can invoke latex_repositories() via a module extension.

I prefer the latter since it allows me to gracefully capture any changes to the set of dependencies brought in by latex_repositories().  (Indeed, option 1 isn't super-easy since the function's behavior depends on the platform, and there are a lot of dependencies to define.)

I'm taking my cue from these instructions: https://bazel.build/external/migration#fetch-deps-module-extensions.  My reading of it is as follows:

1. Add an extensions.bzl to bazel-latex (via an overlay) that loads //:repositories.bzl and wraps latex_repositories in a module extension:

# extensions.bzl

load("//:repositories.bzl", "latex_repositories")

def _latex_deps_impl(ctx):

    latex_repositories()

latex_deps = module_extension(implementation = _latex_deps_impl)

2. In the overlaid MODULE.bazel, invoke the extension:

# MODULE.bazel

latex_deps = use_extension("//:extensions.bzl", "latex_deps")

use_repo(latex_deps, "some_random_name")

To test this I cloned the bazel-latex repo, added the two files above, and tried building bazel-latex.  I expected the build to succeed, but instead it complains that the dependencies specified by latex_repositories() aren't defined e.g.:

ERROR: no such package '@@[unknown repo 'texlive_texmf__texmf-dist__tex__luatex__luaotfload' requested from @@]//': The repository '@@[unknown repo 'texlive_texmf__texmf-dist__tex__luatex__luaotfload' requested from @@]' could not be resolved: No repository visible as '@texlive_texmf__texmf-dist__tex__luatex__luaotfload' from main repository

Interestingly, when playing around with this, I had a typo in use of use_extension, where I had misspelled "latex_deps" as "latek_deps".  The build complained in the same way (no repo visible from the main repo), which suggests that it wasn't actually looking for the extension correctly.

I did try approach 1 of explicitly pulling in the repos added by latex_repositories (which did seem to work), but there are almost 200 such dependencies, and that way lies madness, I believe.  I've reached the limit of my ability to debug what's going on, so any advice would be appreciated.

[Xudong Yang]

The build complains about missing repos because it wasn't told how to resolve any of them. Module extensions work lazily: an extension isn't evaluated until Bazel knows a repo the extension generates is being requested.

In this case, you told Bazel that the `latex_deps` extension generates a repo named `some_random_name`, so Bazel knows that when a build refers to `@some_random_name//pkg:target`, the `latex_deps` extension should be evaluated.

Obviously, `some_random_name` isn't an actual repo, and thus never requested, so `latex_deps` is never evaluated. This is also why you ran into the same error with the typo -- `latek_deps` was also never evaluated, so Bazel doesn't even know that it doesn't actually exist.

So what _would_ work? Yes indeed, you'd need to list the 200 repositories. That is indeed madness, but 1) if you make the extension return `module_ctx.extension_metadata(root_module_direct_deps = "all")` (link), then `bazel mod tidy` will help you manage the list; 2) the real solution is to migrate bazel-latex to Bzlmod anyway.

"  YÁNG Xùdōng (杨旭东)

#  SURNAME Givenname

=  Software Engineer

@  Google New York

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CAPezz0FPfc7GY4CH6mm07vPxOw34BLdQ3PRdzP_VaNWsAp3JyQ%40mail.gmail.com.

[Filip Filmar]

[...] the real solution is to migrate bazel-latex to Bzlmod anyway.

I got to ask: does it even make sense to attempt migrating to bazel modules a repo which itself may depend on one or more layers of deps which aren't migrated yet?

F
 

[Ajith Ramanathan]

Thanks Xudong, for the help!

@Filip: Yeah, it is a significant errand, but I have some time to spare...