Why is external library dependency works, but not data (filegroup) dependency?

[HaiXin Tie]

Hi Bazel experts,

Bzlmod works great for external library dependencies from another module, but for some reason the external data dependency (filegroups) doesn't work. I have a simple setup to demonstrate this problem. bazel_demo_server module depends on bazel_demo_schema module, which contains a python_grpc_library, a py_library, and a filegroup, all public. bazel_demo_server has no problem pulling the python_grpc_library and py_library from bazel_demo_schema module, but always complains about the data (filegroup) dependency not found. What am i missing? Or could it be a bug in Bzlmod?

Here are steps to reproduce the problem:

# check out both modules

git clone https://github.com/thx123/bazel_demo_schema
git clone https://github.com/thx123/bazel_demo_server

# verify that the server target with the data dependency works in the same module

cd bazel_demo_schema

bazel run server:server

INFO: Analyzed target //server:server (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //server:server up-to-date:
  bazel-bin/server/server
INFO: Elapsed time: 0.153s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/server/server
Python RouteGuide Server listening on :50076...

# verify that the server target in the remote module can't find the data dependency

cd ../bazel_demo_schema

bazel run server:server

INFO: Analyzed target //server:server (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //server:server up-to-date:
  bazel-bin/server/server
INFO: Elapsed time: 0.146s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/server/server
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_htie/58aa96fd58f95626101250e0326aeb83/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/server/server.runfiles/_main/server/server.py", line 141, in <module>
    serve()
  File "/private/var/tmp/_bazel_htie/58aa96fd58f95626101250e0326aeb83/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/server/server.runfiles/_main/server/server.py", line 125, in serve
    RouteGuideServicer(), server)
    ^^^^^^^^^^^^^^^^^^^^
  File "/private/var/tmp/_bazel_htie/58aa96fd58f95626101250e0326aeb83/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/server/server.runfiles/_main/server/server.py", line 65, in __init__
    self.db = resources.read_route_guide_database()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/var/tmp/_bazel_htie/58aa96fd58f95626101250e0326aeb83/execroot/_main/bazel-out/darwin_arm64-fastbuild/bin/server/server.runfiles/bazel_demo_schema~/common/resources.py", line 32, in read_route_guide_database
    with open(db_path) as route_guide_db_file:
         ^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'data/routeguide_features.json'

where 'data/routeguide_features.json' is clearly defined under data/BUILD in bazel_demo_schema:

➜  bazel_demo_schema git:(main) cat data/BUILD
package(
    default_visibility = ["//visibility:public"],
)

filegroup(
    name = "routeguide_features",
    srcs = ["routeguide_features.json"],
)

but bazel_demo_server doesn't pull in the data directory like the protos dir:

➜  bazel_demo_server git:(main) ls bazel-out/darwin_arm64-fastbuild/bin/external/bazel_demo_schema\~/
protos

Your guidance on how to fix this simple problem would be greatly appreciated!

Regards,

HaiXin

[Filip Filmar]

I am not convinced that your repo can necessarily access your dep's deps, unless your repo declares that it's using those same deps.

:)

F

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CA%2BOFMV1iu8iqhx6WhcV08R06%2Bip3S-2mR2z00LNnOzNiHpJoFQ%40mail.gmail.com.

[HaiXin Tie]

Thanks Filip. But it is declaring them all as (external) deps. See the bold text in the BUILD file below:

➜  bazel_demo_server git:(main) cat server/BUILD.bazel
load("@rules_python//python:defs.bzl", "py_binary")

py_binary(
    name = "server",
    srcs = [
        "server.py",
    ],
    data = ["@bazel_demo_schema//data:routeguide_features"],
    python_version = "PY3",
    deps = [
        "@bazel_demo_schema//common:resources",
        "@bazel_demo_schema//protos:routeguide_py_grpc",
    ],
)

Regards,

HaiXin