Bazel python proto rules overshadows "google" module

[Chuan Ye]

Hi,

I encountered an issue with the proto rules provided by rules_python. I believe adding protos that utilizes py_proto_library will overshadow the "google" module, making relevant PIP dependencies unable to be found.

I created a repo with a simple example to demonstrate this. In this example, as long as I do not include proto dependencies(py_proto_library), my app is able to locate the "google.protobuf" and "google.auth" modules correctly. However, as soon as I add the proto dependency, I would encounter this issue:

The app code is simple:

```python
import google
print(google.__file__)
import google.protobuf
print(google.protobuf.__file__)
import google.auth
print(google.auth.__file__)
```

If I have proto dependency, I get this error:

```
Traceback (most recent call last):
  File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/7e6ecf527bf713b1567f15e2f26950b5/execroot/__main__/bazel-out/k8-fastbuild/bin/app.runfiles/__main__/app.py", line 5, in <module>
    import google.auth
ModuleNotFoundError: No module named 'google.auth'
```

If I do not add proto dependencies, the google module is located in

```
.../pypi_google_auth/site-packages/google/__init__.py
```

If I switch to include the proto, the google module is in:

```
.../com_github_protocolbuffers_protobuf/python/google/__init__.py
```

How can I continue using proto, but also make google.auth accessible? Thank you!

[carpen...@gmail.com]

I can't see where your dependency on the "google" is in the "requirements.txt", I would expect that anything module you use is declared to bazel. 

Ideally you would provide an reproduction using a bazel provided non system python.

[Chuan Ye]

Thanks for your reply. 

I pulled the requirements.txt straight from the actual project I'm working on. The dependency "@pypi_kubernetes//:pkg" transtively requires google-auth to work, therefore you should also see google-auth in the requirements.txt. So it appears that adding the proto dependencies(under google.protobuf) affect how python sees the locations of the google module. 

I asked around in the slack channel and was offered this context: https://anikevicius.lt/blog/python-lazy-loading-and-namespace-pkgs/

I believe the ordering of the imports played a role here. So I tried reordering the deps in my actual project, it turns out that if I put the "pypi_kubernetes" dep before the "py_pb2" proto def, I no longer get this issue.

Do you know if this is a known problem to Bazel? Thanks!

[shepherd Ou]

rule_proto_dependencies() introduces  com_github_protocolbuffers_protobuf  into your project. (of couse the py_proto_library need protobuf repo to compile proto files, and it's built from source) .

However, you also introduce another protobuf through pip_parse, which is deps of google.auth.

There's kind of namespace conflict for these two version of protobuf. The first one being introduced by deps list of your target takes advantages. That's why the order matters.

I m not sure if bazel team is aware of this or not. Would you please to create an issue in github to track this problem?

[Chuan Ye]

Hi Shepherd, 

I got a reply from another issue: https://github.com/bazelbuild/rules_python/issues/1338

For me, I had to resort to ugly workaround to address this issue, but you can check out the comment above and see if the explanation is reasonable. 

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/1ca13a41-7164-473f-96fd-f4cd96492a47n%40googlegroups.com.