Bazel python binary still require pip dependencies on host machine

[Chuan Ye]

Hi,

I ran into an issue yesterday that made me question my understanding of Bazel. Here's a simplistic example.

I followed kriscfoster's informative youtube tutorial, here's the code that packages an external Flask dependency into the web server. I verified that I do not have Flask installed on my host machine, but I'm able to run it with bazel run. I'm also able to package the entire code in a zip file with "bazel build //projects/python_web:python_web --build_python_zip", and run "python3 bazel-bin/projects/python_web/python_web.zip" to start the webserver.

So this is great, my understanding is that Bazel has packaged all the Flask dependencies into the binary.

But now, if I add a dependency for kubernete's python client, and just import the module in the code, this error would occur:

"
Traceback (most recent call last):
  File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/56720d46a6917155e025bd8202a3c20a/execroot/multi_language_monorepo/bazel-out/k8-fastbuild/bin/projects/python_web/python_web.runfiles/multi_language_monorepo/projects/python_web/python_web.py", line 5, in <module>
    from kubernetes import client
  File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/56720d46a6917155e025bd8202a3c20a/execroot/multi_language_monorepo/bazel-out/k8-fastbuild/bin/projects/python_web/python_web.runfiles/pip/pypi__kubernetes/site-packages/kubernetes/__init__.py", line 20, in <module>
    from . import config
  File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/56720d46a6917155e025bd8202a3c20a/execroot/multi_language_monorepo/bazel-out/k8-fastbuild/bin/projects/python_web/python_web.runfiles/pip/pypi__kubernetes/site-packages/kubernetes/config/__init__.py", line 19, in <module>
    from .kube_config import (KUBE_CONFIG_DEFAULT_LOCATION,
  File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/56720d46a6917155e025bd8202a3c20a/execroot/multi_language_monorepo/bazel-out/k8-fastbuild/bin/projects/python_web/python_web.runfiles/pip/pypi__kubernetes/site-packages/kubernetes/config/kube_config.py", line 28, in <module>
    import google.auth
ModuleNotFoundError: No module named 'google.auth'

"

I wondered if this could be because pip_install did not pull in the google-auth dependency, so I looked into the packaged code, and I found "pypi__google_auth" in the rulefiles.

"
ubuntu@ubuntu-B550M-DS3H-AC:~/code/test/temp/runfiles/pip$ ls
__init__.py               pypi__click        pypi__itsdangerous  pypi__oauthlib         pypi__pyyaml             pypi__six
pypi__cachetools          pypi__flask        pypi__jinja2        pypi__pyasn1           pypi__requests           pypi__urllib3
pypi__certifi             pypi__google_auth  pypi__kubernetes    pypi__pyasn1_modules   pypi__requests_oauthlib  pypi__websocket_client
pypi__charset_normalizer  pypi__idna         pypi__markupsafe    pypi__python_dateutil  pypi__rsa                pypi__werkzeug

"

Now, if I choose to install the google-auth package on my host machine, the issue is resolved. But I consider this more as a workaround. How is it that Flask is able to capture all its dependencies into the package but kubernetes python client is not able to?

Or is there a fundamental misunderstanding I have with how Bazel packages the python package?

Much thanks!

[Chuan Ye]

For some reason I can no longer reproduce this issue with the code I provided above. I will close this post for now. 

I was able to reproduce this issue with much simpler setup, will make another post about this.