Precedence of libraries on PYTHONPATH

[Jon Brandvein]

It is sometimes possible for a Python module defined by (or depended on by) the build to override a module in the standard library.

This can lead to errors. For example (paraphrasing katzdm), you could indirectly depend on the "enum34" PyPI package, which defines an "enum" module intended for older versions of Python that lack their own enum in the standard library. This breaks newer versions of Python unless the standard library is allowed to take precedence over enum34 in the PYTHONPATH.

(Or, from my own experience teaching programming to kids, you might name your local utility file "turtle.py" and suddenly get confusing errors when you "import turtle" in your main program.)

PR #6532 would prevent this by having Bazel append custom import paths to PYTHONPATH, rather than prepend them. But since this is an observable difference to the user, I'd like to get some feedback on whether this is better viewed as a bugfix, or as a feature that needs to be controllable by the user, perhaps via an attribute on py_library or py_binary. Are there legitimate use cases for overriding the standard library / the system PYTHONPATH?

My suspicion is that we can get away with making this change unconditional, and require users who want the old behavior to hack sys.path at startup, or (in the future) customize the launcher stub script.

Thoughts?

[Evan Jones]

My 2c: This seems like a bug fix. AFAIK, the "standard" way of managing Python dependencies by putting them into a virtualenv would not allow you to easily override the built-in standard library imports, since site-packages ends up after the standard library on the import path. I think Bazel Python should be as close to "standard" as possible.

--
You received this message because you are subscribed to the Google Groups "Bazel/Python Special Interest Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-sig-pyth...@googlegroups.com.
To post to this group, send email to bazel-si...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-sig-python/16650b82-179e-44df-9b3b-53c12de9faf2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Clint Harrison]

+1 to the above -- very happy to see this change, as we've also considered it a bug and had to fork various dependencies to work around it.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-sig-python/CACRPLjh%3DoMEjm2jX-SsyRG-MqiLACDsb2qUqosVVXNMiwcOZDA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Clint Harrison

Software Engineer

Flatiron Health | New York, NY

www.flatiron.com

[markus....@ecosia.org]

Agreed with the previous posts that this should be treated as a bugfix.

[Jon Brandvein]

We'll resolve this by merging the PR as soon as the tests pass.