Upcoming native-Python-rule changes to try out in Bazel 0.23

[Jon Brandvein]

I wanted to highlight some new incompatible changes in Bazel that will be available for preview (--all_incompatible_changes) in 0.23 and flipped on in 0.24 or 0.25. Two of the changes are for how the Python version (2 vs 3) is managed, and one is for deprecating the legacy “py” provider in Starlark code that interfaces with the Python rules (“Python sandwich”). Migration can begin in earnest once 0.23 is released; until then you can play around with the changes using a development build.

Python 2 vs 3

If you’ve seen #6583, you know there’s ongoing work to simplify and rationalize the way that Bazel decides whether a target uses Python 2 or Python 3. While this does not address all 2 vs 3 issues (see #6444), it makes the version selection behavior easier to understand. Briefly, under the new model:

• All targets are either built in PY2 mode or PY3 mode.

• py_binary and py_test targets set the mode to whatever their python_version attribute (formerly called default_python_version) says. It is no longer possible to override this attribute with a command line flag.

• If there is no python_version attribute set, py_binary and py_test default to PY2 (in the future, PY3).

• Because each py_binary sets its own version, it’s now possible to have Python 3 programs in the data dependencies of Python 2 programs and vice versa.

• py_library targets no longer care what mode they build in, so you can build PY2-only and PY3-only libraries in the same bazel invocation (such as in bazel build //...). If you try to use a library marked srcs_version = PY3 in a binary marked python_version = PY2, the binary will fail to build. We provide an aspect you can then use to figure out what the bad dependency was.

The two incompatible changes are --incompatible_allow_python_version_transitions, which activates the above model, and --incompatible_remove_old_python_version_api, which disables the old flag --force_python and attribute default_python_version in favor of their renamed aliases --python_version and python_version respectively. To learn more about the change and how to migrate, see their tracking bugs #7307 and #7308.

Note that due to #4815, in order for Bazel to invoke the proper Python interpreter matching the chosen Python version, you need to specify a py_runtime with --python_top that select()s on the version. See the example here. This workaround will not be needed once we have a py_toolchain rule.

Removal of legacy “py” provider

(This change only concerns authors of rules that interface with the built-in Python rules. Users who only write BUILD files and macros may skip this.)

It used to be that a Starlark rule would get information from its dependencies in the form of a struct. For a long time now the preferred way has been to pass instances of a provider object (such as these) instead. But prior to 0.23, the built-in Python rules still used structs exclusively, keyed under the field name “py”.

With 0.23, Python rules support both the legacy struct provider and a dedicated PyInfo provider type. The new type has exactly the same fields and behavior as the legacy struct, so migrating to the new type requires minimal changes. When the new flag --incompatible_disallow_legacy_py_provider is enabled, the Python rules will no longer emit “py” structs and will fail if given any “py” structs by their dependencies. This will help us remove legacy providers from Bazel altogether. See #7298 for more about the change.