PSA: If you're having Python 2/3 errors in Bazel 0.27...
111 views
Skip to first unread message
Jon Brandvein
Jun 17, 2019, 12:33:56 PM6/17/19
to bazel-discuss, Bazel/Python Special Interest Group
Bazel 0.27 flips on --incompatible_use_python_toolchains by default. This has two main benefits:
1. The interpreter is picked by a toolchain, instead of by --python_top or --python_path. This lets you target multiple platforms in the same build, even when the interpreter is installed to different paths on each platform.
2. On non-Windows platforms, a new default toolchain searches the target platform's PATH for an interpreter of the correct version. This fixes longstanding issue #4815, where you'd tell Bazel to use Python 3 and it would in fact launch Python 2, or vice versa.
Unfortunately, a lot of builds were relying on the buggy behavior: Their targets need Python 2, but were declared (perhaps by default) to use Python 3, and just happened to work because Bazel picked a Python 2 interpreter anyway. These builds break in 0.27. Failure modes can include a Python stack trace with a typical 2 vs 3 error (str vs bytes, unknown modules or methods, bad print syntax, etc.), a message about a target in the host config possibly using the wrong version, or a message about not being able to find a suitable interpreter.
Know that Bazel 0.27 does not introduce any new requirements on what versions of Python you need, it only ensures that existing declared requirements are enforced.
If your targets are now attempting to use the wrong version of Python, you can fix this by making sure that any py_binary or py_test containing Python 2 source code is marked with the python_version = "PY2" attribute. For Python 2 targets built in the host config (e.g. genrule tools), make sure you pass --host_force_python=PY2 (add it to your bazelrc for the "build" command).
If your targets are trying to use the right version, but the autodetecting toolchain has trouble finding it, you can either make sure it's available on your PATH, or else define your own custom Python toolchain that knows the correct path.
If your environment has only one Python version (e.g. default Mac install) and all Python targets in your build should be compatible with that version, you also have the option to use the non-strict toolchain. But be aware that this opts back into the buggy behavior of #4815.
For more information, see the migration tracking issue #7899.
1. The interpreter is picked by a toolchain, instead of by --python_top or --python_path. This lets you target multiple platforms in the same build, even when the interpreter is installed to different paths on each platform.
2. On non-Windows platforms, a new default toolchain searches the target platform's PATH for an interpreter of the correct version. This fixes longstanding issue #4815, where you'd tell Bazel to use Python 3 and it would in fact launch Python 2, or vice versa.
Unfortunately, a lot of builds were relying on the buggy behavior: Their targets need Python 2, but were declared (perhaps by default) to use Python 3, and just happened to work because Bazel picked a Python 2 interpreter anyway. These builds break in 0.27. Failure modes can include a Python stack trace with a typical 2 vs 3 error (str vs bytes, unknown modules or methods, bad print syntax, etc.), a message about a target in the host config possibly using the wrong version, or a message about not being able to find a suitable interpreter.
Know that Bazel 0.27 does not introduce any new requirements on what versions of Python you need, it only ensures that existing declared requirements are enforced.
If your targets are now attempting to use the wrong version of Python, you can fix this by making sure that any py_binary or py_test containing Python 2 source code is marked with the python_version = "PY2" attribute. For Python 2 targets built in the host config (e.g. genrule tools), make sure you pass --host_force_python=PY2 (add it to your bazelrc for the "build" command).
If your targets are trying to use the right version, but the autodetecting toolchain has trouble finding it, you can either make sure it's available on your PATH, or else define your own custom Python toolchain that knows the correct path.
If your environment has only one Python version (e.g. default Mac install) and all Python targets in your build should be compatible with that version, you also have the option to use the non-strict toolchain. But be aware that this opts back into the buggy behavior of #4815.
For more information, see the migration tracking issue #7899.
Reply all
Reply to author
Forward
0 new messages