Bazel Blog

[Bazel Blog]

Bazel Blog

---

Bazel 0.27 Posted: 16 Jun 2019 05:00 PM PDT Bazel 0.27 has just been released. With Bazel 0.27, we start a 3 month stability window. Bazel 0.28 and Bazel 0.29 will be backward compatible with Bazel 0.27. See our announcement about Bazel Stability and Semantic Versioning. This release contains more incompatible changes than usual, as we try to fix the important issues before Bazel 1.0. Incompatible changes Bazel 0.27 Linux release binaries have been built on Ubuntu 16.04 (before: Ubuntu 14.04, which is now EOL). This change will probably break compatibility with older Linux distributions like Ubuntu 14.04. If this causes you problems, please open an issue and let us know. Before updating to Bazel 0.27, first check if your codebase is compatible either by running bazelisk --migrate or by building your code with Bazel 0.26 and the following flags: --incompatible_disable_deprecated_attr_params --incompatible_depset_for_libraries_to_link_getter --incompatible_depset_is_not_iterable --incompatible_disallow_native_in_build_file --incompatible_disallow_old_octal_notation --incompatible_do_not_split_linking_cmdline --incompatible_dont_enable_host_nonhost_crosstool_features --incompatible_list_based_execution_strategy_selection --incompatible_new_actions_api --incompatible_no_kwargs_in_build_files --incompatible_no_support_tools_in_action_inputs --incompatible_objc_framework_cleanup --incompatible_remove_binary_profile --incompatible_require_ctx_in_configure_features --incompatible_static_name_resolution_in_build_files --incompatible_string_join_requires_strings --incompatible_use_python_toolchains --incompatible_windows_escape_python_args For more information about the changes, please click on the links and read the GitHub issues. Comment there if you need help. If you use external repositories, you might first have to update the dependencies in your WORKSPACE file. Android Introduced the --incompatible_disable_native_android_rules flag. This flag facilitates the migration of the Android native rules to Starlark. For more information, see Android Native to Starlark Migration. Android desugaring actions now use persistent workers by default. This has been measured to provide up to 20% reduction in build times. To disable it, use the --strategy=Desugar=sandboxed flag. See Bazel issue 8342 and Bazel issue 8427 for more details on local build speed optimization for Android apps. Fixed an issue with Android builds where --fat_apk_cpu doesn't pack all selected shared libraries from aar_import targets into the APK. See Bazel issue 8283. Added support for Android NDK 19 and 20. Optimized build times further by 10% by trimming unused build actions. See Bazel issue 8340 for more details. Apple platforms BAZEL_USE_XCODE_TOOLCHAIN=1 tells Bazel not to look for Xcode to decide whether to enable toolchains for Apple rules, but to assume Xcode is available. It can be also used when building on Darwin and no C++ or ObjC is being built, so there is no need to detect Xcode. C++ Added a generic additional_linker_inputs attribute on cc_binary rules. Windows, C++ autoconfigure: BAZEL_VC and BAZEL_VS can now use quotes. If you set these envvars' values in cmd.exe via TAB-completion, you no longer need to remove the surrounding quotes. cc_import() of a DLL with no interface library on Windows is now allowed. This is used to document runtime dependencies. Repository containing autoconfigured C++ toolchain @local_config_cc has been split. See local_config_cc_toolchains. Added new option --cs_fdo_absolute_path= to support the absolute path profile for LLVM's context-sensitive FDO. Python Python rules now find the Python runtime using toolchains, rather than the --python_top and --python_path flags, which are deprecated. See the migration issue for more information on declaring Python toolchains and migrating your code. As a side-benefit of enabling toolchains, Bazel now enforces that Python targets execute under the same version of Python that was determined in the analysis phase (fixing #4815), on all platforms besides Windows. This means that builds may start to fail with Python 2 vs 3 errors, if they previously relied on getting the "wrong" Python version. To fix this, make sure that any target containing PY2-only code has the attribute python_version = "PY2". Furthermore, if your (or your dependencies') host-configured tools use Python 2, add --host_force_python=PY2 to your bazelrc to tell Bazel about this requirement. When a py_binary built for the host configuration fails, it will now emit a warning on stderr suggesting to check whether --host_force_python=PY2 needs to be set. For users who do not have both a Python 2 and Python 3 runtime installed, there is now a non-strict Python toolchain that launches whatever python command it finds on PATH without caring about its version. Effectively this opts back into the buggy behavior of #4815. Pass --extra_toolchains=@bazel_tools//tools/python:autodetecting_toolchain_nonstrict to use it. Configurations Starlark configuration transitions no longer require --experimental_starlark_config_transitions. Remote Execution The flag --incompatible_list_based_execution_strategy_selection has been flipped. Users of remote execution no longer need to specify --spawn_strategy=remote, --strategy=Javac=remote, --strategy=Closure=remote, --strategy=Genrule=remote. See Bazel issue 7480. The flag --remote_local_fallback_strategy has been deprecated. Setting it is no longer be necessary as Bazel now automates selects the best strategy. See Bazel issue 7480. The flag --tls_enabled has been deprecated and will be removed once --incompatible_tls_enabled_removed will be flipped. Instead, enable TLS directly via the URI passed to --remote_executor, --remote_cache and --bes_backend. See Bazel issue 8061 for details. The new flag --remote_retries has been introduced to tune the number of times RPCs should be retried by Bazel. Setting --remote_retries=0 will disable retries. The flags --experimental_remote_retry_* have been deprecated and made a no-op. Other changes The copy-pasted init code of the Bash runfiles library is now shorter. See comments in tools/bash/runfiles/runfiles.bash. To use the new init code, you need Bazel 0.27 or newer. The old init code still works. pkg_deb has new attributes: config and templates that can be used for integration with debconf. Community Wix published Exodus, a Maven to Bazel migration tool. Shanee Nishry wrote a tutorial for using Bazel with Visual Studio Code, including steps to debug a C++ project. Rob Sayre wrote extensive lessons on Bazel concepts, combining languages, Go, Rust, and package manager integration. (Lesson 1, Lesson 2) At Kubecon 2019, there were some Bazel-related talks: Reproducible Development and Deployment with Bazel and Telepresence by Christian Roggia; Streamlining Kubernetes Application CI/CD with Bazel by Gregg Donovanand Chris Love; Repeatable Deployments with Kubernetes, Helm & Bazel by Rohan Sing. Natan Silnitsky gave a talk at JEEConf about How to successfully migrate to Bazel from Maven or Gradle. This release contains contributions from many people at Google, as well as Alex Thompson, Andy Scott, Benjamin Peterson, David McNett, Drew Gassaway, James deBoer, Jay Vercellone, Josh Rosen, Keith Smiley, Lee Mracek, Marwan Tammam, Matt Passell, Michael Hackner, Michal Majcherski, Patrick Niklaus, Peter Mounce, Ricky Pai, Steeve Morin, szakmary, Takuto Ikuta, Vladimir Chebotarev, and Yen-Chi Chen.

You are subscribed to email updates from Bazel Blog. To stop receiving these emails, you may unsubscribe now.	Email delivery powered by Google
Google, 1600 Amphitheatre Parkway, Mountain View, CA 94043, United States

[Austin Schuh]

Thanks for reposting this to the mailing list.  The information here is really good.  Congrats on the release!

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/0000000000009a57c2058b995d02%40google.com.
For more options, visit https://groups.google.com/d/optout.

[Bazel Blog]

Bazel Blog

---

Automatic execution strategy selection in Bazel 0.27 Posted: 18 Jun 2019 05:00 PM PDT tl;dr: Exciting news! After 0.27 release Bazel can auto select a suitable execution strategy, eliminating the need for manual configuration via command line flags in most cases. When Bazel executes commands that are a part of the build, such as compiler and linker invocations, test runs etc., it has a choice on how to execute those commands (also called actions): locally, remotely, in a sandbox, and so on. This is controlled by execution strategies. Starting with 0.27 release, we implemented a feature in Bazel that will allow it to auto select a suitable execution strategy, eliminating the need for manual configuration via command line flags in most cases; and even in cases when customization and strategy enforcement is needed, it is still fully possible and has become much simpler. Before Bazel 0.27 It was possible to configure Bazel's execution strategy for a build via flags --spawn_strategy=, --strategy=Mnemonic= and --strategy_regexp=Regex=. This mechanism was quite powerful and widely used, but had some drawbacks: Bazel's defaults did not account for remote execution at all, therefore we had to make sure to set the execution strategy for every mnemonic manually. Starlark rules providing a persistent worker had to ship a .bazelrc file that sets --strategy=Mnemonic=worker (e.g. look at rules_scala, where --strategy=Scalac=worker is required in order to run it with a persistent worker). Bazel had hardcoded defaults for native action mnemonics. For example, Bazel would have used sandboxing (if it's available) for all actions by default but at the same time had a hardcoded default to use persistent workers for Java compilation actions. To configure fallback strategy one had to provide additional flag --remote_local_fallback_strategy=. Example Let's configure a build to run remotely and fallback onto local strategy in case remote execution is not possible. The configuration would be something like: $ bazel build --spawn_strategy=remote --strategy=Javac=remote --strategy=Closure=remote --strategy=Genrule=remote --remote_local_fallback_strategy=local ... After Bazel 0.27 Bazel now auto-detects the execution strategy, if no strategy flag is provided. If none of the strategy flags was used, bazel will generate a default list of strategies remote,worker,sandboxed,local and, for every action it wants to execute, will pick up the first strategy that can execute it. The user can pass a comma-separated list of strategies to the above mentioned flags: --spawn_strategy=remote,worker,linux-sandbox. Each strategy now knows whether it can execute a given action. For any action that it wants to execute, Bazel just picks the first strategy from the given list that can execute the action. If an action cannot be executed with any of the given strategies, the build will fail. If any of the strategy flags is provided, then Bazel will use ONLY those strategies that are listed there. Which means that --spawn_strategy=remote will NOT fallback to a sandboxed strategy or to any other strategy in case remote action failed or is not possible at all. Moreover, --remote_local_fallback_strategy=local is now deprecated and made no-op. Why is it better? Now you can control your execution strategy and have your build fail if any non-sandboxable or non-remotable action sneaks in! More reproducibility and safety for your builds! The strategies no longer do their own custom fallback, simplifying the code and unifying the behavior. You might even completely forget about strategies configurations if the default behavior satisfies your needs. Example This is how the previous example will look like now: $ bazel build --spawn_strategy=remote,local ... And if a custom strategy for Java rules is required, we can still add --strategy=Javac=: $ bazel build --spawn_strategy=remote,local --strategy=Javac=worker ... In this case, Java rules will be executed using a persistent worker, everything else will be executed remotely or locally (if remote is not possible). Isn't it much shorter and simpler? How to use it You want a build to be executed properly with minimal configuration: don't set any strategy flags and Bazel will try to do the best automatically. Bazel will use remote execution if it's available, otherwise persistent workers, otherwise sandboxed execution, otherwise non-sandboxed execution. You want the best sandboxed build on your Linux machine and no automatic fallback to symlink-only sandboxing, non-sandboxed execution and no persistent workers: --spawn_strategy=linux-sandbox. You want persistent workers for actions that support it, but otherwise only sandboxed execution: --spawn_strategy=worker,sandboxed. You want to ensure everything runs remotely: --spawn_strategy=remote. Be careful, this configuration will cause a build failure in case bazel finds any action that cannot be executed remotely. How to migrate You would know that you need to migrate if you see the following error: ERROR: No usable spawn strategy found for spawn with mnemonic %Mnemonic%. Are your --spawn_strategy or --strategy flags too strict? The general advice here would be to remove any --spawn_strategy, --strategy, --remote_local_fallback_strategy= or --strategy_regexp=Regex= flags that you might be setting manually or via a .bazelrc file, this will allow bazel to automatically detect suitable execution strategy. In case this is not working for you, review the above how-to-use section for ideas. Contact us Note that the change was made behind the incompatible flag, so please ping us in case of any difficulties.