configure checks in bazel?

[Daniel Moody]

I want to run a configure check if the target platform to see if I should set some flag or condition to change the build based on the current selected compiler or current installed libraries if they are available.

For a specific example to demonstrate, lets say I want to automatically test if the current libc has explicit_bzero. If it does the build will compile source file A.cpp otherwise it compiles B.cpp.

How can I do this in bazel?

[David Turner]

You probably cannot do that in general, but there are probably ways to get something usable for you. Let me explain:

The short version is that you cannot use the result of a build action (e.g. trying to compile) to modify the shape of the build graph. This is not limited to Bazel, nearly all other build systems like make, CMake or GN do have an independent "configuration" step that generates the final build graph, and this is where the kind of probing you describe is performed.

Bazel does allow you to probe the host system before the build graph is loaded, by writing a custom repository rule. This is a Starlark function that can inspect your system, even download things for the network, and do plenty of other non-hermetic things. The result is stored in a "repository directory", which is stored in the same location as build outputs, and its content is later available to your BUILD files. For example see https://github.com/solarhess/rules_build_secrets which loads environment variables into a @my_secrets//:BUILD.bazel file, so you can access their values in your project BUILD definitions (without ever committing these values to your source tree).

Thus you could write a custom repository rule to invoke your compiler and probe the C runtime, look what libraries are installed on the host system, etc. And then use the result in your project. But there are a few gotchas:

• Repository rules don't know anything about the build configuration, the toolchain you want to use, and you cannot use target rules like cc_binary() in their implementation function (instead you would use repository_ctx.execute() directly to invoke the compiler and parse the result manually). There is also no Bazel command-line flag you can pass to change their behavior (they take arguments from your WORKSPACE file exclusively, which also doesn't know anything about your build configuration either).
  
  
• At build time, which C library is used to build a C++ executable depends on which C++ toolchain is used, which itself depends on the current build configuration.
  And this information is not available to repository rules. Besides, Bazel supports using multiple build configurations in a single build invocation / graph, and repository rules are run before any graph is constructed.
  

In other words, if all you care are the C library and the installed libraries of your host system, and that you always build with the default C++ toolchain configuration (which matches your host system), you may be able to use a repository rule. Because the information it will find will be correct.

Otherwise, e.g. when cross-compiling, or using a different toolchain, this is not possible, just like you cannot write a Makefile or Ninja build plan that would do the probing itself and modify its dependencies based on the result.

A simpler solution would be to model possible values for the flags you are interested in into build configuration attributes, and set them either on the command-line (e.g. `bazel build --//my_build_config:libc_has_explicit_bzero --cpu=<cpu> --crosstool_top=<toolchain> <targets>`, or in your .bazelrc to avoid too much typing).

 
Hope this helps,

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/47569205-55c0-49be-9ab5-b66a72053c91n%40googlegroups.com.

[Daniel Moody]

Thanks! This is great info. I have a follow on question.

Can an output of bazel rules be bazel BUILD files? and then could another bazel rule then execute a bazel command line which includes that new output BUILD file as part of its build? 

{ name     : "Daniel Moody",
  title    : "Sen Server Development Platform Eng",
  phone    : "321-506-6255",
  location : "Cedar Rapids, IA",
  twitter  : "@MongoDB",
  facebook : "MongoDB" }

[David Turner]

On Thu, Jun 8, 2023 at 5:55 PM Daniel Moody <daniel...@mongodb.com> wrote:

Thanks! This is great info. I have a follow on question.

Can an output of bazel rules be bazel BUILD files?

Yes, of course, they can generate any type of files, but these build outputs will be located in the output_base directory, not in your source tree or one of its sub-directories, and so not directly visible to your other BUILD files.

 

and then could another bazel rule then execute a bazel command line which includes that new output BUILD file as part of its build? 

You would need to copy the new BUILD file to the source tree. `bazel build` cannot modify your source tree (that's a feature), but you can do that with a `bazel run` command (i.e. you would add a Bazel target that creates a script to copy the new BUILD output to the right location, then invoke `bazel run //my_build:configure` to run it).
Keep in mind that the content of that new BUILD file is likely dependent on the build configuration you used when invoking `bazel run` (which does a `bazel build` + runs an actual executable artifact).
I don't think `bazel run` command can invoke Bazel recursively though, you would probably need a second `bazel build` command for the final build, and in this case, be careful about using the same configuration for both.
What I mean is that if you do `bazel run --cpu=aarch64 -copt=debug //my_build:configure`, your next command should really be `bazel build --cpu=aarch64 -copt=debug <your_targets>`, otherwise you will not configure your build graph correctly in the second command.
The safe option would be for your `bazel run ...` command to generate a script that invokes bazel with the right options to avoid any mistakes which would be extremely difficult to debug.

[Filip Filmar]

On Mon, Jun 12, 2023 at 10:18 AM 'David Turner' via bazel-discuss <bazel-...@googlegroups.com> wrote:

and then could another bazel rule then execute a bazel command line which includes that new output BUILD file as part of its build? 

You would need to copy the new BUILD file to the source tree. `bazel build` cannot modify your source tree (that's a feature), but you can do that with a `bazel run` command (i.e. you would add a Bazel target that creates a script to copy the new BUILD output to the right location, then invoke `bazel run //my_build:configure` to run it).
Keep in mind that the content of that new BUILD file is likely dependent on the build configuration you used when invoking `bazel run` (which does a `bazel build` + runs an actual executable artifact).
I don't think `bazel run` command can invoke Bazel recursively though, 

You can use this trick from Alex Eagle to have a `bazel run` update your BUILD files, *and* have some tests fail if you forget to regenerate the files.

F

[Chuck Grindel]

If you are looking for an example of a ruleset that updates a BUILD file and copies it back to the source tree, check out bzlformat. 

"This project contains Bazel rules and macros that format Bazel Starlark files (e.g. .bzl, BUILD, BUILD.bazel) using Buildifier, test that the formatted files exist in the workspace directory, and copy the formatted files to the workspace directory."