Example usage of libc_top attribute for a cc toolchain
5 views
Skip to first unread message
Mike Krinkin
5:13 PM (4 hours ago) 5:13 PM
to bazel-discuss
Hi Folks,
1. Content of MDOULE.bazel
2. Content of bazel/wasi-sysroot.bazel that just collects all the files into a filegroup:
I'm playing with configuring a custom cc toolchain in bazel and I'm exploring different options to specify sysroot location.
Looking through the docs and over the Internet I found quite a few examples when folks just used cxx_builtin_include_directories in the toolchain config info to provide path to system headers.
Poking around toolchain config info I also noticed that I can provide builtin_sysroot argument when creating CcToolchainConfigInfo, but, from what I can tell, this argument is a plain string, e.g. I cannot pass a label to a filegroup for example. Passing a label is super convenient if I want to download a prebuilt sysroot using something like http_archive.
Looking through the docs and over the Internet I found quite a few examples when folks just used cxx_builtin_include_directories in the toolchain config info to provide path to system headers.
Poking around toolchain config info I also noticed that I can provide builtin_sysroot argument when creating CcToolchainConfigInfo, but, from what I can tell, this argument is a plain string, e.g. I cannot pass a label to a filegroup for example. Passing a label is super convenient if I want to download a prebuilt sysroot using something like http_archive.
Exploring further, I found that cc_toolchain build rule in general allows passing in a bunch of files required by the toolchain and has libc_top attribute. And, unlike builtin_sysroot, it actually accepts a label, so I thought that I can pass in a filegroup.
However, when I tried, I noticed that when I use libc_top, C++ compiler gets called with `--sysroot=` (that's not a typo, it gets called with an empty path after =) and that does not look correct and, naturally, compiler cannot actually find all the standard C++ headers that the downloaded sysroot archive actually contains.
I wonder if somebody can share a working example of defining a toolchain with libc_top attribute used?
And stepping back, more generally, I'm looking for a way to provide a sysroot to the toolchain via a file group, instead of hardcoding a filesystem path, does anybody has a working example of that?
Thank you!
Here is some specifics of what I tried to do. I basically have the following directory structure:
And stepping back, more generally, I'm looking for a way to provide a sysroot to the toolchain via a file group, instead of hardcoding a filesystem path, does anybody has a working example of that?
Thank you!
Here is some specifics of what I tried to do. I basically have the following directory structure:
- MODULE.bazel
- BUILD
- bazel/
- BUILD
- toolchain.bzl
- wasi-sysroot.build
1. Content of MDOULE.bazel
```
module(name = "wasi-http-proxy-wasm", version = "0.0.1")
bazel_dep(name = "platforms", version = "0.0.10")
http_archive = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "wasi-sysroot",
url = "https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-20/wasi-sysroot-20.0.tar.gz",
sha256 = "80e95a591ee91b693375576308a3eba0054af7318afdb19e1f00b6efacad58fa",
strip_prefix = "wasi-sysroot",
build_file = "//bazel:wasi-sysroot.bazel",
)
register_toolchains("//bazel:wasm32-wasi-toolchain")
```
bazel_dep(name = "platforms", version = "0.0.10")
http_archive = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "wasi-sysroot",
url = "https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-20/wasi-sysroot-20.0.tar.gz",
sha256 = "80e95a591ee91b693375576308a3eba0054af7318afdb19e1f00b6efacad58fa",
strip_prefix = "wasi-sysroot",
build_file = "//bazel:wasi-sysroot.bazel",
)
register_toolchains("//bazel:wasm32-wasi-toolchain")
```
2. Content of bazel/wasi-sysroot.bazel that just collects all the files into a filegroup:
```
filegroup(
name = "sysroot",
srcs = glob(["**"]),
visibility = ["//visibility:public"],
)
name = "sysroot",
srcs = glob(["**"]),
visibility = ["//visibility:public"],
)
```
3. content of bazel/BUILD that defines the toolchain:
```
```
load(":toolchain.bzl", "wasm32_wasi_toolchain_config");
filegroup(name = "empty")
platform(
name = "wasip1",
constraint_values = [
"@platforms//cpu:wasm32",
"@platforms//os:wasi",
],
)
wasm32_wasi_toolchain_config(name = "wasm32-wasi-cc-toolchain-config")
cc_toolchain(
name = "wasm32-wasi-cc-toolchain",
toolchain_config = ":wasm32-wasi-cc-toolchain-config",
all_files = ":empty",
compiler_files = ":empty",
dwp_files = ":empty",
linker_files = ":empty",
objcopy_files = ":empty",
strip_files = ":empty",
libc_top = "@wasi-sysroot//:sysroot",
supports_param_files = False,
)
toolchain(
name = "wasm32-wasi-toolchain",
toolchain = ":wasm32-wasi-cc-toolchain",
toolchain_type = "@bazel_tools//tools/cpp:toolchain_type",
target_compatible_with = [
"@platforms//cpu:wasm32",
"@platforms//os:wasi",
],
)
```
filegroup(name = "empty")
platform(
name = "wasip1",
constraint_values = [
"@platforms//cpu:wasm32",
"@platforms//os:wasi",
],
)
wasm32_wasi_toolchain_config(name = "wasm32-wasi-cc-toolchain-config")
cc_toolchain(
name = "wasm32-wasi-cc-toolchain",
toolchain_config = ":wasm32-wasi-cc-toolchain-config",
all_files = ":empty",
compiler_files = ":empty",
dwp_files = ":empty",
linker_files = ":empty",
objcopy_files = ":empty",
strip_files = ":empty",
libc_top = "@wasi-sysroot//:sysroot",
supports_param_files = False,
)
toolchain(
name = "wasm32-wasi-toolchain",
toolchain = ":wasm32-wasi-cc-toolchain",
toolchain_type = "@bazel_tools//tools/cpp:toolchain_type",
target_compatible_with = [
"@platforms//cpu:wasm32",
"@platforms//os:wasi",
],
)
```
4. Finally, the content of bazel/toolchain.bzl:
```
load("@bazel_tools//tools/cpp:cc_toolchain_config_lib.bzl", "feature", "flag_set", "flag_group", "tool_path")
load("@bazel_tools//tools/build_defs/cc:action_names.bzl", "ACTION_NAMES")
COMPILE_ACTIONS = [
ACTION_NAMES.cpp_compile,
ACTION_NAMES.c_compile,
]
LINK_ACTIONS = [
ACTION_NAMES.cpp_link_executable,
]
COMPILE_AND_LINK_ACTIONS = COMPILE_ACTIONS + LINK_ACTIONS
target_wasm32 = feature(
name = "target-wasm32",
enabled = True,
flag_sets = [
flag_set(
actions = COMPILE_AND_LINK_ACTIONS,
flag_groups = [
flag_group(
flags = ["--target=wasm32"],
),
],
)
],
)
nostdlib = feature(
name = "nostdlib",
enabled = True,
flag_sets = [
flag_set(
actions = COMPILE_AND_LINK_ACTIONS,
flag_groups = [
flag_group(
flags = ["-nostdlib"],
),
],
),
],
)
no_entry = feature(
name = "no-entry",
enabled = True,
flag_sets = [
flag_set(
actions = LINK_ACTIONS,
flag_groups = [
flag_group(
flags = ["-Wl,--no-entry"],
),
],
),
],
)
load("@bazel_tools//tools/build_defs/cc:action_names.bzl", "ACTION_NAMES")
COMPILE_ACTIONS = [
ACTION_NAMES.cpp_compile,
ACTION_NAMES.c_compile,
]
LINK_ACTIONS = [
ACTION_NAMES.cpp_link_executable,
]
COMPILE_AND_LINK_ACTIONS = COMPILE_ACTIONS + LINK_ACTIONS
target_wasm32 = feature(
name = "target-wasm32",
enabled = True,
flag_sets = [
flag_set(
actions = COMPILE_AND_LINK_ACTIONS,
flag_groups = [
flag_group(
flags = ["--target=wasm32"],
),
],
)
],
)
nostdlib = feature(
name = "nostdlib",
enabled = True,
flag_sets = [
flag_set(
actions = COMPILE_AND_LINK_ACTIONS,
flag_groups = [
flag_group(
flags = ["-nostdlib"],
),
],
),
],
)
no_entry = feature(
name = "no-entry",
enabled = True,
flag_sets = [
flag_set(
actions = LINK_ACTIONS,
flag_groups = [
flag_group(
flags = ["-Wl,--no-entry"],
),
],
),
],
)
export_all = feature(
name = "export-all",
enabled = True,
flag_sets = [
flag_set(
actions = LINK_ACTIONS,
flag_groups = [
flag_group(
flags = ["-Wl,--export-all", "-Wl,--no-gc-sections"],
),
],
),
],
)
name = "export-all",
enabled = True,
flag_sets = [
flag_set(
actions = LINK_ACTIONS,
flag_groups = [
flag_group(
flags = ["-Wl,--export-all", "-Wl,--no-gc-sections"],
),
],
),
],
)
def _toolchain_config_impl(ctx):
return cc_common.create_cc_toolchain_config_info(
ctx = ctx,
toolchain_identifier = "wasm32-wasi",
target_system_name = "wasm32-wasi",
target_cpu = "wasm32",
target_libc = "wasi",
compiler = "clang",
# This seem to be the only thing that matters in the rule, all the
# values above, I think, ignored in the ne enough versions of Bazel.
tool_paths = [
tool_path(
name = "gcc",
path = "/usr/bin/clang",
),
tool_path(
name = "ar",
path = "/usr/bin/llvm-ar",
),
tool_path(
name = "ld",
path = "/bin/false",
),
tool_path(
name = "cpp",
path = "/bin/false",
),
tool_path(
name = "nm",
path = "/bin/false",
),
tool_path(
name = "objdump",
path = "/bin/false",
),
tool_path(
name = "strip",
path = "/bin/false",
),
],
cxx_builtin_include_directories = [
"%sysroot%/include/c++/v1",
],
features = [
target_wasm32,
export_all,
no_entry,
nostdlib,
],
)
wasm32_wasi_toolchain_config = rule(
implementation = _toolchain_config_impl,
attrs = {},
provides = [CcToolchainConfigInfo],
)
return cc_common.create_cc_toolchain_config_info(
ctx = ctx,
toolchain_identifier = "wasm32-wasi",
target_system_name = "wasm32-wasi",
target_cpu = "wasm32",
target_libc = "wasi",
compiler = "clang",
# This seem to be the only thing that matters in the rule, all the
# values above, I think, ignored in the ne enough versions of Bazel.
tool_paths = [
tool_path(
name = "gcc",
path = "/usr/bin/clang",
),
tool_path(
name = "ar",
path = "/usr/bin/llvm-ar",
),
tool_path(
name = "ld",
path = "/bin/false",
),
tool_path(
name = "cpp",
path = "/bin/false",
),
tool_path(
name = "nm",
path = "/bin/false",
),
tool_path(
name = "objdump",
path = "/bin/false",
),
tool_path(
name = "strip",
path = "/bin/false",
),
],
cxx_builtin_include_directories = [
"%sysroot%/include/c++/v1",
],
features = [
target_wasm32,
export_all,
no_entry,
nostdlib,
],
)
wasm32_wasi_toolchain_config = rule(
implementation = _toolchain_config_impl,
attrs = {},
provides = [CcToolchainConfigInfo],
)
```
And if I try to build a simple C++ binary using this configuration, I can see that the command line used by bazel to compile it is not correct, e.g. this is what I get:
```
SUBCOMMAND: # //:test.wasm [action 'Compiling test.cc', configuration: c295ed8ddc178a26d1b4c0bf7f717254df9a0646250860acef1e4054f48130cd, execution platform: @@platforms//host:host, mnemonic: CppCompile]
(cd /mount/data/bazel/cache/_bazel_kmu/37cd2e8f7b2d43f58d4f2c5d226b367b/execroot/_main && \
exec env - \
PATH=/home/kmu/.cache/bazelisk/downloads/sha256/48ea0ff9d397a48add6369c261c5a4431fe6d5d5348cfb81411782fb80c388d3/bin:/home/kmu/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/kmu/.dotnet/tools:/usr/local/go/bin:/home/kmu/go/bin:/usr/local/go/bin:/home/kmu/go/bin \
PWD=/proc/self/cwd \
/usr/bin/clang -MD -MF bazel-out/k8-fastbuild/bin/_objs/test.wasm/test.d '-frandom-seed=bazel-out/k8-fastbuild/bin/_objs/test.wasm/test.o' -iquote . -iquote bazel-out/k8-fastbuild/bin -iquote external/bazel_tools -iquote bazel-out/k8-fastbuild/bin/external/bazel_tools '--target=wasm32' -nostdlib '--sysroot=' -c test.cc -o bazel-out/k8-fastbuild/bin/_objs/test.wasm/test.o)
# Configuration: c295ed8ddc178a26d1b4c0bf7f717254df9a0646250860acef1e4054f48130cd
# Execution platform: @@platforms//host:host
ERROR: /mount/data/ws/wasi-http/BUILD:1:10: Compiling test.cc failed: (Exit 1): clang failed: error executing CppCompile command (from target //:test.wasm) /usr/bin/clang -MD -MF bazel-out/k8-fastbuild/bin/_objs/test.wasm/test.d '-frandom-seed=bazel-out/k8-fastbuild/bin/_objs/test.wasm/test.o' -iquote . -iquote bazel-out/k8-fastbuild/bin -iquote ... (remaining 10 arguments skipped)
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
test.cc:1:10: fatal error: 'iostream' file not found
#include <iostream>
^~~~~~~~~~
1 error generated.
Target //:test.wasm failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 3.096s, Critical Path: 0.15s
INFO: 5 processes: 5 internal.
ERROR: Build did NOT complete successfully
(cd /mount/data/bazel/cache/_bazel_kmu/37cd2e8f7b2d43f58d4f2c5d226b367b/execroot/_main && \
exec env - \
PATH=/home/kmu/.cache/bazelisk/downloads/sha256/48ea0ff9d397a48add6369c261c5a4431fe6d5d5348cfb81411782fb80c388d3/bin:/home/kmu/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/kmu/.dotnet/tools:/usr/local/go/bin:/home/kmu/go/bin:/usr/local/go/bin:/home/kmu/go/bin \
PWD=/proc/self/cwd \
/usr/bin/clang -MD -MF bazel-out/k8-fastbuild/bin/_objs/test.wasm/test.d '-frandom-seed=bazel-out/k8-fastbuild/bin/_objs/test.wasm/test.o' -iquote . -iquote bazel-out/k8-fastbuild/bin -iquote external/bazel_tools -iquote bazel-out/k8-fastbuild/bin/external/bazel_tools '--target=wasm32' -nostdlib '--sysroot=' -c test.cc -o bazel-out/k8-fastbuild/bin/_objs/test.wasm/test.o)
# Configuration: c295ed8ddc178a26d1b4c0bf7f717254df9a0646250860acef1e4054f48130cd
# Execution platform: @@platforms//host:host
ERROR: /mount/data/ws/wasi-http/BUILD:1:10: Compiling test.cc failed: (Exit 1): clang failed: error executing CppCompile command (from target //:test.wasm) /usr/bin/clang -MD -MF bazel-out/k8-fastbuild/bin/_objs/test.wasm/test.d '-frandom-seed=bazel-out/k8-fastbuild/bin/_objs/test.wasm/test.o' -iquote . -iquote bazel-out/k8-fastbuild/bin -iquote ... (remaining 10 arguments skipped)
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
test.cc:1:10: fatal error: 'iostream' file not found
#include <iostream>
^~~~~~~~~~
1 error generated.
Target //:test.wasm failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 3.096s, Critical Path: 0.15s
INFO: 5 processes: 5 internal.
ERROR: Build did NOT complete successfully
```
I can see that it's actually using my toolchain configuration (e.g., I can see `--target=wasm32` and `-nostdlib` flags), but other parts of the command just aren't right. As I mentioned earlier `'--sysroot='` without an actual path does not look right, but aside from that, I don't see anything similar to what I specified in cxx_builtin_include_directories.
I also tried to use %package()% syntax in cxx_builtin_include_directories, but that didn't work, though with a different error:
```
I also tried to use %package()% syntax in cxx_builtin_include_directories, but that didn't work, though with a different error:
```
ERROR: /mount/data/ws/wasi-http/bazel/BUILD:17:13: in cc_toolchain rule //bazel:wasm32-wasi-cc-toolchain:
Traceback (most recent call last):
File "/virtual_builtins_bzl/common/cc/cc_toolchain.bzl", line 55, column 45, in _cc_toolchain_impl
File "/virtual_builtins_bzl/common/cc/cc_toolchain_provider_helper.bzl", line 232, column 64, in get_cc_toolchain_provider
File "/virtual_builtins_bzl/common/cc/cc_toolchain_provider_helper.bzl", line 129, column 34, in _resolve_include_dir
Error: 'workspace_root' is not allowed on invalid Label @@[unknown repo 'wasi-sysroot' requested from @@_builtins]//:dummy_target
ERROR: /mount/data/ws/wasi-http/bazel/BUILD:17:13: Analysis of target '//bazel:wasm32-wasi-cc-toolchain' failed
Traceback (most recent call last):
File "/virtual_builtins_bzl/common/cc/cc_toolchain.bzl", line 55, column 45, in _cc_toolchain_impl
File "/virtual_builtins_bzl/common/cc/cc_toolchain_provider_helper.bzl", line 232, column 64, in get_cc_toolchain_provider
File "/virtual_builtins_bzl/common/cc/cc_toolchain_provider_helper.bzl", line 129, column 34, in _resolve_include_dir
Error: 'workspace_root' is not allowed on invalid Label @@[unknown repo 'wasi-sysroot' requested from @@_builtins]//:dummy_target
ERROR: /mount/data/ws/wasi-http/bazel/BUILD:17:13: Analysis of target '//bazel:wasm32-wasi-cc-toolchain' failed
```
The basel version I'm using is 7.4.0 (via bazelisk).
Reply all
Reply to author
Forward
0 new messages