How do I setup/configure custom toolchain that is not external?

2,504 views
Skip to first unread message

Indra Gunawan

unread,
Jul 6, 2017, 7:28:34 PM7/6/17
to bazel-discuss
On https://github.com/bazelbuild/bazel/wiki/Building-with-a-custom-toolchain, the toolchain is external where the tool needs to be hosted externally.
But for my case, I would like to setup/configure for instance x86_64_linux_gcc_4.7.0.BUILD where the location of all the compiler components are actually local on NFS mount.
How do I have a separate CROSSTOOL file for each different compiler for each platform.  It is a C/C++ embedded project.


Do I create a custom .bzl file instead if it is local?  I do not want to add the above to default CROSSTOOL in the bazel source such that it is auto-detected.  The platform x86_64 is a target platform parameter to the build and it should choose gcc_4.7.0 compiler by default (somehow it should be a variable declared that bazel read to choose default compiler).
How do I do that for local custom compiler.  Is there wiki and Github example project with the sample .bzl file and CROSSTOOL file?

Thank you
-Indra

Brian Silverman

unread,
Jul 6, 2017, 7:37:45 PM7/6/17
to Indra Gunawan, bazel-discuss
You can something almost identical to that wiki page, except with new_local_repository (with an appropriate path) instead of new_http_archive.

If you want separate CROSSTOOL files for each platform, they'll need to be in separate packages and then you'll use a different --crosstool_top flag for each one. I would keep them all in the same one though and use --cpu and --compiler to pick which one though, because I've found it easier to maintain.

You don't have to create a custom .bzl file like the default auto-detection does if you're using a static toolchain at a predefined location, and probably shouldn't to keep it simpler.

tools/bazel.rc is the place to set a "default" CROSSTOOL etc. You can also use a repository-local .bazelrc file to override that for each repository, if that's what you mean by "variable". The default_toolchain declarations in CROSSTOOL specify which toolchain to pick for a given cpu if no --compiler flag is specified.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/13679089-7e25-45b5-8886-e3cdea19cc3c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Indra Gunawan

unread,
Jul 17, 2017, 7:37:18 PM7/17/17
to bazel-discuss, ind...@gmail.com
Thank you for reply.  Is there a simpler Github project as example for configuring custom toolchain for multiple target platform each with different compiler/gcc?

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.

Indra Gunawan

unread,
Aug 1, 2017, 5:06:11 PM8/1/17
to bazel-discuss, ind...@gmail.com
Do I need to setup wrapper scripts as well for internal new_local_repository?


for example here is the new_local_repository defined in the WORKSPACE:

new_local_repository(
name = 'LinuxGcc47Repo',
build_file = 'compilers/x86_64_linux_gcc_4.7.0.BUILD',
path = '/sw/packages/gcc/c4.7.0-p1',
)


/sw/packages is the NFS filer location of the gcc compiler.

On Thursday, July 6, 2017 at 4:37:45 PM UTC-7, Brian Silverman wrote:
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.

Brian Silverman

unread,
Aug 3, 2017, 7:00:02 PM8/3/17
to Indra Gunawan, bazel-discuss
Depends on how your compiler is built. I've found wrapper scripts necessary for most GCC/Clang builds to set LD_LIBRARY_PATH and exec the compiler driver with the correct path so it finds the assembler/linker/cc1/etc from itself and not off the host system, but I believe there are ways to build the compilers so that's not required. Some cross compilers also end up needing PATH set to find some of those tools.

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discuss+unsubscribe@googlegroups.com.
 To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/209b083d-8c53-4bf2-a503-cfccee7ebf3b%40googlegroups.com.

Marcel Hlopko

unread,
Aug 11, 2017, 7:11:29 AM8/11/17
to bazel-discuss, ind...@gmail.com
Hi Indra,

I'm not sure I follow and understand all your questions, please do not hesitate to ask more. 

From the crosstool perspective it doesn't matter if you use local or remote repository. 
Having a separate CROSSTOOL for each platform is doable, but I think you'll achieve the same with having single CROSSTOOL with multiple toolchains, just like Brian suggests.
Bzl file you see around crosstools is only there to autogenerate the crosstool, to tweak its content to be usable in more environments. You don't need it at all, for example this tensorflow crosstool is just static: https://github.com/tensorflow/tensorflow/tree/master/third_party/toolchains/cpus
Documentation about crosstool is very minimal, we're fixing and changing it (most of the cases in a backwards compatible way), and we'll improve the docs once it stabilizes.

Wrapper scripts are usually there for 2 reasons (so you might not need them): 
* to put extra environment variables around the compiler invocation, such as LD_LIBRARY_PATH
* to access files above the current directory (../../../gcc is not supported in crosstool right now, although we'll propably fix it eventually)

It also makes sense to me to keep the crosstool and compilers it uses together in a single workspace/repository/package.

Sorry for the late reply and I hope I answered at least some of the questions. Feel free to ask more, I'm aware how frustrating writing crosstool can be.

Indra Gunawan

unread,
Aug 11, 2017, 4:04:37 PM8/11/17
to Marcel Hlopko, bazel-discuss
HI Marcel,

Thank you for the reply on Github and to the hazel-discuss thread.
If the only difference I need to make is to use “new_local_repository” instead of the example’s external,  does referring to the include headers that come with the toolchain work with “external” directory like this below too in CROSSTOOL?

compiler_flag: “-isystem” compiler_flag: “external/LinuxGcc47Repo/x86_64-linux/lib/gcc/x86_64-linux/4.7.0/include”

Does Bazel copy the toolchain over to “external” directory or create a symlink inside this “external” directory.

The WORKSPACE where new_local_repository is defined looks like:


new_local_repository(
name = 'LinuxGcc47Repo',
build_file = 'compilers/x86_64_linux_gcc_4.7.0.BUILD',
path = '/sw/packages/gcc/c4.7.0-p1',
)


the /sw/packages/gcc/c4.7.0-p1 is the GCC toolchain version 4.7 location on NFS filer.

Thank you
-Indra

Marcel Hlopko

unread,
Aug 14, 2017, 2:37:44 AM8/14/17
to Indra Gunawan, bazel-discuss
I think I see now what is confusing. By "external" repository we mean external to the build tree, so both local_repository and e.g. git_repository qualify as external (we use term remote to be the opposite to local, in git_repository or http_archive are both remote repositories). So your crosstool files will be accessible in external/LinuxGcc47Repo/...
--
-- 
Marcel Hlopko | Software Engineer | hlo...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle | Registergericht und -nummer: Hamburg, HRB 86891

Indra Gunawan

unread,
Aug 15, 2017, 7:55:52 PM8/15/17
to Marcel Hlopko, bazel-discuss
Thank you for the clarification.

A little background about the project.   We are evaluating Bazel to replace a legacy build framework.  It is using a system that have been customized from Perforce JAM.
The workspace is a mono repo and in addition to the toolchain custom gcc on the external NFS filer; it looks like in the jam file definition for the X86_64 there are flags defined for Linkflags and libgcc to refers to $(TOP) directory (this is the root directory of the workspace).

Therefore, some of these flags can’t be defined in CROSSTOOL file as the full path of root workspace is unknown.
How do I refer to the root directory of the workspace in this case?

the following are excerpts in the ToolChainVars.x86_64.jam file:

LINKMAP_x86_64           = $(TOP)/build/src-x86_64/link$(SUFMAP) ;
DLL_LINKMAP_x86_64       = $(TOP)/build/src-x86_64/dll_link$(SUFMAP) ;
LINKLIBS_x86_64          = $(TOP)/linux/os/lib/bin-x86_64/libc$(SUFLIB) ;
CZIP_LINKMAP_x86_64      = $(TOP)/czip/src/link-x86_64$(SUFMAP) ;

LIBGCC_x86_64    default = -L $(TOP)/build/bin-x86_64/$(GCC_NUM_x86_64) -lgcc ;

LINKFLAGS_x86_64         = -r -u _start --build-id=sha1 ;
LINKFLAGS_DLL_x86_64     = -r -G 0 --build-id=sha1 ;
LINKLIB_DLL_x86_64       = --no-whole-archive -Bsymbolic-functions ;

CCFLAGS_DLL_x86_64       = -D__NEED_DLLMAIN__ -fPIC -Wl,--build-id=sha1 ;
CCFLAGS_NODLL_x86_64     = $(ASLR_CFLAGS) ;
if $(GCOV) || $(CFLOW_NG) {
    # enabling gcov (-fprofile-arcs) exposes more implicit function declarations for now bypass the error.
    WARN_x86_64 default = "-Wall -Wpointer-arith -Wformat -Wno-error" ;
} else {
    # -Werror-implicit-function-declaration is a GCC 4.2.1 problem.  When the
    # compiler is upgraded we can use -Werror=implicit-function-declaration
    WARN_x86_64 default = -Wall -Werror -Wpointer-arith -Wformat
                          -Wno-error=enum-compare
                          -Wno-error=switch
                          -Wno-error=unused-variable
                          -Wno-error=unused-function
                          -Wno-error=unused-but-set-variable ;
}

Where and how should I refer to the root workspace variable that is equivalent to $(TOP) above to define additional CCFLAGS for the x86_64 target?
Where should I define conditions ?  I searched about “feature” in CROSSTOOL inside “toolchain_flags”?

Thank you
-Indra

Indra Gunawan

unread,
Aug 17, 2017, 5:20:38 PM8/17/17
to Marcel Hlopko, bazel-discuss
HI Marcel,

any chance you can answer my questions?  

Thank you
-Indra

Marcel Hlopko

unread,
Aug 18, 2017, 5:39:03 AM8/18/17
to Indra Gunawan, bazel-discuss
Hi Indra,

do those files that you reference through $(TOP) need to be in the top level directory? Cannot they be next to/under the directory where the CROSSTOOL is? Since they are all part of the toolchain that's the logical place where we expect them. Or you can make them bazel targets (filegroup, cc_libray, maybe other) a dependency of the cc_toolchain, bazel will then copy them when building the toolchain and you can reference them relatively to the crosstool.

For conditions, right now we try not to have any. If you need conditions, you can take a step back and generate the crosstool using repository rule, similarly to how we do it (https://www.bazel.build/blog/2016/03/31/autoconfiguration.html). But you're lucky, because coverage is actually supported in bazel out of the box (only gcov though). So you can either define 'coverage feature' and bazel it enable it for you when you run `bazel coverage //foo:bar`, or query the existence of 'gcov_gcno_file' variable (such as expand_if_all_available: 'gcov_gcno_file', and expand_none_available: 'gcov_gcno_file'). See the examples in the CppActionConfigs for how to write features.

Be prepared that this will be frustrating, docs are bad, and we change this often (it's mainly what I'm working on these days). Because we're changing it so often now, we don't plan to improve the documentation before we're fully done. So feel free to ask when you get stuck. If you can share a repo with your work in progress that will make answering simpler.



.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/F23AD79C-F111-4CFF-9C36-312B95454FC3%40gmail.com.

For more options, visit https://groups.google.com/d/optout.
--
-- 
Marcel Hlopko | Software Engineer | hlo...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Geschäftsführer: Paul Manicle, Halimah DeLaine Prado | Registergericht und -nummer: Hamburg, HRB 86891

Indra Gunawan

unread,
Aug 25, 2017, 2:27:04 PM8/25/17
to Marcel Hlopko, bazel-discuss
As you recommended, I can have the CROSSTOOL under the directory structure of “build” it is under build/tools directory.
The extra headers I need CROSSTOOL to refer is in ‘build/bin-x86_64/4.7.0/include

The root workspace is the parent directory of “build”
Directly under the root workspace there is “tools” and “build”.
In tools, I put the bazel.rc file


What does the filegroup stanza look like in build/tools/BUILD file to refer to build/bin-x86_64/4.7.0/include as you are saying I do not need build/WORKSPACE to define a new_local_repository.

I have question about cc_toolchain() in the build/tools/BUILD file. 
Shall I define filegroup named “linux_all_files” like this in build/tools/BUILD file.

filegroup(
name = “linux_all_files”,
srcs = [
@buildToolChainRepo//:x86_64_gcc_headers”,
],
)


and set cc_toolchain(
name = ‘cc-compiler_gcc”,
all_files = “:linux_all_files”,
        …
)

Are those above necessary?

Below is what the current build looks like calling gcc:

gcc.c4.7.0-p1.x86_64 -c -include /nobackup/ingunawa/xrv9kws/build/icc-preincl.h \
        -g -pipe -fsigned-char -fno-builtin -nostdinc -fdebug-prefix-map=/nobackup/ingunawa/xrv9kws=/XR_SRC -D__CCI_HW_ENABLE__ -D__SPIRIT_ARCHITECTURE__ -D__QNXNTO__ -D__IOSNG__ -DNTO_BUILD=64 -fPIC -D__X86__ -D__x86_64__ -D__LINUX__ -D__XR_LINUX__ -D_GNU_SOURCE -D__LITTLEENDIAN__ -DOBJECT_X86 -DPLATFORM_INDIRECT_32BIT -DPLATFORM_INDIRECT_16BIT -DPLATFORM_INDIRECT_8BIT -Dfalse=0 -Dtrue=1 -D__BASE_OS_LINUX__ -D__PLATFORM_XRV9K__ -D__NEED_DLLMAIN__ -fPIC -Wl,--build-id=sha1 -D__X86_LINUX__ -DTARGET_CISCO -DQSTAR_REVISIT -D__IOX_DLL__ -DBUILDENV_iox -DRM_DLLMGR_TMP= -D__USE_SHARED_OBJECTS__ -Wall -Werror -Wpointer-arith -Wformat -Wno-error=enum-compare -Wno-error=switch -Wno-error=unused-variable -Wno-error=unused-function -Wno-error=unused-but-set-variable -O0 -D__DO4ARG__  $BASESRCOPT \
        -O1 -fno-strict-aliasing -fno-omit-frame-pointer -Itty/infra/error/src -Itty/infra/error/obj-x86_64-linux-spirit -Icrypto/cepki/ciscossl/include -Ibuild/bin-x86_64/4.7.0/include -I/sw/packages/protoc/v.1.14/include -Iinc/x86_64-linux-spirit/global/iosxr-os -Iinc/x86_64-linux-spirit/global/iosxr-os/os -Iinc/x86_64-linux-spirit/global/iosxr-os/os/lnx -Iinc/x86_64-linux-spirit/global/iosxr-os/os/lnx-wrappers -Iinc/x86_64-linux-spirit/global/iosxr-infra/scripting/slang/slang_210 -Iinfra/xoslib/include -Iinc/x86_64-linux-spirit/package/iosxr-os/tty/infra/error -Iinc/x86_64-linux-spirit/global/iosxr-os/tty/infra/error -Iinc/x86_64-linux-spirit/package/iosxr-os -Iinc/x86_64-linux-spirit/global/iosxr-os -Iinc/x86_64-linux-spirit/global/iosxr-parser -Iinc/x86_64-linux-spirit/global/iosxr-fwding -Iinc/x86_64-linux-spirit/global/iosxr-infra \
         -include /nobackup/ingunawa/xrv9kws/infra/rtd/osc/bosc/include/osc_header.h  -o /nobackup/ingunawa/xrv9kws/tty/infra/error/obj-x86_64-linux-spirit/G0_tty_error.o /nobackup/ingunawa/xrv9kws/tty/infra/error/src/tty_error.c “


See above "-Ibuild/bin-x86_64/4.7.0/include” .  How to refer to that in CROSSTOOL with cxx_builtin_include_directory? 

I am also asking about conditions in general.  Does “compiler_flags” feature work for any if I define it in CROSSTOOL?

Can you provide an example with a sample CROSSTOOL that uses it?

Thank you


On Aug 25, 2017, at 5:59 AM, Marcel Hlopko <hlo...@google.com> wrote:

Hi Indra,

can we move this discussion back to the bazel-discuss so other people can benefit from the solutions we come up with? Obviously helping every crosstool author in private doesn't scale well.

Re the project you've sent:

WORKSPACE: You don't need new_local_repository for packages that are under the current WORKSPACE. 
tools/BUILD: cc_toolchain rule is there to collect all the extra input files your toolchain depends on. In your case, you depend on x86_64_gcc_headers, so that filegroup needs to be referenced from cc_toolchain. Bazel will then make sure to copy all the headers to the appropriate directories when compiling in sandbox.
tools/CROSSTOOL: seems ok.

With the headers you provide, don't you need e.g. -lssl somewhere? If you can trust your linker to find the libraries, then just adding linker_flag: '-lssl' is enough. If you cannot, then this is where you would use new_local_repository - to get to the libssl.a/libssl.so :

new_local_repository(
    name = "ssl",
    path = "/usr/lib/x86_64-linux-gnu",
    build_file_content = """
cc_library(
    name = "ssl",
    srcs = ["libssl.so"],
    visibility = ["//visibility:public"],
)
"" "
)

And then depend on @ssl//:ssl from your cc_libraries/cc_binaries.

Does it make sense at all what I wrote? :)

On Wed, Aug 23, 2017 at 10:42 PM Indra Gunawan <ind...@gmail.com> wrote:
HI Marcel,

Do you have a chance to take a look at my working in progress repo?

Thank you
-Indra

On Aug 23, 2017, at 7:28 AM, Indra Gunawan <ind...@gmail.com> wrote:

Hi Marcel,

I can move the location of the toolchain CROSSTOOL to be inside $(TOP)/build tree directory such that the $(TOP)/build/bin-x86_64/4.7.0/include can be referred in the CROSSTOOL.
Would you please review if I have done so correctly.  Attached is the zip file of my work in progress git repo.


<bazel-repo.zip.bin>



I have question about cc_toolchain() in the build/tools/BUILD file. 
Shall I define filegroup named “linux_all_files” like this in build/tools/BUILD file.

filegroup(
name = “linux_all_files”,
srcs = [
@buildToolChainRepo//:x86_64_gcc_headers”,
],
)


and set cc_toolchain(
name = ‘cc-compiler_gcc”,
all_files = “:linux_all_files”,
        …
)

Are those above necessary?

I just want to make sure that gcc compiler is called like below eventually:

gcc.c4.7.0-p1.x86_64 -c -include /nobackup/ingunawa/xrv9kws/build/icc-preincl.h \
        -g -pipe -fsigned-char -fno-builtin -nostdinc -fdebug-prefix-map=/nobackup/ingunawa/xrv9kws=/XR_SRC -D__CCI_HW_ENABLE__ -D__SPIRIT_ARCHITECTURE__ -D__QNXNTO__ -D__IOSNG__ -DNTO_BUILD=64 -fPIC -D__X86__ -D__x86_64__ -D__LINUX__ -D__XR_LINUX__ -D_GNU_SOURCE -D__LITTLEENDIAN__ -DOBJECT_X86 -DPLATFORM_INDIRECT_32BIT -DPLATFORM_INDIRECT_16BIT -DPLATFORM_INDIRECT_8BIT -Dfalse=0 -Dtrue=1 -D__BASE_OS_LINUX__ -D__PLATFORM_XRV9K__ -D__NEED_DLLMAIN__ -fPIC -Wl,--build-id=sha1 -D__X86_LINUX__ -DTARGET_CISCO -DQSTAR_REVISIT -D__IOX_DLL__ -DBUILDENV_iox -DRM_DLLMGR_TMP= -D__USE_SHARED_OBJECTS__ -Wall -Werror -Wpointer-arith -Wformat -Wno-error=enum-compare -Wno-error=switch -Wno-error=unused-variable -Wno-error=unused-function -Wno-error=unused-but-set-variable -O0 -D__DO4ARG__  $BASESRCOPT \
        -O1 -fno-strict-aliasing -fno-omit-frame-pointer -Itty/infra/error/src -Itty/infra/error/obj-x86_64-linux-spirit -Icrypto/cepki/ciscossl/include -Ibuild/bin-x86_64/4.7.0/include -I/sw/packages/protoc/v.1.14/include -Iinc/x86_64-linux-spirit/global/iosxr-os -Iinc/x86_64-linux-spirit/global/iosxr-os/os -Iinc/x86_64-linux-spirit/global/iosxr-os/os/lnx -Iinc/x86_64-linux-spirit/global/iosxr-os/os/lnx-wrappers -Iinc/x86_64-linux-spirit/global/iosxr-infra/scripting/slang/slang_210 -Iinfra/xoslib/include -Iinc/x86_64-linux-spirit/package/iosxr-os/tty/infra/error -Iinc/x86_64-linux-spirit/global/iosxr-os/tty/infra/error -Iinc/x86_64-linux-spirit/package/iosxr-os -Iinc/x86_64-linux-spirit/global/iosxr-os -Iinc/x86_64-linux-spirit/global/iosxr-parser -Iinc/x86_64-linux-spirit/global/iosxr-fwding -Iinc/x86_64-linux-spirit/global/iosxr-infra \
         -include /nobackup/ingunawa/xrv9kws/infra/rtd/osc/bosc/include/osc_header.h  -o /nobackup/ingunawa/xrv9kws/tty/infra/error/obj-x86_64-linux-spirit/G0_tty_error.o /nobackup/ingunawa/xrv9kws/tty/infra/error/src/tty_error.c “


See above "-Ibuild/bin-x86_64/4.7.0/include” .  Adding it in CROSSTOOL as :  "cxx_builtin_include_directory: "%package(@buildToolChainRepo//bin-x86_64/4.7.0/include)%””  is fine?

I am also asking about conditions in general.  Does “compiler_flags” feature work for any if I define it in CROSSTOOL?

Can you provide an example with a sample CROSSTOOL that uses it?

Thank you
-Indra




-- 
Google Germany GmbH | Erika-Mann-Str. 33  | Contact | Germany Managing Director: Managing Director: Paul Manicle, Halimah DeLaine Prado  | Register court and number: Hamburg, HRB 86891

Marcel Hlopko

unread,
Aug 30, 2017, 4:46:52 AM8/30/17
to Indra Gunawan, bazel-discuss
Hi Indra,

hopefully I'll have a full answer for you now :) Thank you for your patience!

So this is what I assume:

/
  - BUILD
  - tools
    - bazelrc
  - build
    - tools
      BUILD
      CROSSTOOL
      build/bin-x86_64/4.7.0/include
      build/bin-x86_64/4.7.0/include/foo.h
  - srcs
    - main.cc

/tools/bazelrc:

    build --crosstool_top=//build/tools:toolchain

/BUILD:

cc_binary(
    name = "main",
    srcs = [ "main.cc" ],
)


/build/tools/BUILD:

package(default_visibility = ['//visibility:public'])

cc_toolchain_suite(
  name = 'toolchain',
  toolchains = {
    'k8|clang': ':clang_toolchain',
  },
)

cc_toolchain(
  name = 'clang_toolchain',
  all_files = ':custom_includes',
  compiler_files = ':custom_includes',
  cpu = 'k8',
  dwp_files = ':empty',
  dynamic_runtime_libs = [':empty'],
  linker_files = ':empty',
  objcopy_files = ':empty',
  static_runtime_libs = [':empty'],
  strip_files = ':empty',
  supports_param_files = 1,
)

filegroup(
    name = "empty",
)

filegroup(
    name = "custom_includes",
    srcs = glob(["my/foo/includes/**/*.h"]),
)

/build/tools/CROSSTOOL:

major_version: 'local'
minor_version: ''
default_target_cpu: 'k8'

default_toolchain {
  cpu: 'k8'
  toolchain_identifier: 'local_linux'
}

toolchain {
  abi_version: 'local'
  abi_libc_version: 'local'
  compiler: 'clang'
  host_system_name: 'local'
  needsPic: true
  target_libc: 'local'
  target_cpu: 'k8'
  target_system_name: 'local'
  toolchain_identifier: 'local_linux'
  cxx_builtin_include_directory: '/usr/include'
  cxx_builtin_include_directory: '/usr/lib/gcc'
  cxx_builtin_include_directory: 'my/foo/includes'

  cxx_flag: '-isystem'
  cxx_flag: 'my/foo/includes'

  # Set clang as a C/C++ compiler.
  tool_path { name: 'gcc' path: '/usr/bin/gcc' }

  # Use the default system toolchain for everything else.
  tool_path { name: 'ar' path: '/usr/bin/ar' }
  tool_path { name: 'compat-ld' path: '/usr/bin/ld' }
  tool_path { name: 'cpp' path: '/usr/bin/cpp' }
  tool_path { name: 'dwp' path: '/usr/bin/dwp' }
  tool_path { name: 'gcov' path: '/usr/bin/gcov' }
  tool_path { name: 'ld' path: '/usr/bin/ld' }
  tool_path { name: 'nm' path: '/usr/bin/nm' }
  tool_path { name: 'objcopy' path: '/usr/bin/objcopy' }
  tool_path { name: 'objdump' path: '/usr/bin/objdump' }
  tool_path { name: 'strip' path: '/usr/bin/strip' }
}

/build/tools/build/bin-x86_64/4.7.0/include/foo.h:

int foo() { return 42; }

/srcs/main.cc:

#include <foo.h>

int main() {
  return foo();
}




ind...@gmail.com

unread,
Aug 30, 2017, 7:03:31 AM8/30/17
to Marcel Hlopko, bazel-discuss
There is no my/foo/includes directory tree in the folder where CROSSTOOL is located though.

In my case,  there is no /build/tools/build/bin-x86_64/4.7.0/include/ 

There are:
/
  - BUILD
  - tools
        - bazelrc
   - build
         - tools
            BUILD
            CROSSTOOL
         - bin-x86_64/4.7.0/include

At the moment, I decide to not refer to the custom includes from toolchain.  Instead, I define each custom includes  -- there are several include directories not only in /build as cc_library.  They are added as deps in BUILD file of whichever to use it with copts.
I am thinking to try to use MACRO in .bzl file to specify conditionals with "select" to add these copts because I was mistaken that not every component need the custom includes.

Thank you for being patient.  

Sent from my iPhone

Josh Powell

unread,
Dec 4, 2017, 12:57:19 AM12/4/17
to bazel-discuss
Thank you very much for this details post, Marcel!

I'm trying to modify it to use a new_local_repository containing my installation of Clang to exercise some C++17 features present in Clang 6. I think I've set up the globally-present headers wrong. Can you help me spot the error? Except where otherwise stated, my Bazel workspace is set up as the one you've described here.

I've modifed /main.cc to

#include <iostream>
#include <optional>
#include <string>

int main(int argc, char* argv[]) {
  std::cout << std::optional<std::string>("nonempty").value_or("empty") << std::endl;
  return 0;
}

I've modified /WORKSPACE to

new_local_repository(
    name = "clang_installation",
    build_file_content = """
package(default_visibility=["//visibility:public"])

filegroup(
  name = "include",
  srcs = glob(["include/c++/v1/**/*"]),
)

filegroup(
  name = "lib",
  srcs = glob(["lib/**/*"]),
)
""",
    path = "/usr/local/clang/6.0.0",
)

I've modified /build/tools/BUILD to

package(default_visibility = ["//visibility:public"])

cc_toolchain_suite(
    name = "toolchain",
    toolchains = {
        "k8|clang": ":clang_toolchain",
    },
)

cc_toolchain(
    name = "clang_toolchain",
    all_files = "@clang_installation//:include",
    compiler_files = "@clang_installation//:include",
    cpu = "k8",
    dwp_files = ":empty",
    dynamic_runtime_libs = [":empty"],
    linker_files = ":empty",
    objcopy_files = ":empty",
    static_runtime_libs = [":empty"],
    strip_files = ":empty",
    supports_param_files = 1,
)

filegroup(
    name = "empty",
)

Lastly, I've modified /build/tools/CROSSTOOL to

major_version: 'local'
minor_version: ''
default_target_cpu: 'k8'

default_toolchain {
  cpu: 'k8'
  toolchain_identifier: 'local_linux'
}

toolchain {
  abi_version: 'local'
  abi_libc_version: 'local'
  compiler: 'clang'
  host_system_name: 'local'
  needsPic: true
  target_libc: 'local'
  target_cpu: 'k8'
  target_system_name: 'local'
  toolchain_identifier: 'local_linux'
  cxx_builtin_include_directory: '/usr/include'
  cxx_builtin_include_directory: '/usr/lib/gcc'
  cxx_builtin_include_directory: '%my/foo/includes'

  cxx_builtin_include_directory: %package(@clang_installation//include)%

  cxx_flag: "-std=c++1y"
  host_system_name: "local"
  linker_flag: "-latomic"
  linker_flag: "-lm"
  linker_flag: "-lpthread"
  linker_flag: "--sysroot=external/clang_installation"

  cxx_flag: "-std=c++1z"

  # Include paths
  compiler_flag: "--sysroot=external/org_chromium_sysroot_linux_x64"
  compiler_flag: "-nostdinc++"
  compiler_flag: "-isystem"
  compiler_flag: "external/clang_installation/lib/clang/6.0.0/include"

  # Set clang as a C/C++ compiler.
  tool_path { name: 'gcc' path: '/usr/local/clang/6.0.0/bin/clang' }

  # Use the default system toolchain for everything else.
  tool_path { name: 'ar' path: '/usr/bin/ar' }
  tool_path { name: 'compat-ld' path: '/usr/bin/ld' }
  tool_path { name: 'cpp' path: '/usr/bin/cpp' }
  tool_path { name: 'dwp' path: '/usr/bin/dwp' }
  tool_path { name: 'gcov' path: '/usr/bin/gcov' }
  tool_path { name: 'ld' path: '/usr/bin/ld' }
  tool_path { name: 'nm' path: '/usr/bin/nm' }
  tool_path { name: 'objcopy' path: '/usr/bin/objcopy' }
  tool_path { name: 'objdump' path: '/usr/bin/objdump' }
  tool_path { name: 'strip' path: '/usr/bin/strip' }
}

Attempting to build the cc_binary fails.

$ bazel build :main --crosstool_top=//build/tools:toolchain --verbose_failures
ERROR: /home/josh/dev/crosstool_demo/build/tools/BUILD:10:1: in cc_toolchain rule //build/tools:clang_toolchain: The package '@clang_installation//:include' is not valid                                                                  
ERROR: Analysis of target '//:main' failed; build aborted: Analysis of target '//build/tools:clang_toolchain' failed; build aborted                                                                                                        
INFO: Elapsed time: 0.346s
FAILED: Build did NOT complete successfully (1 packages loaded)

I'm struggling to understand what is invalid about that package. I can build it directly without problem.

$ bazel build @clang_installation//:include --verbose_failures
Loading: 
Loading: 0 packages loaded
INFO: Analysed target @clang_installation//:include (5 packages loaded).
INFO: Found 1 target...
bazel: Entering directory `/home/josh/.cache/bazel/_bazel_josh/de18d5e7cc2fb6ae96a00dbaca113739/execroot/__main__/'
[0 / 1] BazelWorkspaceStatusAction stable-status.txt
bazel: Leaving directory `/home/josh/.cache/bazel/_bazel_josh/de18d5e7cc2fb6ae96a00dbaca113739/execroot/__main__/'
Target @clang_installation//:include up-to-date (nothing to build)
INFO: Elapsed time: 0.390s, Critical Path: 0.00s
INFO: Build completed successfully, 1 total action

Any help would be greatly appreciated!

Marcel Hlopko

unread,
Dec 12, 2017, 8:08:45 AM12/12/17
to Josh Powell, bazel-discuss
Can you try just with only using external/clang_installation paths (so not using package(@clang_installation) thing, I thing it's broken but had no time to investigate further)? Plus, I think you want to declare cc_toolchain inputs for @clang_installation//lib too.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/66d26d7b-70d8-43a3-bf7c-b2f03aa8b2d5%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
--
-- 
Marcel Hlopko | Software Engineer | hlo...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Geschäftsführer: Paul Manicle, Halimah DeLaine Prado | Registergericht und -nummer: Hamburg, HRB 86891
Reply all
Reply to author
Forward
0 new messages