Enabling c++17 features using CC (and cc_configure) to generate CROSSTOOL for g++ instead of clang

[Wade]

I want to get C++17 features working in Bazel. My macbook is using clang as its default C++ compiler and does not appear to be supporting c++17, so I've compiled g++ locally (which does support c++17).

I have the following files in a directory:

WORKSPACE

 <empty>

BUILD

  cc_binary(

      name = "main",

      srcs = ["main.cc"],

  )

main.cc

  #include <iostream>

  #include <any>

  int main() {

    std::any val1 = 1;

    std::cout << "val1 = " << std::any_cast<int>(val1) << std::endl;

  }

I'm working on a Macbook Pro using

• macos    = 10.12
  
• xcode     = 9.1
• bazel      = 0.8.1-homebrew
• clang++  = Apple LLVM version 9.0.0 (clang-900.0.38)  [in /usr/bin/clang++]
• g++         = 7.2.0 built from source [in /usr/local/wmh/gcc-7.2/bin/g++]

Some observations follow...

clang doesn't yet handle -std=c++17

% /usr/bin/clang++ -std=c++17 -o main main.cc

error: invalid value 'c++17' in '-std=c++17'

supposedly we use -std=c++1z for LLVM 4.0, but it doesn't appear to understand std::any (and /usr/include has no files defining 'any')

% /usr/bin/clang++ -std=c++1z -o main main.cc

main.cc:2:10: fatal error: 'any' file not found

#include <any>

         ^~~~~

1 error generated.

 

% find /usr/include -name '*any*'

/usr/include/apr-1/apr_anylock.h

g++ does the right thing.

% /usr/local/wmh/gcc-7.2/bin/g++ -std=c++17 -o main main.cc

% ./main

val1 = 1 

bazel out-of-the-box will use clang. Below, I'm setting CC explicitly just to ensure the generated CROSSTOOL is for clang

% bazel clean --expunge

% export CC=/usr/bin/clang++

% blaze build --cxxopt=-std=c++1z --verbose_failures :main

... snip ...

main.cc:2:10: fatal error: 'any' file not found

#include <any>

         ^~~~~

1 error generated.

... snip ...

Let's save the CROSSTOOL file for comparision purposes:

% cp $(find $(bazel info execution_root) -name CROSSTOOL) /tmp/CT.clang

So now I'm hoping cc_configure will fix things for me.  From https://bazel.build/designs/2016/02/16/cpp-autoconf.html and https://blog.bazel.build/2016/03/31/autoconfiguration.html, my understanding is that setting CC should result in a new CROSSTOOL file being generated based on that compiler (g++ in my case):

% bazel clean --expunge

% export CC=/usr/local/wmh/gcc-7.2/bin/g++

% blaze build --cxxopt=-std=c++17 --verbose_failures --sandbox_debug :main

...........

Loading: 

Loading: 0 packages loaded

Analyzing: target //:main (4 packages loaded)

Analyzing: target //:main (4 packages loaded)

Analyzing: target //:main (7 packages loaded)

Analyzing: target //:main (9 packages loaded)

INFO: Analysed target //:main (9 packages loaded).

INFO: Found 1 target...

bazel: Entering directory `/private/var/tmp/_bazel_wmh/e02603ea880b3ada6404b60673612989/execroot/__main__/'

[0 / 5] Creating source manifest for //:main

ERROR: /Users/wmh/src/tests/cc/c++17/BUILD:1:1: C++ compilation of rule '//:main' failed (Exit 1): sandbox-exec failed: error executing command 

  (cd /private/var/tmp/_bazel_wmh/e02603ea880b3ada6404b60673612989/execroot/__main__ && \

  exec env - \

    APPLE_SDK_PLATFORM=MacOSX \

    APPLE_SDK_VERSION_OVERRIDE=10.13 \

    BLAH=blork \

    DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer \

    PATH=/usr/bin:/sw/bin:/opt/local/bin:/usr/texbin:/Applications/Emacs.app/Contents/MacOS/bin:/usr/local/texlive/2016/bin/universal-darwin:/opt/X11/bin:/usr/local/symlinks:/usr/local/scripts:/usr/local/bin:/usr/local/sbin:/sbin:/usr/sbin:/bin:. \

    SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk \

    TMPDIR=/private/var/tmp/_bazel_wmh/e02603ea880b3ada6404b60673612989/bazel-sandbox/3461531722433772538/execroot/__main__/tmp \

    XCODE_VERSION_OVERRIDE=9.1.0 \

  /usr/bin/sandbox-exec -f /private/var/tmp/_bazel_wmh/e02603ea880b3ada6404b60673612989/bazel-sandbox/3461531722433772538/sandbox.sb /private/var/tmp/_bazel_wmh/e02603ea880b3ada6404b60673612989/execroot/__main__/_bin/process-wrapper '--timeout=0' '--kill_delay=15' external/local_config_cc/wrapped_clang '-D_FORTIFY_SOURCE=1' -fstack-protector -fcolor-diagnostics -Wall -Wthread-safety -Wself-assign -fno-omit-frame-pointer -O0 -DDEBUG '-std=c++11' '-std=c++17' -iquote . -iquote bazel-out/darwin-fastbuild/genfiles -iquote external/bazel_tools -iquote bazel-out/darwin-fastbuild/genfiles/external/bazel_tools -isystem external/bazel_tools/tools/cpp/gcc3 -MD -MF bazel-out/darwin-fastbuild/bin/_objs/main/main.d '-frandom-seed=bazel-out/darwin-fastbuild/bin/_objs/main/main.o' '-isysroot __BAZEL_XCODE_SDKROOT__' -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c main.cc -o bazel-out/darwin-fastbuild/bin/_objs/main/main.o)

error: invalid value 'c++17' in '-std=c++17'

bazel: Leaving directory `/private/var/tmp/_bazel_wmh/e02603ea880b3ada6404b60673612989/execroot/__main__/'

Target //:main failed to build

INFO: Elapsed time: 8.678s, Critical Path: 0.11s

FAILED: Build did NOT complete successfully

The "invalid value 'c++17' in '-std=c++17'" error looks an awful lot like it is still using clang.  The fact that the code is invoking external/local_config_cc/wrapped_clang isn't promising either ... and in looking at that file it does indeed appear to be invoking clang.  However, when I compare the new CROSSTOOL file with the old one, there are lots of diffs (so something different happened when I changed the value of CC) but the diffs are ONLY for cxx_builtin_include_directory variables (I would expect some other variable to change to indicate that we are using g++ instead of clang, no?).

% cp $(find $(bazel info execution_root) -name CROSSTOOL) /tmp/CT.gcc

% diff /tmp/CT.clang /tmp/CT.gcc

60c60,63

< cxx_builtin_include_directory: "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1"

---

> cxx_builtin_include_directory: "/usr/local/wmh/gcc-7.2/include/c++/7.2.0"

> cxx_builtin_include_directory: "/usr/local/wmh/gcc-7.2/include/c++/7.2.0/x86_64-apple-darwin16.7.0"

> cxx_builtin_include_directory: "/usr/local/wmh/gcc-7.2/include/c++/7.2.0/backward"

> cxx_builtin_include_directory: "/usr/local/wmh/gcc-7.2/lib/gcc/x86_64-apple-darwin16.7.0/7.2.0/include"

62,63c65,66

< cxx_builtin_include_directory: "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/include"

< cxx_builtin_include_directory: "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include"

---

> cxx_builtin_include_directory: "/usr/local/wmh/gcc-7.2/include"

> cxx_builtin_include_directory: "/usr/local/wmh/gcc-7.2/lib/gcc/x86_64-apple-darwin16.7.0/7.2.0/include-fixed"

... lots more diffs snipped...

Note that in using a much older version (0.3.2) of bazel before upgrading to 0.8.1, I was seeing a different error at this point ... the compilation of the file worked fine (and was thus definitely using the g++ binary) but was failing because it "could not find ld".  The newer version of bazel appears to be a regression ... at least 0.3.2 would use g++ instead of clang.

Questions:

1. Any thoughts on how I proceed from here to get bazel to work with a locally compiled g++? 
   
   
2. Alternatively, how can I get c++17 working with clang 9.0.0?
   
   
3. Alternatively, how can I get c++17 working with bazel, regardless of which compiler it uses ;-)

P.S. I've attached the CROSSTOOL file generated when CC is set to my locally built g++, in case there are some simple changes that could be made to it that would get things using g++ instead of clang.

Thanks!

Wade

[mpi...@gmail.com]

I'm pretty new to bazel, but one thing jumps out at me: you're running one bazel command and one blaze command. Did you intend to do that?

Regards
Mark

On Saturday, December 16, 2017 at 10:36:56 PM UTC-5, Wade wrote:
> I want to get C++17 features working in Bazel. My macbook is using clang as its default C++ compiler and does not appear to be supporting c++17, so I've compiled g++ locally (which does support c++17).
>
>
> I have the following files in a directory:
>
>
> WORKSPACE
>  <empty>
>
>
> BUILD
>   cc_binary(
>       name = "main",
>       srcs = ["main.cc"],
>   )
>
>
> main.cc
>   #include <iostream>
>   #include <any>
>
>
>   int main() {
>     std::any val1 = 1;
>     std::cout << "val1 = " << std::any_cast<int>(val1) << std::endl;
>   }
>
>
> I'm working on a Macbook Pro using
> macos    = 10.12

> xcode     = 9.1bazel      = 0.8.1-homebrewclang++  = Apple LLVM version 9.0.0 (clang-900.0.38)  [in /usr/bin/clang++]g++         = 7.2.0 built from source [in /usr/local/wmh/gcc-7.2/bin/g++]

[Wade Holst]

I'm pretty new to bazel, but one thing jumps out at me: you're running one bazel command and one blaze command. Did you intend to do that?

Thanks for the comment, Mark.  Alas, that isn't the culprit ... I just have 'blaze' aliased to 'bazel' (both blaze and bazel do the same thing).

I can presumably use https://github.com/bazelbuild/bazel/wiki/Building-with-a-custom-toolchain to do this, but https://bazel.build/designs/2016/02/16/cpp-autoconf.html looked promising as a much simpler solution.

 

[Marcel Hlopko]

Hello,

The problem is that bazel by default when it detects xcode installed will generate a toolchain that supports both C++ and ObjC, and for that toolchain it's assumed that you want to use clang that's coming with Xcode, so CC doesn't have an effect. You can verify that by looking into external/local_config_cc/wrapped_clang. If you don't care about ObjC, since https://github.com/bazelbuild/bazel/commit/209a975ea61e40a184dd88aed85025e3f65a778d you can use BAZEL_USE_CPP_ONLY_TOOLCHAIN=1 to force C++ only toolchain that works with CC. The commit is not yet part of any bazel release, you have to build your own bazel from HEAD.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CAHe11MSMcafivP0KmDTZCjzN7Ne%3DSGfiO%3DunGi_NKJa4Td-aVA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

-- 

Marcel Hlopko | Software Engineer | hlo...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Geschäftsführer: Paul Manicle, Halimah DeLaine Prado | Registergericht und -nummer: Hamburg, HRB 86891

[Wade Holst]

Hello,

Hi Marcel.  Thanks for the response!

 

The problem is that bazel by default when it detects xcode installed will generate a toolchain that supports both C++ and ObjC, and for that toolchain it's assumed that you want to use clang that's coming with Xcode, so CC doesn't have an effect. You can verify that by looking into external/local_config_cc/wrapped_clang. If you don't care about ObjC, since https://github.com/bazelbuild/bazel/commit/209a975ea61e40a184dd88aed85025e3f65a778d you can use BAZEL_USE_CPP_ONLY_TOOLCHAIN=1 to force C++ only toolchain that works with CC. The commit is not yet part of any bazel release, you have to build your own bazel from HEAD.

Excellent.  At the moment I don't need ObjC, so I'll try that fix.  At some point in the future I will need both C++ and ObjC.  Are there plans to have configure_cc work for both eventually?

 

[Wade Holst]

I managed to get something working, but only by hacking the generated CROSSTOOL file. 

First, we obtain a version of bazel that supports BAZEL_USE_CPP_ONLY_TOOLCHAIN:

% cd <somedir>

% git clone https://github.com/bazelbuild/bazel.git

% cd bazel

# Note that to compile bazel from head you need a previously installed

# version of bazel to bootstrap from ... I'm using bazel-0.9.0rc6 below:

% bazel build //src:bazel

% cp bazel-bin/src/bazel $HOME/bin/bazel

% which bazel

bazel is /Users/wmh/bin/bazel

Side Note: 'bazel version' does not work properly for these builds:

% bazel version

Extracting Bazel installation...

............

Build target: bazel-out/darwin-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar

Build time: Thu Jan 01 00:00:00 1970 (0)

Build timestamp: Thu Jan 01 00:00:00 1970 (0)

Build timestamp as int: 0

Now back to compiling my toy C++17-enabled main.cc program (from my OP):

% cd <toydir>

% bazel clean --expunge

INFO: Starting clean.

% ls

BUILD   WORKSPACE   main.cc

Let's see if bazel can generate a g++-flavored CROSSTOOL:

% export CC=/usr/local/wmh/gcc-7.2/bin/g++

% export BAZEL_USE_CPP_ONLY_TOOLCHAIN=1

% bazel build --cxxopt=-std=c++17 --verbose_failures --sandbox_debug :main

... output snipped ...

g++: error: unrecognized command line option '-Wthread-safety'; did you mean '-fthread-jumps'?

g++: error: unrecognized command line option '-Wself-assign'; did you mean '-Wcast-align'?

... output snipped ...

Maybe hacking the CROSSTOOL file will help:

% crosstool="$(bazel info output_base)/external/local_config_cc/CROSSTOOL"

% cp ${crosstool} $HOME/CROSSTOOL   # so we can compare later

% perl -i -ne 'print unless m/-Wthread-safety|-Wself-assign/;' ${crosstool}

% bazel build --cxxopt=-std=c++17 --verbose_failures --sandbox_debug :main

... output snipped ...

ld: unknown option: -no-as-needed

... output snipped ...

We have a different error, so we are making progress! Looking at the CROSSTOOL file, the -no-as-needed flag is coming from

    linker_flag: "-Wl,-no-as-needed"

To fix:

% perl -i -ne 'print unless m/-Wl,-no-as-needed/;' ${crosstool}

% bazel build --cxxopt=-std=c++17 --verbose_failures --sandbox_debug :main

... output snipped ...

ld: unknown option: -z

... output snipped ...

The -z flag is used in

    linker_flag: "-Wl,-z,relro,-z,now"

To fix:

% perl -i -ne 'print unless m/-Wl,-z,relro,-z,now/;' ${crosstool}

% bazel build --cxxopt=-std=c++17 --verbose_failures --sandbox_debug :main

... output snipped ...

ld: library not found for -l:libstdc++.a

... output snipped ...

Which is coming from the following stanza in CROSSTOOLS

    linking_mode_flags {

      mode: MOSTLY_STATIC

      linker_flag: "-l:libstdc++.a"

    }

To fix:

% $EDITOR $crosstool

# delete above stanza and save (for me, it is the toolchain with

# toolchain_identifier "local" that is relevant, but it may differ for you)

% bazel build --cxxopt=-std=c++17 --verbose_failures --sandbox_debug :main

Loading: 

Loading: 0 packages loaded

INFO: Analysed target //:main (0 packages loaded).

INFO: Found 1 target...

bazel: Entering directory `/private/var/tmp/_bazel_wmh/e02603ea880b3ada6404b60673612989/execroot/__main__/'

[0 / 4] [-----] BazelWorkspaceStatusAction stable-status.txt

bazel: Leaving directory `/private/var/tmp/_bazel_wmh/e02603ea880b3ada6404b60673612989/execroot/__main__/'

Target //:main up-to-date:

  bazel-bin/main

INFO: Elapsed time: 0.247s, Critical Path: 0.07s

INFO: Build completed successfully, 2 total actions

And bazel has now successfully built a C++ program using g++ on macos:

% bazel-bin/main

val1 = 1

However, since there was some hackery involved in this, it is inherently fragile. Anytime I do a 'bazel clean --expunge', I'll tromp on the CROSSTOOL file.  Is there a way to formally register a specific CROSSTOOL file with bazel such that bazel uses the registered CROSSTOOL file but everything else stays the same?  Something much more lightweight than "Building a custom toolchain" or "Yet Another CROSSTOOL Writing Tutorial" (both of which are very useful, but are a bit much when one just needs to hack up a CROSSTOOL file slightly).

[Austin Schuh]

You can copy the CROSSTOOL file (and BUILD file) you ended up tweaking into your repo and pass it in with crosstool_top in your //tools/bazel.rc file.  Keep notes around on how to do it again if you want to re-generate.

Austin

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CAHe11MRxnCvLmczop5YW7rQmGcJmJvibH5A8N__UVz%3D9QXJsNw%40mail.gmail.com.

[Wade Holst]

You can copy the CROSSTOOL file (and BUILD file) you ended up tweaking into your repo and pass it in with crosstool_top in your //tools/bazel.rc file.  Keep notes around on how to do it again if you want to re-generate.

Thanks Austin ... this sounds promising.  However, I'm not clear about the details.  All of the examples I've seen of how --crosstool_top is used appear to involve building a custom toolchain (https://github.com/bazelbuild/bazel/wiki/Building-with-a-custom-toolchain), and I'm trying to avoid that complexity here.

If I have exactly one modified file (CROSSTOOL), what would the target associated with --crosstool_top look like? Where would my modified CROSSTOOL file reside?  Which pre-existing BUILD file would I bundle with the new CROSSTOOL file (I'm assuming from your comment that both CROSSTOOL and BUILD are needed here).  As you can tell from these questions, I'm still rather fuzzy on the details ;-)

[Austin Schuh]

The cc_configure rule automatically generates everything needed and points --crosstool_top to the folder with the generated CROSSTOOL file.

So, it should be as simple as copying everything in the generated folder  that you found and have been modifying to something like //tools/cpp and passing in --crosstool_top=//tools/cpp:toolchain

I've written/tweaked a number of crosstool files, but haven't done exactly what you are trying to do.

Austin