Using custom clang with libc++

[Henrique Bucher]

Hello folks, 

We a custom clang with additional homegrown optimization passes for our application. We are incorporating tensorflow as a shared library and for that we had to customize the build, which has been achieved. Bazel compiles tensorflow with our custom clang and generates the .so library as expected.

The problem now is that when we try to create the library using libc++ instead of the default GCC libstdc++, it spews out this error:

$ bazel build --cxxopt=-std=c++11 --cxxopt=-stdlib=libc++ tensorflow:libtensorflow_all.so

INFO: Found 1 target...

INFO: From Compiling external/protobuf/src/google/protobuf/compiler/js/embed.cc [for host]:

external/protobuf/src/google/protobuf/compiler/js/embed.cc:37:12: warning: unused variable 'output_file' [-Wunused-const-variable]

const char output_file[] = "well_known_types_embed.cc";

           ^

1 warning generated.

ERROR: /home/hbucher/.cache/bazel/_bazel_hbucher/ad427c7fddd5b68de5e1cfaa7cd8c8cc/external/com_googlesource_code_re2/BUILD:11:1: undeclared inclusion(s) in rule '@com_googlesource_code_re2//:re2':

this rule is missing dependency declarations for the following files included by 'external/com_googlesource_code_re2/re2/bitstate.cc':

  '/home/hbucher/install/include/c++/v1/stddef.h'

  '/home/hbucher/install/include/c++/v1/__config'

  '/home/hbucher/install/include/c++/v1/__nullptr'

  '/home/hbucher/install/include/c++/v1/stdint.h'

  '/home/hbucher/install/include/c++/v1/string.h'

  '/home/hbucher/install/include/c++/v1/stdio.h'

I am struggling with the path of creating a custom toolchain so my question to you is:

Is there any way to just tweak the zipped bazel distribution (bazel-4.5.0-dist.zip) such that I can get through this without creating a custom toolchain?

Alternatively, would you have a ready example of files for a custom clang/x86_64 I can use with --crosstool_top?

Thank you.

[Robert Tsai (Verb Ext.)]

The most easily-accessible "ready example" is:

1. bazel build //something
2. cp $HOME/bazel-<workspace_name>/external/local_config_cc/{BUILD,CROSSTOOL} $HOME/tmp

Use those BUILD and CROSSTOOL files as a template for your own toolchain.

--Rob

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/20533346-7eac-41e6-a785-62660d75a116%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Austin Schuh]

You want something like the following in your CROSSTOOL file.

cxx_builtin_include_directory: "%package(@clang_3_7_repo//lib/clang/3.7.1/include)%"

The %package(@...)% stanza computes the path to the package with the compiler in it.  You can also just do

cxx_builtin_include_directory: "/home/hbucher/install/include/c++/v1"

if you want to require everyone to install the compiler at that path.

Austin

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/20533346-7eac-41e6-a785-62660d75a116%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CACJAxRzvDQ08WF4wbKE9mscfqG2GkG%2BxZ36n14GSk8SFyNMJvQ%40mail.gmail.com.

[Henrique Bucher]

Thanks for your answers!

I added the section below to tensorflor/WORKSPACE 

new_local_repository(

  name="hbclang",

  path="/home/hbucher/BazelCustomToolchain",

  build_file = "BUILD",

)

where /home/hbucher/BazelCustomToolchain points to the git checkout of 

https://github.com/HFTrader/BazelCustomToolchain

$ ls /home/hbucher/BazelCustomToolchain/ -l

-rwxrwxr-x 1 hbucher hbucher   896 May 15 21:46 BUILD

-rwxrwxr-x 1 hbucher hbucher   867 May 15 15:42 cc_wrapper.sh

-rwxrwxr-x 1 hbucher hbucher  2869 May 15 23:21 CROSSTOOL

-rw-rw-r-- 1 hbucher hbucher 35141 May 15 23:17 LICENSE

-rw-rw-r-- 1 hbucher hbucher    80 May 15 23:17 README.md

-rw-rw-r-- 1 hbucher hbucher   111 May 15 15:39 WORKSPACE

But when I run the build I get this:

$ /home/hbucher/build-ts/bazel/output/bazel build --crosstool_top=//hbclang:toolchain tensorflow:libtensorflow_all.so

ERROR: no such package 'hbclang': BUILD file not found on package path.

I am very confused how the wrapping of all these files work. I went through the tutorial 

https://github.com/bazelbuild/bazel/wiki/Building-with-a-custom-toolchain

but that only made everything more confusing. 

Would you be able to lend me a hand here?

Thank you.

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discuss+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/20533346-7eac-41e6-a785-62660d75a116%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discuss+unsubscribe@googlegroups.com.

[Austin Schuh]

On Mon, May 15, 2017 at 9:34 PM Henrique Bucher <hbu...@gmail.com> wrote:

Thanks for your answers!

I added the section below to tensorflor/WORKSPACE 

new_local_repository(

  name="hbclang",

  path="/home/hbucher/BazelCustomToolchain",

  build_file = "BUILD",

)

where /home/hbucher/BazelCustomToolchain points to the git checkout of 

https://github.com/HFTrader/BazelCustomToolchain

Good

 

$ ls /home/hbucher/BazelCustomToolchain/ -l

-rwxrwxr-x 1 hbucher hbucher   896 May 15 21:46 BUILD

-rwxrwxr-x 1 hbucher hbucher   867 May 15 15:42 cc_wrapper.sh

-rwxrwxr-x 1 hbucher hbucher  2869 May 15 23:21 CROSSTOOL

-rw-rw-r-- 1 hbucher hbucher 35141 May 15 23:17 LICENSE

-rw-rw-r-- 1 hbucher hbucher    80 May 15 23:17 README.md

-rw-rw-r-- 1 hbucher hbucher   111 May 15 15:39 WORKSPACE

But when I run the build I get this:

$ /home/hbucher/build-ts/bazel/output/bazel build --crosstool_top=//hbclang:toolchain tensorflow:libtensorflow_all.so

ERROR: no such package 'hbclang': BUILD file not found on package path.

I am very confused how the wrapping of all these files work. I went through the tutorial 

https://github.com/bazelbuild/bazel/wiki/Building-with-a-custom-toolchain

but that only made everything more confusing. 

Would you be able to lend me a hand here?

Thank you.

Try  

$ /home/hbucher/build-ts/bazel/output/bazel build --crosstool_top=@hbclang//:toolchain tensorflow:libtensorflow_all.so

That will at least get you to the next error.  ;)

I was looking for the documentation to point you to for that, but I had to instead file https://github.com/bazelbuild/bazel/issues/3013

https://bazel.build/versions/master/docs/external.html talks about the syntax of the label, but not very directly.

Austin

[Henrique Bucher]

I ultimately gave up on this. It is impossible to get bazel to do anything that is not mainstream. Never seen such cryptic system. Throwing the towel and feeling miserable.

[Irina Iancu]

I'm adding Marcel, who knows a lot about crosstool, to take a look at this.

[Marcel Hlopko]

Hi Henrique, can you share a git repo with your toolchain (or what you have so far) and a hello world cc file? I'll take it from there.

--

-- 

Marcel Hlopko | Software Engineer | hlo...@google.com | 

Google Germany GmbH | Erika-Mann-Str. 33  | 80636 München | Germany | Geschäftsführer: Matthew Scott Sucherman, Paul Terence Manicle | Registergericht und -nummer: Hamburg, HRB 86891

[Henrique Bucher]

The link is already in the email. 

https://github.com/HFTrader/BazelCustomToolchain

[Marcel Hlopko]

Uploaded https://github.com/HFTrader/BazelCustomToolchain/pull/1. Let us know if it did or didn't help, or where you got stuck next. Also, consider asking on Stack Overflow, we monitor questions tagged "Bazel" in the same way we check bazel-discuss, but SO feels like is a better platform for building knowledge base than google groups.

All in all, we are very well aware how hard and frustrating writing Crosstool is and I'm sorry you had to suffer. This is one of our highest priorities on the C++ front so there is a hope it will get simpler and saner. In the meantime, do not hesitate to ask. Thanks!

[Henrique Bucher]

I will post there. Meanwhile, here is what I get now:

hbucher@vbox-ubuntu-server:~/build-ts/tensorflow-github$ /home/hbucher/build-ts/bazel/output/bazel build --crosstool_top=@hbclang//:toolchain tensorflow:libtensorflow_all.so                                                                                             .....................                                                                                                                                                                                                                                                                                                                                   ERROR: The crosstool_top you specified was resolved to '@hbclang//:toolchain', which does not contain a CROSSTOOL file. You can use a crosstool from the depot by specifying its label.

INFO: Elapsed time: 2.216s  

[Henrique Bucher]

I have appended these lines to my tensorflow/WORKSPACE

new_local_repository(                                                                                                                                                                                                                                                                                                                           name="hbclang",                                                                                                                                                                                                                                                                                                                                 path="/home/hbucher/BazelCustomToolchain",                                                                                                                                                                                                                                                                                     build_file = "/home/hbucher/BazelCustomToolchain/BUILD",                                                                                                                                                                                                                                                                 )                                                          

[Henrique Bucher]

Asked on stackoverflow  

http://stackoverflow.com/questions/44052877/compiling-tensorflow-with-a-custom-clang-libc-instead-of-stdlibc

On Wed, May 17, 2017 at 10:27 PM, Marcel Hlopko <hlo...@google.com> wrote:

[Marcel Hlopko]

Replied to the SO question.

[Henrique Bucher]

Seriously, your reply did not bring any new information.

I give up. Bazel is a piece of shit.

[Pedro Kiefer]

How rude you are. There's no need for that, just simply don't use it.

I use it to all my embedded projects, using different crosstools for arm-none, arm-linux, stm8, blackfin, etc. It works great, but I'll give you that the learning curve for writing your own CROSSTOOL and custom rules is a steep one.

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discuss+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CALNgcswDJCx4e_%3DHeaH-98s%2Bs26k9xnzAfSg3%2Bh1gkr6E-6qNQ%40mail.gmail.com.

[Henrique Bucher]

Rude? ROFL. Are we prioritizing feelings now? That might explain why this crap is so half baked.

We have a business and I am almost losing a client because of this tool. 

How about writing some software that works, well documented and without hardcoded paths everywhere?  

[Henrique Bucher]

Sorry it was indeed rude. 

It has been a mad week in here - I'm trying to get this startup off the ground.

Apologies.

[Henrique Bucher]

Solved

export INSTALL_DIR="$HOME/install"
export CC=$INSTALL_DIR/bin/clang
export CXX=$INSTALL_DIR/bin/clang++
export CXXFLAGS="-stdlib=libc++ -L$INSTALL_DIR/lib"
export LDFLAGS="-L$INSTALL_DIR/lib -lm -lrt"
export LD_LIBRARY_PATH="/usr/lib:/lib/x86_64-linux-gnu/:$INSTALL_DIR/lib"
git clone https://github.com/tensorflow/tensorflow.git tensorflow-github
cd tensorflow-github
mkdir build-tp && cd build-tmp
cmake ../tensorflow/contrib/cmake/
make -j4