Public Bazel Container launched

[Xin Gao]

Hi Bazel community,

The RBE toolchains team recently launched a public Bazel container (Ubuntu 16.04 based) in the Google Cloud Marketplace. The Bazel container includes the toolchain necessary to run C++, Java and Python Bazel builds. It includes a recent and well tested version of clang, OpenJDK8, OpenJDK10, Python 2.7, Python 3.6 and other basic tools such as Binutils, Patch, Git, Openssh client, Unzip, Wget, Zip.

To get started, you can check it out at Google Cloud Marketplace or pull the container with command: docker pull l.gcr.io/google/bazel:latest. The latest version of the container comes with Bazel 0.16.1.

The Bazel container makes it very easy to try out Bazel builds on any machine that supports Docker. It gives the same Bazel build environment across machines and users.

We include a default bazel.bazelrc file with sanitizer build configs located under /etc to provide out-of-box support for sanitizers (asan, tsan and msan). Check out doc: Getting started with Bazel Docker Container.

The Bazel container is built with Bazel, using rules_docker, base-images-docker and bazel-toolchains. If you want to build the container yourself, sources are available here.

Thanks,

RBE Toolchains Team

[Ed Schouten]

Hi Xin,

Op ma 10 sep. 2018 om 19:52 schreef 'Xin Gao' via bazel-discuss
<bazel-...@googlegroups.com>:

> The Bazel container makes it very easy to try out Bazel builds on any machine that supports Docker. It gives the same Bazel build environment across machines and users.

Would this also be a container image that we'd want to use as the
basis for remote builds? Phrased differently: for Bazel Buildbarn
(https://github.com/EdSchouten/bazel-buildbarn) I'm currently crafting
a container by hand-picking .deb files from Debian 9. Do the Bazel
authors suggest I use that container image instead?

Furthermore: is the project also interested in offering CROSSTOOL
files to go along with this container image? For Buildbarn I had to
derive one myself to be able to do builds against it from
non-Debian-9-based systems
(https://github.com/EdSchouten/bazel-buildbarn/blob/master/toolchain/CROSSTOOL).

Thanks for working on this.

--
Ed Schouten <e...@nuxi.nl>

[Ed Schouten]

Op ma 10 sep. 2018 om 22:57 schreef Ed Schouten <e...@nuxi.nl>:

> Furthermore: is the project also interested in offering CROSSTOOL
> files to go along with this container image?

Should have browsed before asking this. I suspect this is the
corresponding CROSSTOOL file:

https://github.com/bazelbuild/bazel-toolchains/blob/master/configs/ubuntu16_04_clang/1.1/bazel_0.16.1/default/CROSSTOOL

--
Ed Schouten <e...@nuxi.nl>

[Xin Gao]

Hi Ed,

Thanks for your interest! The Bazel container is more meant for the host container (where to launch Bazel) as opposed to execution container (the container that remote execution uses). But, we do publish two other containers for remote execution. rbe-debian8 and rbe-ubuntu1604. The have very similar tools and compilers installed as the Bazel container. And, as you found out, we also publish the configs to use with them! The configs for rbe-debian8 are here, and the configs for rbe-ubuntu1604 are here.

Thanks,

Xin

[Xin Gao]

The link for rbe-ubuntu1604 container should be this: https://console.cloud.google.com/marketplace/details/google/rbe-ubuntu16-04

[Yugui]

HI Xin,

That looks cool.It'd help me to make my CI environment more reliable.

I wonder how I compare it with  gcr.io/cloud-builders/bazel.

What advantage can I expect for the new image?  Can I expect that some compatibility between them?

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/CAKcgio6dCpeNuQ7rY2%3DEiq6adTG1Z-gaEw0qrfEpxJhJZKPQqg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Xin Gao]

Hi Yugui,

Thanks for your question. Comparing to gcr.io/cloud-builders/bazel, some differences are:

1. l.gcr.io/google/bazel uses rules_docker to build the container image, while gcr.io/cloud-builders/bazel use Dockerfile.

2. They have different cc compiler installed. l.gcr.io/google/bazel includes a Google-tested version of Clang with sanitizer support, while gcr.io/cloud-builders/bazel includes gcc.

3. gcr.io/cloud-builders/bazel currently doesn't have tags for Bazel version. But you can pull l.gcr.io/google/bazel with specific Bazel versions. Currently available ones are 0.15.2 and 0.16.1.

They have things in common as well, like they both based on l.gcr.io/google/ubuntu16_04.

Thanks,

Xin

On Mon, Sep 10, 2018 at 5:28 PM, Yugui <yu...@yugui.jp> wrote:

HI Xin,

That looks cool.It'd help me to make my CI environment more reliable.

I wonder how I compare it with  gcr.io/cloud-builders/bazel.

What advantage can I expect for the new image?  Can I expect that some compatibility between them?

On Tue, Sep 11, 2018 at 2:52 AM 'Xin Gao' via bazel-discuss <bazel-discuss@googlegroups.com> wrote:

Hi Bazel community,

The RBE toolchains team recently launched a public Bazel container (Ubuntu 16.04 based) in the Google Cloud Marketplace. The Bazel container includes the toolchain necessary to run C++, Java and Python Bazel builds. It includes a recent and well tested version of clang, OpenJDK8, OpenJDK10, Python 2.7, Python 3.6 and other basic tools such as Binutils, Patch, Git, Openssh client, Unzip, Wget, Zip.

To get started, you can check it out at Google Cloud Marketplace or pull the container with command: docker pull l.gcr.io/google/bazel:latest. The latest version of the container comes with Bazel 0.16.1.

The Bazel container makes it very easy to try out Bazel builds on any machine that supports Docker. It gives the same Bazel build environment across machines and users.

We include a default bazel.bazelrc file with sanitizer build configs located under /etc to provide out-of-box support for sanitizers (asan, tsan and msan). Check out doc: Getting started with Bazel Docker Container.

The Bazel container is built with Bazel, using rules_docker, base-images-docker and bazel-toolchains. If you want to build the container yourself, sources are available here.

Thanks,

RBE Toolchains Team

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discuss+unsubscribe@googlegroups.com.

[Yugui]

Awesome, thanks!

2018年9月11日(火) 7:07 Xin Gao <xin...@google.com>:

Hi Yugui,

Thanks for your question. Comparing to gcr.io/cloud-builders/bazel, some differences are:

1. l.gcr.io/google/bazel uses rules_docker to build the container image, while gcr.io/cloud-builders/bazel use Dockerfile.

2. They have different cc compiler installed. l.gcr.io/google/bazel includes a Google-tested version of Clang with sanitizer support, while gcr.io/cloud-builders/bazel includes gcc.

3. gcr.io/cloud-builders/bazel currently doesn't have tags for Bazel version. But you can pull l.gcr.io/google/bazel with specific Bazel versions. Currently available ones are 0.15.2 and 0.16.1.

They have things in common as well, like they both based on l.gcr.io/google/ubuntu16_04.

Thanks,

Xin

On Mon, Sep 10, 2018 at 5:28 PM, Yugui <yu...@yugui.jp> wrote:

HI Xin,

That looks cool.It'd help me to make my CI environment more reliable.

I wonder how I compare it with  gcr.io/cloud-builders/bazel.

What advantage can I expect for the new image?  Can I expect that some compatibility between them?

On Tue, Sep 11, 2018 at 2:52 AM 'Xin Gao' via bazel-discuss <bazel-...@googlegroups.com> wrote:

Hi Bazel community,

The RBE toolchains team recently launched a public Bazel container (Ubuntu 16.04 based) in the Google Cloud Marketplace. The Bazel container includes the toolchain necessary to run C++, Java and Python Bazel builds. It includes a recent and well tested version of clang, OpenJDK8, OpenJDK10, Python 2.7, Python 3.6 and other basic tools such as Binutils, Patch, Git, Openssh client, Unzip, Wget, Zip.

To get started, you can check it out at Google Cloud Marketplace or pull the container with command: docker pull l.gcr.io/google/bazel:latest. The latest version of the container comes with Bazel 0.16.1.

The Bazel container makes it very easy to try out Bazel builds on any machine that supports Docker. It gives the same Bazel build environment across machines and users.

We include a default bazel.bazelrc file with sanitizer build configs located under /etc to provide out-of-box support for sanitizers (asan, tsan and msan). Check out doc: Getting started with Bazel Docker Container.

The Bazel container is built with Bazel, using rules_docker, base-images-docker and bazel-toolchains. If you want to build the container yourself, sources are available here.

Thanks,

RBE Toolchains Team

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.

[Ed Schouten]

Hi Xin,

Op ma 10 sep. 2018 om 23:14 schreef Xin Gao <xin...@google.com>:

> But, we do publish two other containers for remote execution. rbe-debian8 and rbe-ubuntu1604.

That's pretty slick. I'll see if I can find some time to give those a
try with Buildbarn. Thanks! \o/

--
Ed Schouten <e...@nuxi.nl>

[Jakob Buchgraber]

Hi Xin,

very cool! Thanks for doing this! I have three questions:

1. Could we publish the documentation that's currently in the .pdf on Bazel's website? And maybe add some more (happy to help)?
2. Is building this container fully automated and could we trigger it via Bazel's release process?
3. You mentioned that this image and the RBE toolchains image  "have very similar tools and compilers installed".
   Could they have identical tools and compilers installed so that it's a safe to use the two images together for
   remote execution/caching? And could we document this use case on Bazel's website?

Thanks,

Jakob

Jakob Buchgraber

Software Engineer

Google Germany GmbH

Erika-Mann-Straße 33

80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado

Registergericht und -nummer: Hamburg, HRB 86891

Sitz der Gesellschaft: Hamburg

Diese E-Mail ist vertraulich. Falls sie diese fälschlicherweise erhalten haben sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail an die falsche Person gesendet wurde.

    

This e-mail is confidential. If you received this communication by mistake, please don't forward it to anyone else, please erase all copies and attachments, and please let me know that it has gone to the wrong person.

[Xin Gao]

On Tue, Sep 11, 2018 at 7:27 AM, Jakob Buchgraber <buc...@google.com> wrote:

Hi Xin,

very cool! Thanks for doing this! I have three questions:

1. Could we publish the documentation that's currently in the .pdf on Bazel's website? And maybe add some more (happy to help)?

Yea definitely. I'll work on it. We are also thinking of adding a short description and a link to the container in the Getting Started with Bazel page as well.

1. Is building this container fully automated and could we trigger it via Bazel's release process?

It's a mixed of automated and manual process at this moment. We are working towards a fully automated container release process (for all RBE containers). Yea, we would love to have it integrated with Bazel's release process in some kind. Let's talk about it more offline.

1. You mentioned that this image and the RBE toolchains image  "have very similar tools and compilers installed".
   Could they have identical tools and compilers installed so that it's a safe to use the two images together for
   remote execution/caching? And could we document this use case on Bazel's website?

Yes, we could potentially. We just need to be careful to make sure the compiler version matches in the two containers.

[Jakob Buchgraber]

On Tue, Sep 11, 2018 at 4:46 PM 'Xin Gao' via bazel-discuss <bazel-...@googlegroups.com> wrote:

Yes, we could potentially. We just need to be careful to make sure the compiler version matches in the two containers.

Wasn't the intention of releasing these two containers to provide users a compatible local and remote environment

to safely do remote execution?

[Xin Gao]

Umm. Not quite. Bazel container is kind of a standalone thing itself. It has similar tools as other RBE containers as we use the same layer targets in them but the versions of compilers can be different depending on when we release them. And I think the execution in the remote environment shouldn't necessarily be the same as the host environment except for issues related to @local_jdk for java iiuc? 

I think making them the same to support the use case that running Bazel with local execution and remote caching in the Bazel container, and using RBE ubuntu1604 container as the execution container for builds not necessarily started from the Bazel container can be something we can evaluate.

Thanks,
Xin

On Wed, Sep 12, 2018 at 3:53 AM, Jakob Buchgraber <buc...@google.com> wrote:

[Jakob Buchgraber]

+Nicolas Lopez 

For these two to be released in sync and with identical tool versions seems to enable a very important

use case to me (and that at little to no cost):

• Initially use the container on your CI system with remote caching and be able to then seamlessly
  go to remote execution.
• Do safe remote execution on your CI system. If your build works in the local container it also works
  on remote execution.

Best,

Jakob

On Thu, Sep 13, 2018 at 7:12 PM Xin Gao <xin...@google.com> wrote:

Umm. Not quite. Bazel container is kind of a standalone thing itself. It has similar tools as other RBE containers as we use the same layer targets in them but the versions of compilers can be different depending on when we release them. And I think the execution in the remote environment shouldn't necessarily be the same as the host environment except for issues related to @local_jdk for java iiuc? 

I think making them the same to support the use case that running Bazel with local execution and remote caching in the Bazel container, and using RBE ubuntu1604 container as the execution container for builds not necessarily started from the Bazel container can be something we can evaluate.

Thanks,
Xin

On Wed, Sep 12, 2018 at 3:53 AM, Jakob Buchgraber <buc...@google.com> wrote:

[emaildev...@gmail.com]

Hi I am trying to build binaries for x86_64 linux boxes from mac os . If i understand correctly i should be using this containers to run bazel do the cross compilation ?

[emaildev...@gmail.com]

Could you please elaborate the difference between image rbe-ubuntu1604 and gcr.io/google/bazel:latest ? in the context where everything runs locally ? What does host and remote execution container mean when i am trying to build for a specific target locally ?

[Xin Gao]

- "Hi I am trying to build binaries for x86_64 linux boxes from mac os . If i understand correctly i should be using this containers to run bazel do the cross compilation ? "

I think so, as long as Docker on Mac is hermetic. But we've never tried running this container on Mac, please let us know if you encounter any issues with that.

- "Could you please elaborate the difference between image  rbe-ubuntu1604 and gcr.io/google/bazel:latest ? in the context where everything runs locally ? What does host and remote execution container mean when i am trying to build for a specific target locally ?"

rbe-ubuntu1604 and gcr.io/google/bazel:latest are built for different purposes. rbe-ubuntu1604 is to be used as an remote execution environment. It only has certain toolchains installed (clang, java, python, etc) but does NOT have Bazel installed. In your use case (where everything runs locally), you don't need to worry about this rbe-ubuntu1604 container. You just need the gcr.io/google/bazel:latest to build your targets locally.

Thanks,

Xin

--
You received this message because you are subscribed to the Google Groups "bazel-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discuss+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/d780f680-9597-488f-970a-c0006fa0ba1f%40googlegroups.com.

[developer pp]

During compilation i get the following error with gcr.io/google/bazel:latest : this is using bazel version 0.19.0

ERROR: /root/.cache/bazel/_bazel_root/18fc53a52a033fe75f7d0542124ca8c0/external/com_github_google_glog/BUILD.bazel:3:1: C++ compilation of rule '@com_github_google_glog//:glog' failed (Exit 1) clang failed: error executing command /usr/local/bin/clang -U_FORTIFY_SOURCE -fstack-protector -Wall -B/usr/local/bin -B/usr/bin -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 85 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox

In file included from external/com_github_google_glog/src/vlog_is_on.cc:35:

In file included from external/com_github_google_glog/src/utilities.h:73:

external/com_github_google_glog/src/base/mutex.h:147:3: error: Need to implement mutex.h for your architecture, or #define NO_THREADS

# error Need to implement mutex.h for your architecture, or #define NO_THREADS

  ^

external/com_github_google_glog/src/base/mutex.h:188:3: error: unknown type name 'MutexType'

  MutexType mutex_;

  ^

In file included from external/com_github_google_glog/src/vlog_is_on.cc:35:

external/com_github_google_glog/src/utilities.h:137:1: error: unknown type name '_START_GOOGLE_NAMESPACE_'

_START_GOOGLE_NAMESPACE_

^

external/com_github_google_glog/src/utilities.h:139:1: error: expected unqualified-id

namespace glog_internal_namespace_ {

^

external/com_github_google_glog/src/utilities.h:154:1: error: unknown type name 'int64'; did you mean 'google::int64'?

Any ideas on how to resolve this ?

To unsubscribe from this group and stop receiving emails from it, send an email to bazel-discus...@googlegroups.com.

[Nicolas Lopez]

could you run with --verbose_failures and copu here a full error log. Also, could you provide instructions for repro (what project are you compiling and what command you are using to build?)

[Nicolas Lopez]

also, is this error only occuring with the 0.19.0 container, or does it reproduce with other ones?

[developer pp]

Yes this is with 0.19.0 and 0.18.0 , both i can reproduce the error.

In priorversion there are numerous other bugs that required me to upgrade bazel version as i founf those public github issues

DETAILED error :

ERROR: /root/.cache/bazel/_bazel_root/18fc53a52a033fe75f7d0542124ca8c0/external/com_github_google_glog/BUILD.bazel:3:1: C++ compilation of rule '@com_github_google_glog//:glog' failed (Exit 1) clang failed: error executing command

  (cd /root/.cache/bazel/_bazel_root/18fc53a52a033fe75f7d0542124ca8c0/execroot/search && \

  exec env - \

    LD_LIBRARY_PATH=/usr/local/lib \

    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \

    PWD=/proc/self/cwd \

  /usr/local/bin/clang -U_FORTIFY_SOURCE -fstack-protector -Wall -B/usr/local/bin -B/usr/bin -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections '-std=c++0x' -MD -MF bazel-out/k8-opt/bin/external/com_github_google_glog/_objs/glog/logging.d '-frandom-seed=bazel-out/k8-opt/bin/external/com_github_google_glog/_objs/glog/logging.o' -iquote external/com_github_google_glog -iquote bazel-out/k8-opt/genfiles/external/com_github_google_glog -iquote bazel-out/k8-opt/bin/external/com_github_google_glog -iquote external/com_github_gflags_gflags -iquote bazel-out/k8-opt/genfiles/external/com_github_gflags_gflags -iquote bazel-out/k8-opt/bin/external/com_github_gflags_gflags -iquote external/bazel_tools -iquote bazel-out/k8-opt/genfiles/external/bazel_tools -iquote bazel-out/k8-opt/bin/external/bazel_tools -isystem external/com_github_google_glog -isystem bazel-out/k8-opt/genfiles/external/com_github_google_glog -isystem bazel-out/k8-opt/bin/external/com_github_google_glog -isystem external/com_github_google_glog/src -isystem bazel-out/k8-opt/genfiles/external/com_github_google_glog/src -isystem bazel-out/k8-opt/bin/external/com_github_google_glog/src -isystem external/com_github_gflags_gflags/include -isystem bazel-out/k8-opt/genfiles/external/com_github_gflags_gflags/include -isystem bazel-out/k8-opt/bin/external/com_github_gflags_gflags/include '-D_START_GOOGLE_NAMESPACE_=namespace google {' '-D_END_GOOGLE_NAMESPACE_=}' '-DGOOGLE_NAMESPACE=google' '-DGOOGLE_GLOG_DLL_DECL=' -DHAVE_DLADDR -DHAVE_SNPRINTF -DHAVE_DLFCN_H -DHAVE_FCNTL -DHAVE_GLOB_H -DHAVE_INTTYPES_H -DHAVE_LIBPTHREAD -DHAVE_LIB_GFLAGS -DHAVE_MEMORY_H -DHAVE_NAMESPACES -DHAVE_PREAD -DHAVE_PTHREAD -DHAVE_PWD_H -DHAVE_PWRITE -DHAVE_RWLOCK -DHAVE_SIGACTION -DHAVE_SIGALTSTACK -DHAVE_STDINT_H -DHAVE_STRING_H -DHAVE_SYS_SYSCALL_H -DHAVE_SYS_TIME_H -DHAVE_SYS_TYPES_H -DHAVE_SYS_UCONTEXT_H -DHAVE_SYS_UTSNAME_H -DHAVE_UNISTD_H -DHAVE_USING_OPERATOR -DHAVE_HAVE___ATTRIBUTE___ -DHAVE_HAVE___BUILTIN_EXPECT -D_GNU_SOURCE -Iexternal/glog_repo/src -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c external/com_github_google_glog/src/logging.cc -o bazel-out/k8-opt/bin/external/com_github_google_glog/_objs/glog/logging.o)

Use --sandbox_debug to see verbose messages from the sandbox: clang failed: error executing command

  (cd /root/.cache/bazel/_bazel_root/18fc53a52a033fe75f7d0542124ca8c0/execroot/search && \

  exec env - \

    LD_LIBRARY_PATH=/usr/local/lib \

    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \

    PWD=/proc/self/cwd \

  /usr/local/bin/clang -U_FORTIFY_SOURCE -fstack-protector -Wall -B/usr/local/bin -B/usr/bin -fcolor-diagnostics -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections '-std=c++0x' -MD -MF bazel-out/k8-opt/bin/external/com_github_google_glog/_objs/glog/logging.d '-frandom-seed=bazel-out/k8-opt/bin/external/com_github_google_glog/_objs/glog/logging.o' -iquote external/com_github_google_glog -iquote bazel-out/k8-opt/genfiles/external/com_github_google_glog -iquote bazel-out/k8-opt/bin/external/com_github_google_glog -iquote external/com_github_gflags_gflags -iquote bazel-out/k8-opt/genfiles/external/com_github_gflags_gflags -iquote bazel-out/k8-opt/bin/external/com_github_gflags_gflags -iquote external/bazel_tools -iquote bazel-out/k8-opt/genfiles/external/bazel_tools -iquote bazel-out/k8-opt/bin/external/bazel_tools -isystem external/com_github_google_glog -isystem bazel-out/k8-opt/genfiles/external/com_github_google_glog -isystem bazel-out/k8-opt/bin/external/com_github_google_glog -isystem external/com_github_google_glog/src -isystem bazel-out/k8-opt/genfiles/external/com_github_google_glog/src -isystem bazel-out/k8-opt/bin/external/com_github_google_glog/src -isystem external/com_github_gflags_gflags/include -isystem bazel-out/k8-opt/genfiles/external/com_github_gflags_gflags/include -isystem bazel-out/k8-opt/bin/external/com_github_gflags_gflags/include '-D_START_GOOGLE_NAMESPACE_=namespace google {' '-D_END_GOOGLE_NAMESPACE_=}' '-DGOOGLE_NAMESPACE=google' '-DGOOGLE_GLOG_DLL_DECL=' -DHAVE_DLADDR -DHAVE_SNPRINTF -DHAVE_DLFCN_H -DHAVE_FCNTL -DHAVE_GLOB_H -DHAVE_INTTYPES_H -DHAVE_LIBPTHREAD -DHAVE_LIB_GFLAGS -DHAVE_MEMORY_H -DHAVE_NAMESPACES -DHAVE_PREAD -DHAVE_PTHREAD -DHAVE_PWD_H -DHAVE_PWRITE -DHAVE_RWLOCK -DHAVE_SIGACTION -DHAVE_SIGALTSTACK -DHAVE_STDINT_H -DHAVE_STRING_H -DHAVE_SYS_SYSCALL_H -DHAVE_SYS_TIME_H -DHAVE_SYS_TYPES_H -DHAVE_SYS_UCONTEXT_H -DHAVE_SYS_UTSNAME_H -DHAVE_UNISTD_H -DHAVE_USING_OPERATOR -DHAVE_HAVE___ATTRIBUTE___ -DHAVE_HAVE___BUILTIN_EXPECT -D_GNU_SOURCE -Iexternal/glog_repo/src -no-canonical-prefixes -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c external/com_github_google_glog/src/logging.cc -o bazel-out/k8-opt/bin/external/com_github_google_glog/_objs/glog/logging.o)

Use --sandbox_debug to see verbose messages from the sandbox

In file included from external/com_github_google_glog/src/logging.cc:32:

In file included from external/com_github_google_glog/src/utilities.h:73:

external/com_github_google_glog/src/base/mutex.h:147:3: error: Need to implement mutex.h for your architecture, or #define NO_THREADS

# error Need to implement mutex.h for your architecture, or #define NO_THREADS

  ^

external/com_github_google_glog/src/base/mutex.h:188:3: error: unknown type name 'MutexType'

  MutexType mutex_;

  ^

In file included from external/com_github_google_glog/src/logging.cc:32:

external/com_github_google_glog/src/utilities.h:137:1: error: unknown type name '_START_GOOGLE_NAMESPACE_'

_START_GOOGLE_NAMESPACE_

^

external/com_github_google_glog/src/utilities.h:139:1: error: expected unqualified-id

namespace glog_internal_namespace_ {

^

external/com_github_google_glog/src/utilities.h:154:1: error: unknown type name 'int64'; did you mean 'google::int64'?

int64 CycleClock_Now();

^~~~~

google::int64

bazel-out/k8-opt/genfiles/external/com_github_google_glog/glog/logging.h:94:17: note: 'google::int64' declared here

typedef int64_t int64;

                ^

In file included from external/com_github_google_glog/src/logging.cc:32:

external/com_github_google_glog/src/utilities.h:156:1: error: unknown type name 'int64'; did you mean 'google::int64'?

int64 UsecToCycles(int64 usec);

^~~~~

google::int64

bazel-out/k8-opt/genfiles/external/com_github_google_glog/glog/logging.h:94:17: note: 'google::int64' declared here

typedef int64_t int64;

                ^

In file included from external/com_github_google_glog/src/logging.cc:32:

external/com_github_google_glog/src/utilities.h:156:20: error: unknown type name 'int64'; did you mean 'google::int64'?

int64 UsecToCycles(int64 usec);

                   ^~~~~

                   google::int64

bazel-out/k8-opt/genfiles/external/com_github_google_glog/glog/logging.h:94:17: note: 'google::int64' declared here

typedef int64_t int64;

                ^

In file included from external/com_github_google_glog/src/logging.cc:32:

external/com_github_google_glog/src/utilities.h:161:1: error: unknown type name 'int32'; did you mean 'google::int32'?

int32 GetMainThreadPid();

^~~~~

google::int32

bazel-out/k8-opt/genfiles/external/com_github_google_glog/glog/logging.h:92:17: note: 'google::int32' declared here

typedef int32_t int32;

                ^

In file included from external/com_github_google_glog/src/logging.cc:32:

external/com_github_google_glog/src/utilities.h:222:1: error: unknown type name '_END_GOOGLE_NAMESPACE_'

_END_GOOGLE_NAMESPACE_

^

external/com_github_google_glog/src/utilities.h:224:1: error: expected unqualified-id

using namespace GOOGLE_NAMESPACE::glog_internal_namespace_;

^

external/com_github_google_glog/src/logging.cc:117:1: error: unknown type name '_START_GOOGLE_NAMESPACE_'

_START_GOOGLE_NAMESPACE_

^

external/com_github_google_glog/src/logging.cc:118:1: error: expected unqualified-id

namespace logging {

^

external/com_github_google_glog/src/logging.cc:119:14: error: unknown type name 'int64'; did you mean 'google::int64'?

static const int64 kPageSize = getpagesize();

             ^~~~~

             google::int64

bazel-out/k8-opt/genfiles/external/com_github_google_glog/glog/logging.h:94:17: note: 'google::int64' declared here

typedef int64_t int64;

                ^

external/com_github_google_glog/src/logging.cc:121:1: error: unknown type name '_END_GOOGLE_NAMESPACE_'

_END_GOOGLE_NAMESPACE_

^

external/com_github_google_glog/src/logging.cc:129:1: error: expected unqualified-id

DEFINE_int32(stderrthreshold,

^

external/com_github_google_glog/src/base/commandlineflags.h:86:3: note: expanded from macro 'DEFINE_int32'

  DEFINE_VARIABLE(GOOGLE_NAMESPACE::int32, I, name, value, meaning, int32)

  ^

external/com_github_google_glog/src/base/commandlineflags.h:70:3: note: expanded from macro 'DEFINE_VARIABLE'

  namespace fL##shorttype {                                             \

  ^

external/com_github_google_glog/src/logging.cc:129:1: error: use of undeclared identifier 'GOOGLE_NAMESPACE'

external/com_github_google_glog/src/base/commandlineflags.h:86:19: note: expanded from macro 'DEFINE_int32'

  DEFINE_VARIABLE(GOOGLE_NAMESPACE::int32, I, name, value, meaning, int32)

                  ^

external/com_github_google_glog/src/logging.cc:130:14: error: use of undeclared identifier 'GOOGLE_NAMESPACE'

             GOOGLE_NAMESPACE::GLOG_ERROR,

             ^

external/com_github_google_glog/src/logging.cc:139:1: error: use of undeclared identifier 'GOOGLE_NAMESPACE'

GLOG_DEFINE_int32(minloglevel, 0, "Messages logged at a lower level than this don't "

^

external/com_github_google_glog/src/base/commandlineflags.h:116:3: note: expanded from macro 'GLOG_DEFINE_int32'

  DEFINE_int32(name, EnvToInt("GLOG_" #name, value), meaning)

  ^

external/com_github_google_glog/src/base/commandlineflags.h:86:19: note: expanded from macro 'DEFINE_int32'

  DEFINE_VARIABLE(GOOGLE_NAMESPACE::int32, I, name, value, meaning, int32)

                  ^

external/com_github_google_glog/src/logging.cc:141:1: error: use of undeclared identifier 'GOOGLE_NAMESPACE'

GLOG_DEFINE_int32(logbuflevel, 0,

^

external/com_github_google_glog/src/base/commandlineflags.h:116:3: note: expanded from macro 'GLOG_DEFINE_int32'

  DEFINE_int32(name, EnvToInt("GLOG_" #name, value), meaning)

  ^

external/com_github_google_glog/src/base/commandlineflags.h:86:19: note: expanded from macro 'DEFINE_int32'

  DEFINE_VARIABLE(GOOGLE_NAMESPACE::int32, I, name, value, meaning, int32)

On similar note : I do have a custom docker file where i have ubuntu 16 installed and some other libraries installed. (which i have been using to build my image)

Primarily i use gcc (5.4) for compilation and not clang . 

Iin my custom docker with bazel 0.19.0 I am facing another issue : I have 2 gce boxes and 1 mac 

If i create gce instance using my docker image as the base image , everything runs fine and builds. 

But if i use another GCE box that had separate base image , but i pull my docker image and start running build within this docker image , the build fails with a linking error. 

The only difference between the 2 GCE is that kernel version are different , since i bring up one GCE using my custom docker image and in the other GCE i have a default debian , and after the startup i pull the custom docker image and start bazel build.

As i understand the build should be reproducible and uniform as long as i am withing the docker world , but why i am observing this behaviour , can different kernel versions some how affects the build and linkage process ?

On Thu, Nov 8, 2018 at 2:45 PM 'Nicolas Lopez' via bazel-discuss <bazel-...@googlegroups.com> wrote:

also, is this error only occuring with the 0.19.0 container, or does it reproduce with other ones?

--
You received this message because you are subscribed to a topic in the Google Groups "bazel-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/bazel-discuss/fKEdrrCUC0s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to bazel-discus...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bazel-discuss/c61f8829-567c-4f54-b0a0-a57b8456b81c%40googlegroups.com.

[Nicolas Lopez]

Thanks for the additional clarifications, se responses inline

These look like legitimate clang build failures to me.

 

On similar note : I do have a custom docker file where i have ubuntu 16 installed and some other libraries installed. (which i have been using to build my image)

Primarily i use gcc (5.4) for compilation and not clang . 

Clang and gcc do have differences. iiuc (i'm no expert), clang can catch more errors / issues in your code

 

Iin my custom docker with bazel 0.19.0 I am facing another issue : I have 2 gce boxes and 1 mac 

If i create gce instance using my docker image as the base image , everything runs fine and builds. 

But if i use another GCE box that had separate base image , but i pull my docker image and start running build within this docker image , the build fails with a linking error. 

The only difference between the 2 GCE is that kernel version are different , since i bring up one GCE using my custom docker image and in the other GCE i have a default debian , and after the startup i pull the custom docker image and start bazel build.

the kernel version can affect the build, we've seen it in some rare cases, but it can happen (it mostly, iiuc, depends on the kernel features that your code expects there to be present)

 

As i understand the build should be reproducible and uniform as long as i am withing the docker world , but why i am observing this behaviour , can different kernel versions some how affects the build and linkage process ?

unfortunately, i think it can, but in practice we've only seen this in very rare cases. 

[developer pp]

So , how should i proceed , should i open a github issue for this  ?

And what would be recommended way to solve the linker issue if kernels are different , any ideas , since i am not a build/bazel expert .

[Nicolas Lopez]

On Thu, Nov 8, 2018 at 7:13 PM developer pp <emaildev...@gmail.com> wrote:

So , how should i proceed , should i open a github issue for this  ?

 

I'm not sure how you should proceed, the owners of the code that is failing might be able to provide more guidance? There is no issue (afaict) with the bazel container to resolve here.

And what would be recommended way to solve the linker issue if kernels are different , any ideas , since i am not a build/bazel expert .

Sorry, do not know enough about your setup, or the code you're trying to test to provide further advice, if the kernel versions are indeed to blame, the owners of the code that has the specific kernel requirements might be able to help. In general, this looks like issues that you need to resolve with the code owners (i.e., which platforms they support and how to build their code and in which containers)

[developer pp]

I am bit confused when you say "owner of the code will provide more details" . The bazel container fails to build some binaries/image, whereas the code is fine and can be built using gcc and other custom images. Are you saying the errors caught by clang are legit errors and should also fail with gcc , but some how gcc is not as efficient and doesn't catch all errors ??

This seems more like a bazel or crosstool chain error to me rather than code related error. In fact on looking at the code the error doesnt even make sense to me that bazel-clang is throwing , it seems like some config.h is being included that should not have been.

When you say owners of the code you mean : which piece of code you are referring to  ?  Because i am the owner of the code that i am compiling and to me it seems more like bazel container failing to build the code. 

[rodr...@google.com]

Could you provide us precise instructions on how to reproduce the problem? I'm the maintainer of the Bazel build files in https://github.com/google/glog, so I might be able to help.

If I run:

docker run -it --entrypoint bash l.gcr.io/google/bazel:latest
git clone https://github.com/google/glog
cd glog
bazel build //...

it works fine.

However, it appears that you're building a workspace that includes glog as an external dependency. Is it open-source code that you can point us at?

One possible cause is using older gflags code, as this had issues with "leaking" config.h. In particular, glog is tested against gflags@77592648. gflags' latest release (v2.2.1) is quite old and doesn't include fixes to the Bazel build files.

Best,
Rodrigo

[Nicolas Lopez]

On Fri, Nov 9, 2018 at 1:48 AM developer pp <emaildev...@gmail.com> wrote:

I am bit confused when you say "owner of the code will provide more details" . The bazel container fails to build some binaries/image, whereas the code is fine and can be built using gcc and other custom images. Are you saying the errors caught by clang are legit errors and should also fail with gcc , but some how gcc is not as efficient and doesn't catch all errors ??

Yes that is exactly what I'm saying.

 

This seems more like a bazel or crosstool chain error to me rather than code related error.

What makes you assume that? From the error log (all that I have) this seems like a legit clang error that you (or whomever wrote the failing code) should be able to resolve. If we do find out that its a bazel or crosstool error I'd be happy to help you fix.  

 

In fact on looking at the code the error doesnt even make sense to me that bazel-clang is throwing , it seems like some config.h is being included that should not have been.

When you say owners of the code you mean : which piece of code you are referring to  ?  Because i am the owner of the code that i am compiling and to me it seems more like bazel container failing to build the code. 

It seems to me you are compiling a dependency of your code, perhaps try to find advice from them?

[Rodrigo Queiro]

FYI, gflags 2.2.2 was just released, so if you're having problems with gflags 2.2.1 and glog you should update to 2.2.2.

-- 

Google Germany GmbH | Erika-Mann-Strasse | 80636 Muenchen | Germany